1
|
Nowroozi A, Salehi MA, Shobeiri P, Agahi S, Momtazmanesh S, Kaviani P, Kalra MK. Artificial intelligence diagnostic accuracy in fracture detection from plain radiographs and comparing it with clinicians: a systematic review and meta-analysis. Clin Radiol 2024; 79:579-588. [PMID: 38772766 DOI: 10.1016/j.crad.2024.04.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2024] [Revised: 04/09/2024] [Accepted: 04/15/2024] [Indexed: 05/23/2024]
Abstract
PURPOSE Fracture detection is one of the most commonly used and studied aspects of artificial intelligence (AI) in medicine. In this systematic review and meta-analysis, we aimed to summarize available literature and data regarding AI performance in fracture detection on plain radiographs and various factors affecting it. METHODS We systematically reviewed studies evaluating AI algorithms in detecting bone fractures in plain radiographs, combined their performance using meta-analysis (a bivariate regression approach), and compared it with that of clinicians. We also analyzed the factors potentially affecting algorithm performance using meta-regression. RESULTS Our analysis included 100 studies. In 83 studies with confusion matrices, AI algorithms showed a sensitivity of 91.43% and a specificity of 92.12% (Area under the summary receiver operator curve = 0.968). After adjustment and false discovery rate correction, tibia/fibula (excluding ankle) fractures were associated with higher (7.0%, p=0.004) AI sensitivity, while more recent publications (5.5%, p=0.003) and Xception architecture (6.6%, p<0.001) were associated with higher specificity. Clinicians and AI showed similar specificity in fracture identification, although AI leaned to higher sensitivity (7.6%, p=0.07). Radiologists, on the other hand, were more specific than AI overall and in several subgroups, and more sensitive to hip fractures before FDR correction. CONCLUSIONS Currently available AI aids could result in a significant improvement in care where radiologists are not readily available. Moreover, identifying factors affecting algorithm performance could guide AI development teams in their process of optimizing their products.
Collapse
Affiliation(s)
- A Nowroozi
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - M A Salehi
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - P Shobeiri
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - S Agahi
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - S Momtazmanesh
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - P Kaviani
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA
| | - M K Kalra
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA.
| |
Collapse
|
2
|
Kim H, Kim K, Oh SJ, Lee S, Woo JH, Kim JH, Cha YK, Kim K, Chung MJ. AI-assisted Analysis to Facilitate Detection of Humeral Lesions on Chest Radiographs. Radiol Artif Intell 2024; 6:e230094. [PMID: 38446041 PMCID: PMC11140509 DOI: 10.1148/ryai.230094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2023] [Revised: 01/10/2024] [Accepted: 02/15/2024] [Indexed: 03/07/2024]
Abstract
Purpose To develop an artificial intelligence (AI) system for humeral tumor detection on chest radiographs (CRs) and evaluate the impact on reader performance. Materials and Methods In this retrospective study, 14 709 CRs (January 2000 to December 2021) were collected from 13 468 patients, including CT-proven normal (n = 13 116) and humeral tumor (n = 1593) cases. The data were divided into training and test groups. A novel training method called false-positive activation area reduction (FPAR) was introduced to enhance the diagnostic performance by focusing on the humeral region. The AI program and 10 radiologists were assessed using holdout test set 1, wherein the radiologists were tested twice (with and without AI test results). The performance of the AI system was evaluated using holdout test set 2, comprising 10 497 normal images. Receiver operating characteristic analyses were conducted for evaluating model performance. Results FPAR application in the AI program improved its performance compared with a conventional model based on the area under the receiver operating characteristic curve (0.87 vs 0.82, P = .04). The proposed AI system also demonstrated improved tumor localization accuracy (80% vs 57%, P < .001). In holdout test set 2, the proposed AI system exhibited a false-positive rate of 2%. AI assistance improved the radiologists' sensitivity, specificity, and accuracy by 8.9%, 1.2%, and 3.5%, respectively (P < .05 for all). Conclusion The proposed AI tool incorporating FPAR improved humeral tumor detection on CRs and reduced false-positive results in tumor visualization. It may serve as a supportive diagnostic tool to alert radiologists about humeral abnormalities. Keywords: Artificial Intelligence, Conventional Radiography, Humerus, Machine Learning, Shoulder, Tumor Supplemental material is available for this article. © RSNA, 2024.
Collapse
Affiliation(s)
- Harim Kim
- From the Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-Ro, Gangnam-Gu, Seoul 06351, South Korea (H.K., J.H.W., J.H.K., Y.K.C., M.J.C.); Medical AI Research Center, Samsung Medical Center, Seoul, South Korea (Kyungsu Kim, M.J.C.); Department of Data Convergence and Future Medicine, Sungkyunkwan University School of Medicine, Seoul, South Korea (Kyungsu Kim, Kyunga Kim, M.J.C.); and Department of Health Sciences and Technology (S.J.O.) and Department of Digital Health (S.L., Kyunga Kim), Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, South Korea
| | - Kyungsu Kim
- From the Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-Ro, Gangnam-Gu, Seoul 06351, South Korea (H.K., J.H.W., J.H.K., Y.K.C., M.J.C.); Medical AI Research Center, Samsung Medical Center, Seoul, South Korea (Kyungsu Kim, M.J.C.); Department of Data Convergence and Future Medicine, Sungkyunkwan University School of Medicine, Seoul, South Korea (Kyungsu Kim, Kyunga Kim, M.J.C.); and Department of Health Sciences and Technology (S.J.O.) and Department of Digital Health (S.L., Kyunga Kim), Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, South Korea
| | - Seong Je Oh
- From the Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-Ro, Gangnam-Gu, Seoul 06351, South Korea (H.K., J.H.W., J.H.K., Y.K.C., M.J.C.); Medical AI Research Center, Samsung Medical Center, Seoul, South Korea (Kyungsu Kim, M.J.C.); Department of Data Convergence and Future Medicine, Sungkyunkwan University School of Medicine, Seoul, South Korea (Kyungsu Kim, Kyunga Kim, M.J.C.); and Department of Health Sciences and Technology (S.J.O.) and Department of Digital Health (S.L., Kyunga Kim), Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, South Korea
| | - Sungjoo Lee
- From the Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-Ro, Gangnam-Gu, Seoul 06351, South Korea (H.K., J.H.W., J.H.K., Y.K.C., M.J.C.); Medical AI Research Center, Samsung Medical Center, Seoul, South Korea (Kyungsu Kim, M.J.C.); Department of Data Convergence and Future Medicine, Sungkyunkwan University School of Medicine, Seoul, South Korea (Kyungsu Kim, Kyunga Kim, M.J.C.); and Department of Health Sciences and Technology (S.J.O.) and Department of Digital Health (S.L., Kyunga Kim), Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, South Korea
| | - Jung Han Woo
- From the Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-Ro, Gangnam-Gu, Seoul 06351, South Korea (H.K., J.H.W., J.H.K., Y.K.C., M.J.C.); Medical AI Research Center, Samsung Medical Center, Seoul, South Korea (Kyungsu Kim, M.J.C.); Department of Data Convergence and Future Medicine, Sungkyunkwan University School of Medicine, Seoul, South Korea (Kyungsu Kim, Kyunga Kim, M.J.C.); and Department of Health Sciences and Technology (S.J.O.) and Department of Digital Health (S.L., Kyunga Kim), Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, South Korea
| | - Jong Hee Kim
- From the Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-Ro, Gangnam-Gu, Seoul 06351, South Korea (H.K., J.H.W., J.H.K., Y.K.C., M.J.C.); Medical AI Research Center, Samsung Medical Center, Seoul, South Korea (Kyungsu Kim, M.J.C.); Department of Data Convergence and Future Medicine, Sungkyunkwan University School of Medicine, Seoul, South Korea (Kyungsu Kim, Kyunga Kim, M.J.C.); and Department of Health Sciences and Technology (S.J.O.) and Department of Digital Health (S.L., Kyunga Kim), Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, South Korea
| | - Yoon Ki Cha
- From the Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-Ro, Gangnam-Gu, Seoul 06351, South Korea (H.K., J.H.W., J.H.K., Y.K.C., M.J.C.); Medical AI Research Center, Samsung Medical Center, Seoul, South Korea (Kyungsu Kim, M.J.C.); Department of Data Convergence and Future Medicine, Sungkyunkwan University School of Medicine, Seoul, South Korea (Kyungsu Kim, Kyunga Kim, M.J.C.); and Department of Health Sciences and Technology (S.J.O.) and Department of Digital Health (S.L., Kyunga Kim), Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, South Korea
| | - Kyunga Kim
- From the Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-Ro, Gangnam-Gu, Seoul 06351, South Korea (H.K., J.H.W., J.H.K., Y.K.C., M.J.C.); Medical AI Research Center, Samsung Medical Center, Seoul, South Korea (Kyungsu Kim, M.J.C.); Department of Data Convergence and Future Medicine, Sungkyunkwan University School of Medicine, Seoul, South Korea (Kyungsu Kim, Kyunga Kim, M.J.C.); and Department of Health Sciences and Technology (S.J.O.) and Department of Digital Health (S.L., Kyunga Kim), Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, South Korea
| | - Myung Jin Chung
- From the Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-Ro, Gangnam-Gu, Seoul 06351, South Korea (H.K., J.H.W., J.H.K., Y.K.C., M.J.C.); Medical AI Research Center, Samsung Medical Center, Seoul, South Korea (Kyungsu Kim, M.J.C.); Department of Data Convergence and Future Medicine, Sungkyunkwan University School of Medicine, Seoul, South Korea (Kyungsu Kim, Kyunga Kim, M.J.C.); and Department of Health Sciences and Technology (S.J.O.) and Department of Digital Health (S.L., Kyunga Kim), Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, South Korea
| |
Collapse
|
3
|
Jacques T, Cardot N, Ventre J, Demondion X, Cotten A. Commercially-available AI algorithm improves radiologists' sensitivity for wrist and hand fracture detection on X-ray, compared to a CT-based ground truth. Eur Radiol 2024; 34:2885-2894. [PMID: 37919408 DOI: 10.1007/s00330-023-10380-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Revised: 09/25/2023] [Accepted: 09/27/2023] [Indexed: 11/04/2023]
Abstract
OBJECTIVES Algorithms for fracture detection are spreading in clinical practice, but the use of X-ray-only ground truth can induce bias in their evaluation. This study assessed radiologists' performances to detect wrist and hand fractures on radiographs, using a commercially-available algorithm, compared to a computerized tomography (CT) ground truth. METHODS Post-traumatic hand and wrist CT and concomitant X-ray examinations were retrospectively gathered. Radiographs were labeled based on CT findings. The dataset was composed of 296 consecutive cases: 118 normal (39.9%), 178 pathological (60.1%) with a total of 267 fractures visible in CT. Twenty-three radiologists with various levels of experience reviewed all radiographs without AI, then using it, blinded towards CT results. RESULTS Using AI improved radiologists' sensitivity (Se, 0.658 to 0.703, p < 0.0001) and negative predictive value (NPV, 0.585 to 0.618, p < 0.0001), without affecting their specificity (Sp, 0.885 vs 0.891, p = 0.91) or positive predictive value (PPV, 0.887 vs 0.899, p = 0.08). On the radiographic dataset, based on the CT ground truth, stand-alone AI performances were 0.771 (Se), 0.898 (Sp), 0.684 (NPV), 0.915 (PPV), and 0.764 (AUROC) which were lower than previously reported, suggesting a potential underestimation of the number of missed fractures in the AI literature. CONCLUSIONS AI enabled radiologists to improve their sensitivity and negative predictive value for wrist and hand fracture detection on radiographs, without affecting their specificity or positive predictive value, compared to a CT-based ground truth. Using CT as gold standard for X-ray labels is innovative, leading to algorithm performance poorer than reported elsewhere, but probably closer to clinical reality. CLINICAL RELEVANCE STATEMENT Using an AI algorithm significantly improved radiologists' sensitivity and negative predictive value in detecting wrist and hand fractures on radiographs, with ground truth labels based on CT findings. KEY POINTS • Using CT as a ground truth for labeling X-rays is new in AI literature, and led to algorithm performance significantly poorer than reported elsewhere (AUROC: 0.764), but probably closer to clinical reality. • AI enabled radiologists to significantly improve their sensitivity (+ 4.5%) and negative predictive value (+ 3.3%) for the detection of wrist and hand fractures on X-rays. • There was no significant change in terms of specificity or positive predictive value.
Collapse
Affiliation(s)
- Thibaut Jacques
- Department of Musculoskeletal Radiology, Lille University Hospital, Rue du Professeur Emile Laine, 59000, Lille, France.
- IRIS Radiology - Clinique Lille Sud, SOS Hands and Fingers, 96 Rue Gustave Delory, 59810, Lesquin, France.
| | - Nicolas Cardot
- Department of Musculoskeletal Radiology, Lille University Hospital, Rue du Professeur Emile Laine, 59000, Lille, France
| | | | - Xavier Demondion
- Department of Musculoskeletal Radiology, Lille University Hospital, Rue du Professeur Emile Laine, 59000, Lille, France
- Lille University School of Medicine, 59000, Lille, France
| | - Anne Cotten
- Department of Musculoskeletal Radiology, Lille University Hospital, Rue du Professeur Emile Laine, 59000, Lille, France
- Lille University School of Medicine, 59000, Lille, France
- MABLab - Marrow Adiposity and Bone Lab ULR4490, University of Lille, 59000, Lille, France
| |
Collapse
|
4
|
Cheng CT, Ooyang CH, Kang SC, Liao CH. Applications of Deep Learning in Trauma Radiology: A Narrative Review. Biomed J 2024:100743. [PMID: 38679199 DOI: 10.1016/j.bj.2024.100743] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 03/26/2024] [Accepted: 04/24/2024] [Indexed: 05/01/2024] Open
Abstract
Diagnostic imaging is essential in modern trauma care for initial evaluation and identifying injuries requiring intervention. Deep learning (DL) has become mainstream in medical image analysis and has shown promising efficacy for classification, segmentation, and lesion detection. This narrative review provides the fundamental concepts for developing DL algorithms in trauma imaging and presents an overview of current progress in each modality. DL has been applied to detect free fluid on Focused Assessment with Sonography for Trauma (FAST), traumatic findings on chest and pelvic X-rays, and computed tomography (CT) scans, identify intracranial hemorrhage on head CT, detect vertebral fractures, and identify injuries to organs like the spleen, liver, and lungs on abdominal and chest CT. Future directions involve expanding dataset size and diversity through federated learning, enhancing model explainability and transparency to build clinician trust, and integrating multimodal data to provide more meaningful insights into traumatic injuries. Though some commercial artificial intelligence products are Food and Drug Administration-approved for clinical use in the trauma field, adoption remains limited, highlighting the need for multi-disciplinary teams to engineer practical, real-world solutions. Overall, DL shows immense potential to improve the efficiency and accuracy of trauma imaging, but thoughtful development and validation are critical to ensure these technologies positively impact patient care.
Collapse
Affiliation(s)
- Chi-Tung Cheng
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan Taiwan
| | - Chun-Hsiang Ooyang
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan Taiwan
| | - Shih-Ching Kang
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan Taiwan.
| | - Chien-Hung Liao
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan Taiwan
| |
Collapse
|
5
|
Villarreal JA, Forrester JD. Eyes on the prize: harnessing computer vision for automated detection of traumatic rib and clavicle fractures in chest radiographs. Trauma Surg Acute Care Open 2024; 9:e001455. [PMID: 38646616 PMCID: PMC11029340 DOI: 10.1136/tsaco-2024-001455] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/23/2024] Open
Affiliation(s)
- Joshua Aaron Villarreal
- Department of Surgery, Stanford University Department of Medicine, Stanford, California, USA
| | - Joseph D Forrester
- Department of Surgery, Stanford University Department of Medicine, Stanford, California, USA
| |
Collapse
|
6
|
Sabottke C, Lee J, Chiang A, Spieler B, Mushtaq R. Text Report Analysis to Identify Opportunities for Optimizing Target Selection for Chest Radiograph Artificial Intelligence Models. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:402-411. [PMID: 38343239 DOI: 10.1007/s10278-023-00927-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/17/2023] [Revised: 09/06/2023] [Accepted: 09/06/2023] [Indexed: 03/02/2024]
Abstract
Our goal was to analyze radiology report text for chest radiographs (CXRs) to identify imaging findings that have the most impact on report length and complexity. Identifying these imaging findings can highlight opportunities for designing CXR AI systems which increase radiologist efficiency. We retrospectively analyzed text from 210,025 MIMIC-CXR reports and 168,949 reports from our local institution collected from 2019 to 2022. Fifty-nine categories of imaging finding keywords were extracted from reports using natural language processing (NLP), and their impact on report length was assessed using linear regression with and without LASSO regularization. Regression was also used to assess the impact of additional factors contributing to report length, such as the signing radiologist and use of terms of perception. For modeling CXR report word counts with regression, mean coefficient of determination, R2, was 0.469 ± 0.001 for local reports and 0.354 ± 0.002 for MIMIC-CXR when considering only imaging finding keyword features. Mean R2 was significantly less at 0.067 ± 0.001 for local reports and 0.086 ± 0.002 for MIMIC-CXR, when only considering use of terms of perception. For a combined model for the local report data accounting for the signing radiologist, imaging finding keywords, and terms of perception, the mean R2 was 0.570 ± 0.002. With LASSO, highest value coefficients pertained to endotracheal tubes and pleural drains for local data and masses, nodules, and cavitary and cystic lesions for MIMIC-CXR. Natural language processing and regression analysis of radiology report textual data can highlight imaging targets for AI models which offer opportunities to bolster radiologist efficiency.
Collapse
Affiliation(s)
- Carl Sabottke
- Department of Medical Imaging, University of Arizona College of Medicine, Tucson, AZ, USA.
| | - Jason Lee
- Department of Medical Imaging, University of Arizona College of Medicine, Tucson, AZ, USA
| | - Alan Chiang
- Department of Medical Imaging, University of Arizona College of Medicine, Tucson, AZ, USA
| | - Bradley Spieler
- Department of Radiology, Louisiana State University Health Sciences Center, New Orleans, LA, USA
| | - Raza Mushtaq
- Department of Neuroradiology, Barrow Neurological Institute, Phoenix, AZ, USA
| |
Collapse
|
7
|
Katzman BD, Alabousi M, Islam N, Zha N, Patlas MN. Deep Learning for Pneumothorax Detection on Chest Radiograph: A Diagnostic Test Accuracy Systematic Review and Meta Analysis. Can Assoc Radiol J 2024:8465371231220885. [PMID: 38189265 DOI: 10.1177/08465371231220885] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2024] Open
Abstract
BACKGROUND Pneumothorax is a common acute presentation in healthcare settings. A chest radiograph (CXR) is often necessary to make the diagnosis, and minimizing the time between presentation and diagnosis is critical to deliver optimal treatment. Deep learning (DL) algorithms have been developed to rapidly identify pathologic findings on various imaging modalities. PURPOSE The purpose of this systematic review and meta-analysis was to evaluate the overall performance of studies utilizing DL algorithms to detect pneumothorax on CXR. METHODS A study protocol was created and registered a priori (PROSPERO CRD42023391375). The search strategy included studies published up until January 10, 2023. Inclusion criteria were studies that used adult patients, utilized computer-aided detection of pneumothorax on CXR, dataset was evaluated by a qualified physician, and sufficient data was present to create a 2 × 2 contingency table. Risk of bias was assessed using the QUADAS-2 tool. Bivariate random effects meta-analyses and meta-regression modeling were performed. RESULTS Twenty-three studies were selected, including 34 011 patients and 34 075 CXRs. The pooled sensitivity and specificity were 87% (95% confidence interval, 81%, 92%) and 95% (95% confidence interval, 92%, 97%), respectively. The study design, use of an institutional/public data set and risk of bias had no significant effect on the sensitivity and specificity of pneumothorax detection. CONCLUSIONS The relatively high sensitivity and specificity of pneumothorax detection by deep-learning showcases the vast potential for implementation in clinical settings to both augment the workflow of radiologists and assist in more rapid diagnoses and subsequent patient treatment.
Collapse
Affiliation(s)
- Benjamin D Katzman
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada
| | - Mostafa Alabousi
- Department of Medical Imaging, McMaster University, Hamilton, ON, Canada
| | - Nabil Islam
- Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Nanxi Zha
- Department of Medical Imaging, McMaster University, Hamilton, ON, Canada
| | - Michael N Patlas
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
8
|
Pauling C, Kanber B, Arthurs OJ, Shelmerdine SC. Commercially available artificial intelligence tools for fracture detection: the evidence. BJR Open 2024; 6:tzad005. [PMID: 38352182 PMCID: PMC10860511 DOI: 10.1093/bjro/tzad005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 09/20/2023] [Accepted: 09/30/2023] [Indexed: 02/16/2024] Open
Abstract
Missed fractures are a costly healthcare issue, not only negatively impacting patient lives, leading to potential long-term disability and time off work, but also responsible for high medicolegal disbursements that could otherwise be used to improve other healthcare services. When fractures are overlooked in children, they are particularly concerning as opportunities for safeguarding may be missed. Assistance from artificial intelligence (AI) in interpreting medical images may offer a possible solution for improving patient care, and several commercial AI tools are now available for radiology workflow implementation. However, information regarding their development, evidence for performance and validation as well as the intended target population is not always clear, but vital when evaluating a potential AI solution for implementation. In this article, we review the range of available products utilizing AI for fracture detection (in both adults and children) and summarize the evidence, or lack thereof, behind their performance. This will allow others to make better informed decisions when deciding which product to procure for their specific clinical requirements.
Collapse
Affiliation(s)
- Cato Pauling
- UCL Great Ormond Street Institute of Child Health, University College London, London WC1E 6BT, United Kingdom
| | - Baris Kanber
- Queen Square Multiple Sclerosis Centre, Department of Neuroinflammation, University College London (UCL) Queen Square Institute of Neurology, Faculty of Brain Sciences, University College London, London WC1N 3BG, United Kingdom
- Department of Medical Physics and Biomedical Engineering, Centre for Medical Image Computing, University College London, London WC1E 6BT, United Kingdom
| | - Owen J Arthurs
- UCL Great Ormond Street Institute of Child Health, University College London, London WC1E 6BT, United Kingdom
- Department of Clinical Radiology, Great Ormond Street Hospital for Children NHS Foundation Trust, London WC1N 3JH, United Kingdom
- NIHR Great Ormond Street Hospital Biomedical Research Centre, Bloomsbury, London WC1N 1EH, United Kingdom
| | - Susan C Shelmerdine
- UCL Great Ormond Street Institute of Child Health, University College London, London WC1E 6BT, United Kingdom
- Department of Clinical Radiology, Great Ormond Street Hospital for Children NHS Foundation Trust, London WC1N 3JH, United Kingdom
- NIHR Great Ormond Street Hospital Biomedical Research Centre, Bloomsbury, London WC1N 1EH, United Kingdom
| |
Collapse
|
9
|
Lind Plesner L, Müller FC, Brejnebøl MW, Laustrup LC, Rasmussen F, Nielsen OW, Boesen M, Brun Andersen M. Commercially Available Chest Radiograph AI Tools for Detecting Airspace Disease, Pneumothorax, and Pleural Effusion. Radiology 2023; 308:e231236. [PMID: 37750768 DOI: 10.1148/radiol.231236] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/27/2023]
Abstract
Background Commercially available artificial intelligence (AI) tools can assist radiologists in interpreting chest radiographs, but their real-life diagnostic accuracy remains unclear. Purpose To evaluate the diagnostic accuracy of four commercially available AI tools for detection of airspace disease, pneumothorax, and pleural effusion on chest radiographs. Materials and Methods This retrospective study included consecutive adult patients who underwent chest radiography at one of four Danish hospitals in January 2020. Two thoracic radiologists (or three, in cases of disagreement) who had access to all previous and future imaging labeled chest radiographs independently for the reference standard. Area under the receiver operating characteristic curve, sensitivity, and specificity were calculated. Sensitivity and specificity were additionally stratified according to the severity of findings, number of findings on chest radiographs, and radiographic projection. The χ2 and McNemar tests were used for comparisons. Results The data set comprised 2040 patients (median age, 72 years [IQR, 58-81 years]; 1033 female), of whom 669 (32.8%) had target findings. The AI tools demonstrated areas under the receiver operating characteristic curve ranging 0.83-0.88 for airspace disease, 0.89-0.97 for pneumothorax, and 0.94-0.97 for pleural effusion. Sensitivities ranged 72%-91% for airspace disease, 63%-90% for pneumothorax, and 62%-95% for pleural effusion. Negative predictive values ranged 92%-100% for all target findings. In airspace disease, pneumothorax, and pleural effusion, specificity was high for chest radiographs with normal or single findings (range, 85%-96%, 99%-100%, and 95%-100%, respectively) and markedly lower for chest radiographs with four or more findings (range, 27%-69%, 96%-99%, 65%-92%, respectively) (P < .001). AI sensitivity was lower for vague airspace disease (range, 33%-61%) and small pneumothorax or pleural effusion (range, 9%-94%) compared with larger findings (range, 81%-100%; P value range, > .99 to < .001). Conclusion Current-generation AI tools showed moderate to high sensitivity for detecting airspace disease, pneumothorax, and pleural effusion on chest radiographs. However, they produced more false-positive findings than radiology reports, and their performance decreased for smaller-sized target findings and when multiple findings were present. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Yanagawa and Tomiyama in this issue.
Collapse
Affiliation(s)
- Louis Lind Plesner
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Felix C Müller
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Mathias W Brejnebøl
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Lene C Laustrup
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Finn Rasmussen
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Olav W Nielsen
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Mikael Boesen
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Michael Brun Andersen
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| |
Collapse
|
10
|
Sugibayashi T, Walston SL, Matsumoto T, Mitsuyama Y, Miki Y, Ueda D. Deep learning for pneumothorax diagnosis: a systematic review and meta-analysis. Eur Respir Rev 2023; 32:32/168/220259. [PMID: 37286217 DOI: 10.1183/16000617.0259-2022] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Accepted: 03/16/2023] [Indexed: 06/09/2023] Open
Abstract
BACKGROUND Deep learning (DL), a subset of artificial intelligence (AI), has been applied to pneumothorax diagnosis to aid physician diagnosis, but no meta-analysis has been performed. METHODS A search of multiple electronic databases through September 2022 was performed to identify studies that applied DL for pneumothorax diagnosis using imaging. Meta-analysis via a hierarchical model to calculate the summary area under the curve (AUC) and pooled sensitivity and specificity for both DL and physicians was performed. Risk of bias was assessed using a modified Prediction Model Study Risk of Bias Assessment Tool. RESULTS In 56 of the 63 primary studies, pneumothorax was identified from chest radiography. The total AUC was 0.97 (95% CI 0.96-0.98) for both DL and physicians. The total pooled sensitivity was 84% (95% CI 79-89%) for DL and 85% (95% CI 73-92%) for physicians and the pooled specificity was 96% (95% CI 94-98%) for DL and 98% (95% CI 95-99%) for physicians. More than half of the original studies (57%) had a high risk of bias. CONCLUSIONS Our review found the diagnostic performance of DL models was similar to that of physicians, although the majority of studies had a high risk of bias. Further pneumothorax AI research is needed.
Collapse
Affiliation(s)
- Takahiro Sugibayashi
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Shannon L Walston
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Toshimasa Matsumoto
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
- Smart Life Science Lab, Center for Health Science Innovation, Osaka Metropolitan University, Osaka, Japan
| | - Yasuhito Mitsuyama
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Yukio Miki
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Daiju Ueda
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
- Smart Life Science Lab, Center for Health Science Innovation, Osaka Metropolitan University, Osaka, Japan
| |
Collapse
|
11
|
Ouyang CH, Chen CC, Tee YS, Lin WC, Kuo LW, Liao CA, Cheng CT, Liao CH. The Application of Design Thinking in Developing a Deep Learning Algorithm for Hip Fracture Detection. Bioengineering (Basel) 2023; 10:735. [PMID: 37370666 DOI: 10.3390/bioengineering10060735] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2023] [Revised: 06/05/2023] [Accepted: 06/12/2023] [Indexed: 06/29/2023] Open
Abstract
(1) Background: Design thinking is a problem-solving approach that has been applied in various sectors, including healthcare and medical education. While deep learning (DL) algorithms can assist in clinical practice, integrating them into clinical scenarios can be challenging. This study aimed to use design thinking steps to develop a DL algorithm that accelerates deployment in clinical practice and improves its performance to meet clinical requirements. (2) Methods: We applied the design thinking process to interview clinical doctors and gain insights to develop and modify the DL algorithm to meet clinical scenarios. We also compared the DL performance of the algorithm before and after the integration of design thinking. (3) Results: After empathizing with clinical doctors and defining their needs, we identified the unmet need of five trauma surgeons as "how to reduce the misdiagnosis of femoral fracture by pelvic plain film (PXR) at initial emergency visiting". We collected 4235 PXRs from our hospital, of which 2146 had a hip fracture (51%) from 2008 to 2016. We developed hip fracture DL detection models based on the Xception convolutional neural network by using these images. By incorporating design thinking, we improved the diagnostic accuracy from 0.91 (0.84-0.96) to 0.95 (0.93-0.97), the sensitivity from 0.97 (0.89-1.00) to 0.97 (0.94-0.99), and the specificity from 0.84 (0.71-0.93) to 0.93(0.990-0.97). (4) Conclusions: In summary, this study demonstrates that design thinking can ensure that DL solutions developed for trauma care are user-centered and meet the needs of patients and healthcare providers.
Collapse
Affiliation(s)
- Chun-Hsiang Ouyang
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Chang Gung University, Linkou, Taoyuan 33328, Taiwan
| | - Chih-Chi Chen
- Department of Rehabilitation and Physical Medicine, Chang Gung Memorial Hospital, Chang Gung University, Linkou, Taoyuan 33328, Taiwan
| | - Yu-San Tee
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Chang Gung University, Linkou, Taoyuan 33328, Taiwan
| | - Wei-Cheng Lin
- Department of Electrical Engineering, Chang Gung University, Taoyuan 33327, Taiwan
| | - Ling-Wei Kuo
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Chang Gung University, Linkou, Taoyuan 33328, Taiwan
| | - Chien-An Liao
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Chang Gung University, Linkou, Taoyuan 33328, Taiwan
| | - Chi-Tung Cheng
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Chang Gung University, Linkou, Taoyuan 33328, Taiwan
| | - Chien-Hung Liao
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Chang Gung University, Linkou, Taoyuan 33328, Taiwan
| |
Collapse
|
12
|
Fanni SC, Marcucci A, Volpi F, Valentino S, Neri E, Romei C. Artificial Intelligence-Based Software with CE Mark for Chest X-ray Interpretation: Opportunities and Challenges. Diagnostics (Basel) 2023; 13:2020. [PMID: 37370915 DOI: 10.3390/diagnostics13122020] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Revised: 04/26/2023] [Accepted: 06/08/2023] [Indexed: 06/29/2023] Open
Abstract
Chest X-ray (CXR) is the most important technique for performing chest imaging, despite its well-known limitations in terms of scope and sensitivity. These intrinsic limitations of CXR have prompted the development of several artificial intelligence (AI)-based software packages dedicated to CXR interpretation. The online database "AI for radiology" was queried to identify CE-marked AI-based software available for CXR interpretation. The returned studies were divided according to the targeted disease. AI-powered computer-aided detection software is already widely adopted in screening and triage for pulmonary tuberculosis, especially in countries with few resources and suffering from high a burden of this disease. AI-based software has also been demonstrated to be valuable for the detection of lung nodules detection, automated flagging of positive cases, and post-processing through the development of digital bone suppression software able to produce digital bone suppressed images. Finally, the majority of available CE-marked software packages for CXR are designed to recognize several findings, with potential differences in sensitivity and specificity for each of the recognized findings.
Collapse
Affiliation(s)
- Salvatore Claudio Fanni
- Department of Translational Research, Academic Radiology, University of Pisa, 56126 Pisa, Italy
| | - Alessandro Marcucci
- Department of Translational Research, Academic Radiology, University of Pisa, 56126 Pisa, Italy
| | - Federica Volpi
- Department of Translational Research, Academic Radiology, University of Pisa, 56126 Pisa, Italy
| | | | - Emanuele Neri
- Department of Translational Research, Academic Radiology, University of Pisa, 56126 Pisa, Italy
| | - Chiara Romei
- Department of Diagnostic Imaging, 2nd Radiology Unit, Pisa University-Hospital, Via Paradisa 2, 56124 Pisa, Italy
| |
Collapse
|
13
|
Agrawal A, Khatri GD, Khurana B, Sodickson AD, Liang Y, Dreizin D. A survey of ASER members on artificial intelligence in emergency radiology: trends, perceptions, and expectations. Emerg Radiol 2023; 30:267-277. [PMID: 36913061 PMCID: PMC10362990 DOI: 10.1007/s10140-023-02121-0] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2023] [Accepted: 02/28/2023] [Indexed: 03/14/2023]
Abstract
PURPOSE There is a growing body of diagnostic performance studies for emergency radiology-related artificial intelligence/machine learning (AI/ML) tools; however, little is known about user preferences, concerns, experiences, expectations, and the degree of penetration of AI tools in emergency radiology. Our aim is to conduct a survey of the current trends, perceptions, and expectations regarding AI among American Society of Emergency Radiology (ASER) members. METHODS An anonymous and voluntary online survey questionnaire was e-mailed to all ASER members, followed by two reminder e-mails. A descriptive analysis of the data was conducted, and results summarized. RESULTS A total of 113 members responded (response rate 12%). The majority were attending radiologists (90%) with greater than 10 years' experience (80%) and from an academic practice (65%). Most (55%) reported use of commercial AI CAD tools in their practice. Workflow prioritization based on pathology detection, injury or disease severity grading and classification, quantitative visualization, and auto-population of structured reports were identified as high-value tasks. Respondents overwhelmingly indicated a need for explainable and verifiable tools (87%) and the need for transparency in the development process (80%). Most respondents did not feel that AI would reduce the need for emergency radiologists in the next two decades (72%) or diminish interest in fellowship programs (58%). Negative perceptions pertained to potential for automation bias (23%), over-diagnosis (16%), poor generalizability (15%), negative impact on training (11%), and impediments to workflow (10%). CONCLUSION ASER member respondents are in general optimistic about the impact of AI in the practice of emergency radiology and its impact on the popularity of emergency radiology as a subspecialty. The majority expect to see transparent and explainable AI models with the radiologist as the decision-maker.
Collapse
Affiliation(s)
- Anjali Agrawal
- New Delhi operations, Teleradiology Solutions, Delhi, India
| | - Garvit D Khatri
- Nuclear Medicine, Department of Radiology, University of Washington School of Medicine, Seattle, WA, USA
| | - Bharti Khurana
- Emergency Radiology, Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Aaron D Sodickson
- Emergency Radiology, Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Yuanyuan Liang
- Epidemiology & Public Health, University of Maryland School of Medicine, Baltimore, MD, USA
| | - David Dreizin
- Trauma and Emergency Radiology, Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma Center, University of Maryland School of Medicine, Baltimore, MD, USA.
| |
Collapse
|
14
|
Dreizin D. The American Society of Emergency Radiology (ASER) AI/ML expert panel: inception, mandate, work products, and goals. Emerg Radiol 2023; 30:279-283. [PMID: 37071272 DOI: 10.1007/s10140-023-02135-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Accepted: 04/11/2023] [Indexed: 04/19/2023]
Affiliation(s)
- David Dreizin
- Emergency and Trauma Imaging, Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma , Center, University of Maryland School of Medicine, Baltimore, MD, USA.
| |
Collapse
|
15
|
Dreizin D, Staziaki PV, Khatri GD, Beckmann NM, Feng Z, Liang Y, Delproposto ZS, Klug M, Spann JS, Sarkar N, Fu Y. Artificial intelligence CAD tools in trauma imaging: a scoping review from the American Society of Emergency Radiology (ASER) AI/ML Expert Panel. Emerg Radiol 2023; 30:251-265. [PMID: 36917287 PMCID: PMC10640925 DOI: 10.1007/s10140-023-02120-1] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Accepted: 02/27/2023] [Indexed: 03/16/2023]
Abstract
BACKGROUND AI/ML CAD tools can potentially improve outcomes in the high-stakes, high-volume model of trauma radiology. No prior scoping review has been undertaken to comprehensively assess tools in this subspecialty. PURPOSE To map the evolution and current state of trauma radiology CAD tools along key dimensions of technology readiness. METHODS Following a search of databases, abstract screening, and full-text document review, CAD tool maturity was charted using elements of data curation, performance validation, outcomes research, explainability, user acceptance, and funding patterns. Descriptive statistics were used to illustrate key trends. RESULTS A total of 4052 records were screened, and 233 full-text articles were selected for content analysis. Twenty-one papers described FDA-approved commercial tools, and 212 reported algorithm prototypes. Works ranged from foundational research to multi-reader multi-case trials with heterogeneous external data. Scalable convolutional neural network-based implementations increased steeply after 2016 and were used in all commercial products; however, options for explainability were narrow. Of FDA-approved tools, 9/10 performed detection tasks. Dataset sizes ranged from < 100 to > 500,000 patients, and commercialization coincided with public dataset availability. Cross-sectional torso datasets were uniformly small. Data curation methods with ground truth labeling by independent readers were uncommon. No papers assessed user acceptance, and no method included human-computer interaction. The USA and China had the highest research output and frequency of research funding. CONCLUSIONS Trauma imaging CAD tools are likely to improve patient care but are currently in an early stage of maturity, with few FDA-approved products for a limited number of uses. The scarcity of high-quality annotated data remains a major barrier.
Collapse
Affiliation(s)
- David Dreizin
- Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma Center, University of Maryland School of Medicine, Baltimore, MD, USA.
| | - Pedro V Staziaki
- Cardiothoracic Imaging, Department of Radiology, Larner College of Medicine, University of Vermont, Burlington, VT, USA
| | - Garvit D Khatri
- Department of Radiology, University of Washington School of Medicine, Seattle, WA, USA
| | - Nicholas M Beckmann
- Memorial Hermann Orthopedic & Spine Hospital, McGovern Medical School at UTHealth, Houston, TX, USA
| | - Zhaoyong Feng
- Epidemiology & Public Health, University of Maryland School of Medicine, Baltimore, MD, USA
| | - Yuanyuan Liang
- Epidemiology & Public Health, University of Maryland School of Medicine, Baltimore, MD, USA
| | - Zachary S Delproposto
- Division of Emergency Radiology, Department of Radiology, University of Michigan, Ann Arbor, MI, USA
| | | | - J Stephen Spann
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine, Birmingham, AL, USA
| | - Nathan Sarkar
- University of Maryland School of Medicine, Baltimore, MD, USA
| | - Yunting Fu
- Health Sciences and Human Services Library, University of Maryland, Baltimore, Baltimore, MD, USA
| |
Collapse
|
16
|
Mitra B, Law A, Mathew J, Crabtree A, Mertin H, Underhill A, Noonan M, Hunter P, Smit DV. Telehealth consultation before inter-hospital transfer after falls in a subacute hospital (the PREVENT-2 study). Emerg Med Australas 2023; 35:306-311. [PMID: 36358005 DOI: 10.1111/1742-6723.14130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2022] [Revised: 10/09/2022] [Accepted: 10/11/2022] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Inter-hospital transfers are increasingly common due to the regionalisation of healthcare, but are associated with patient discomfort, high costs and adverse events. The aim of the present study was to evaluate the effectiveness of a trauma outreach service for preventing inter-hospital transfers to a major trauma centre. METHODS This was an observational pre- and post-intervention study over a 12-month period from 1 October 2020 to 30 September 2021. Eligible patients sustained a fall at Caulfield Hospital, a subacute care hospital specialising in community services, rehabilitation, geriatric medicine and aged mental health. The intervention was delivery of site-specific education at Caulfield Hospital and a trauma outreach service by specialist trauma clinicians at The Alfred Hospital who provided remote assessment, assisted with clinical management decisions and advised on appropriateness of transfer. RESULTS The present study included 160 patients in the pre-intervention phase and 203 after the intervention. The primary outcome of transfer occurred in 19 (11.9%) patients in the pre-intervention phase and 4 (2.0%) in the post-intervention phase (P < 0.001). In the subgroup of patients without pelvis or long bone fractures, pre-intervention transfer occurred for 17 (10.9%) patients and post-intervention transfer occurred for 4 (2.0%) patients (P < 0.001). CT imaging was performed for 54 (33.8%) patients in the pre-intervention and 45 (22.2%) patients in the post-intervention group (P = 0.014). CONCLUSIONS Telehealth consultation with a trauma specialist was associated with significant reduction of inter-hospital transfers, and significant reduction of CT imaging. This supports continuation of the service with scope for expansion and evaluation of patient-centred outcomes.
Collapse
Affiliation(s)
- Biswadev Mitra
- Emergency and Trauma Centre, The Alfred Hospital, Melbourne, Victoria, Australia
- School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia
- National Trauma Research Institute, The Alfred Hospital, Melbourne, Victoria, Australia
| | - Amelia Law
- Emergency and Trauma Centre, The Alfred Hospital, Melbourne, Victoria, Australia
| | - Joseph Mathew
- Emergency and Trauma Centre, The Alfred Hospital, Melbourne, Victoria, Australia
- National Trauma Research Institute, The Alfred Hospital, Melbourne, Victoria, Australia
- Trauma Service, The Alfred Hospital, Melbourne, Victoria, Australia
| | - Amelia Crabtree
- Health of Older People Unit, Caulfield Hospital, Melbourne, Victoria, Australia
| | - Helen Mertin
- Health of Older People Unit, Caulfield Hospital, Melbourne, Victoria, Australia
| | - Andrew Underhill
- Emergency and Trauma Centre, The Alfred Hospital, Melbourne, Victoria, Australia
| | - Michael Noonan
- Emergency and Trauma Centre, The Alfred Hospital, Melbourne, Victoria, Australia
- National Trauma Research Institute, The Alfred Hospital, Melbourne, Victoria, Australia
- Trauma Service, The Alfred Hospital, Melbourne, Victoria, Australia
- Central Clinical School, Monash University, Melbourne, Victoria, Australia
| | - Peter Hunter
- Health of Older People Unit, Caulfield Hospital, Melbourne, Victoria, Australia
| | - De Villiers Smit
- Emergency and Trauma Centre, The Alfred Hospital, Melbourne, Victoria, Australia
- School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia
- National Trauma Research Institute, The Alfred Hospital, Melbourne, Victoria, Australia
| |
Collapse
|
17
|
Ahmad HK, Milne MR, Buchlak QD, Ektas N, Sanderson G, Chamtie H, Karunasena S, Chiang J, Holt X, Tang CHM, Seah JCY, Bottrell G, Esmaili N, Brotchie P, Jones C. Machine Learning Augmented Interpretation of Chest X-rays: A Systematic Review. Diagnostics (Basel) 2023; 13:diagnostics13040743. [PMID: 36832231 PMCID: PMC9955112 DOI: 10.3390/diagnostics13040743] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2023] [Revised: 02/13/2023] [Accepted: 02/14/2023] [Indexed: 02/18/2023] Open
Abstract
Limitations of the chest X-ray (CXR) have resulted in attempts to create machine learning systems to assist clinicians and improve interpretation accuracy. An understanding of the capabilities and limitations of modern machine learning systems is necessary for clinicians as these tools begin to permeate practice. This systematic review aimed to provide an overview of machine learning applications designed to facilitate CXR interpretation. A systematic search strategy was executed to identify research into machine learning algorithms capable of detecting >2 radiographic findings on CXRs published between January 2020 and September 2022. Model details and study characteristics, including risk of bias and quality, were summarized. Initially, 2248 articles were retrieved, with 46 included in the final review. Published models demonstrated strong standalone performance and were typically as accurate, or more accurate, than radiologists or non-radiologist clinicians. Multiple studies demonstrated an improvement in the clinical finding classification performance of clinicians when models acted as a diagnostic assistance device. Device performance was compared with that of clinicians in 30% of studies, while effects on clinical perception and diagnosis were evaluated in 19%. Only one study was prospectively run. On average, 128,662 images were used to train and validate models. Most classified less than eight clinical findings, while the three most comprehensive models classified 54, 72, and 124 findings. This review suggests that machine learning devices designed to facilitate CXR interpretation perform strongly, improve the detection performance of clinicians, and improve the efficiency of radiology workflow. Several limitations were identified, and clinician involvement and expertise will be key to driving the safe implementation of quality CXR machine learning systems.
Collapse
Affiliation(s)
- Hassan K. Ahmad
- Annalise.ai, Sydney, NSW 2000, Australia
- Department of Emergency Medicine, Royal North Shore Hospital, Sydney, NSW 2065, Australia
- Correspondence:
| | | | - Quinlan D. Buchlak
- Annalise.ai, Sydney, NSW 2000, Australia
- School of Medicine, University of Notre Dame Australia, Sydney, NSW 2007, Australia
- Department of Neurosurgery, Monash Health, Melbourne, VIC 3168, Australia
| | | | | | | | | | - Jason Chiang
- Annalise.ai, Sydney, NSW 2000, Australia
- Department of General Practice, University of Melbourne, Melbourne, VIC 3010, Australia
- Westmead Applied Research Centre, University of Sydney, Sydney, NSW 2006, Australia
| | | | | | - Jarrel C. Y. Seah
- Annalise.ai, Sydney, NSW 2000, Australia
- Department of Radiology, Alfred Health, Melbourne, VIC 3004, Australia
| | | | - Nazanin Esmaili
- School of Medicine, University of Notre Dame Australia, Sydney, NSW 2007, Australia
- Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW 2007, Australia
| | - Peter Brotchie
- Annalise.ai, Sydney, NSW 2000, Australia
- Department of Radiology, St Vincent’s Health Australia, Melbourne, VIC 3065, Australia
| | - Catherine Jones
- Annalise.ai, Sydney, NSW 2000, Australia
- I-MED Radiology Network, Brisbane, QLD 4006, Australia
- School of Public and Preventive Health, Monash University, Clayton, VIC 3800, Australia
- Department of Clinical Imaging Science, University of Sydney, Sydney, NSW 2006, Australia
| |
Collapse
|