1
|
Geppert J, Asgharzadeh A, Brown A, Stinton C, Helm EJ, Jayakody S, Todkill D, Gallacher D, Ghiasvand H, Patel M, Auguste P, Tsertsvadze A, Chen YF, Grove A, Shinkins B, Clarke A, Taylor-Phillips S. Software using artificial intelligence for nodule and cancer detection in CT lung cancer screening: systematic review of test accuracy studies. Thorax 2024:thorax-2024-221662. [PMID: 39322406 DOI: 10.1136/thorax-2024-221662] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2024] [Accepted: 09/04/2024] [Indexed: 09/27/2024]
Abstract
OBJECTIVES To examine the accuracy and impact of artificial intelligence (AI) software assistance in lung cancer screening using CT. METHODS A systematic review of CE-marked, AI-based software for automated detection and analysis of nodules in CT lung cancer screening was conducted. Multiple databases including Medline, Embase and Cochrane CENTRAL were searched from 2012 to March 2023. Primary research reporting test accuracy or impact on reading time or clinical management was included. QUADAS-2 and QUADAS-C were used to assess risk of bias. We undertook narrative synthesis. RESULTS Eleven studies evaluating six different AI-based software and reporting on 19 770 patients were eligible. All were at high risk of bias with multiple applicability concerns. Compared with unaided reading, AI-assisted reading was faster and generally improved sensitivity (+5% to +20% for detecting/categorising actionable nodules; +3% to +15% for detecting/categorising malignant nodules), with lower specificity (-7% to -3% for correctly detecting/categorising people without actionable nodules; -8% to -6% for correctly detecting/categorising people without malignant nodules). AI assistance tended to increase the proportion of nodules allocated to higher risk categories. Assuming 0.5% cancer prevalence, these results would translate into additional 150-750 cancers detected per million people attending screening but lead to an additional 59 700 to 79 600 people attending screening without cancer receiving unnecessary CT surveillance. CONCLUSIONS AI assistance in lung cancer screening may improve sensitivity but increases the number of false-positive results and unnecessary surveillance. Future research needs to increase the specificity of AI-assisted reading and minimise risk of bias and applicability concerns through improved study design. PROSPERO REGISTRATION NUMBER CRD42021298449.
Collapse
Affiliation(s)
- Julia Geppert
- Warwick Screening & Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | - Asra Asgharzadeh
- Population Health Science, University of Bristol, Bristol, UK
- Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | - Anna Brown
- Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | - Chris Stinton
- Warwick Screening & Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | - Emma J Helm
- Department of Radiology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Surangi Jayakody
- Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | - Daniel Todkill
- Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | - Daniel Gallacher
- Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | - Hesam Ghiasvand
- Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
- Research Centre for Healthcare and Communities, Coventry University, Coventry, UK
| | - Mubarak Patel
- Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | - Peter Auguste
- Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | | | - Yen-Fu Chen
- Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | - Amy Grove
- Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | - Bethany Shinkins
- Warwick Screening & Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | - Aileen Clarke
- Warwick Evidence, Warwick Medical School, University of Warwick, Coventry, UK
| | | |
Collapse
|
2
|
Fussell DA, Tang CC, Sternhagen J, Marrey VV, Roman KM, Johnson J, Head MJ, Troutt HR, Li CH, Chang PD, Joseph J, Chow DS. Artificial Intelligence Efficacy as a Function of Trainee Interpreter Proficiency: Lessons from a Randomized Controlled Trial. AJNR Am J Neuroradiol 2024:ajnr.A8387. [PMID: 38906673 DOI: 10.3174/ajnr.a8387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2024] [Accepted: 06/13/2024] [Indexed: 06/23/2024]
Abstract
BACKGROUND AND PURPOSE Recently, artificial intelligence tools have been deployed with increasing speed in educational and clinical settings. However, the use of artificial intelligence by trainees across different levels of experience has not been well-studied. This study investigates the impact of artificial intelligence assistance on the diagnostic accuracy for intracranial hemorrhage and large-vessel occlusion by medical students and resident trainees. MATERIALS AND METHODS This prospective study was conducted between March 2023 and October 2023. Medical students and resident trainees were asked to identify intracranial hemorrhage and large-vessel occlusion in 100 noncontrast head CTs and 100 head CTAs, respectively. One group received diagnostic aid simulating artificial intelligence for intracranial hemorrhage only (n = 26); the other, for large-vessel occlusion only (n = 28). Primary outcomes included accuracy, sensitivity, and specificity for intracranial hemorrhage/large-vessel occlusion detection without and with aid. Study interpretation time was a secondary outcome. Individual responses were pooled and analyzed with the t test; differences in continuous variables were assessed with ANOVA. RESULTS Forty-eight participants completed the study, generating 10,779 intracranial hemorrhage or large-vessel occlusion interpretations. With diagnostic aid, medical student accuracy improved 11.0 points (P < .001) and resident trainee accuracy showed no significant change. Intracranial hemorrhage interpretation time increased with diagnostic aid for both groups (P < .001), while large-vessel occlusion interpretation time decreased for medical students (P < .001). Despite worse performance in the detection of the smallest-versus-largest hemorrhages at baseline, medical students were not more likely to accept a true-positive artificial intelligence result for these more difficult tasks. Both groups were considerably less accurate when disagreeing with the artificial intelligence or when supplied with an incorrect artificial intelligence result. CONCLUSIONS This study demonstrated greater improvement in diagnostic accuracy with artificial intelligence for medical students compared with resident trainees. However, medical students were less likely than resident trainees to overrule incorrect artificial intelligence interpretations and were less accurate, even with diagnostic aid, than the artificial intelligence was by itself.
Collapse
Affiliation(s)
- David A Fussell
- From the Department of Radiological Sciences (D.A.F., C.C.T., J.S., V.V.M., J.J., H.R.T., C.H.L., P.D.C., D.S.C.), University of California, Irvine, Irvine, California
| | - Cynthia C Tang
- From the Department of Radiological Sciences (D.A.F., C.C.T., J.S., V.V.M., J.J., H.R.T., C.H.L., P.D.C., D.S.C.), University of California, Irvine, Irvine, California
| | - Jake Sternhagen
- From the Department of Radiological Sciences (D.A.F., C.C.T., J.S., V.V.M., J.J., H.R.T., C.H.L., P.D.C., D.S.C.), University of California, Irvine, Irvine, California
| | - Varun V Marrey
- From the Department of Radiological Sciences (D.A.F., C.C.T., J.S., V.V.M., J.J., H.R.T., C.H.L., P.D.C., D.S.C.), University of California, Irvine, Irvine, California
| | - Kelsey M Roman
- School of Medicine (K.M.R., M.J.H.), University of California, Irvine, Irvine, California
| | - Jeremy Johnson
- From the Department of Radiological Sciences (D.A.F., C.C.T., J.S., V.V.M., J.J., H.R.T., C.H.L., P.D.C., D.S.C.), University of California, Irvine, Irvine, California
| | - Michael J Head
- School of Medicine (K.M.R., M.J.H.), University of California, Irvine, Irvine, California
| | - Hayden R Troutt
- From the Department of Radiological Sciences (D.A.F., C.C.T., J.S., V.V.M., J.J., H.R.T., C.H.L., P.D.C., D.S.C.), University of California, Irvine, Irvine, California
| | - Charles H Li
- From the Department of Radiological Sciences (D.A.F., C.C.T., J.S., V.V.M., J.J., H.R.T., C.H.L., P.D.C., D.S.C.), University of California, Irvine, Irvine, California
| | - Peter D Chang
- From the Department of Radiological Sciences (D.A.F., C.C.T., J.S., V.V.M., J.J., H.R.T., C.H.L., P.D.C., D.S.C.), University of California, Irvine, Irvine, California
| | - John Joseph
- Paul Merage School of Business (J.J.), University of California, Irvine, Irvine, California
| | - Daniel S Chow
- From the Department of Radiological Sciences (D.A.F., C.C.T., J.S., V.V.M., J.J., H.R.T., C.H.L., P.D.C., D.S.C.), University of California, Irvine, Irvine, California
| |
Collapse
|
3
|
Plesner LL, Müller FC, Brejnebøl MW, Krag CH, Laustrup LC, Rasmussen F, Nielsen OW, Boesen M, Andersen MB. Using AI to Identify Unremarkable Chest Radiographs for Automatic Reporting. Radiology 2024; 312:e240272. [PMID: 39162628 DOI: 10.1148/radiol.240272] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/21/2024]
Abstract
Background Radiology practices have a high volume of unremarkable chest radiographs and artificial intelligence (AI) could possibly improve workflow by providing an automatic report. Purpose To estimate the proportion of unremarkable chest radiographs, where AI can correctly exclude pathology (ie, specificity) without increasing diagnostic errors. Materials and Methods In this retrospective study, consecutive chest radiographs in unique adult patients (≥18 years of age) were obtained January 1-12, 2020, at four Danish hospitals. Exclusion criteria included insufficient radiology reports or AI output error. Two thoracic radiologists, who were blinded to AI output, labeled chest radiographs as "remarkable" or "unremarkable" based on predefined unremarkable findings (reference standard). Radiology reports were classified similarly. A commercial AI tool was adapted to output a chest radiograph "remarkableness" probability, which was used to calculate specificity at different AI sensitivities. Chest radiographs with missed findings by AI and/or the radiology report were graded by one thoracic radiologist as critical, clinically significant, or clinically insignificant. Paired proportions were compared using the McNemar test. Results A total of 1961 patients were included (median age, 72 years [IQR, 58-81 years]; 993 female), with one chest radiograph per patient. The reference standard labeled 1231 of 1961 chest radiographs (62.8%) as remarkable and 730 of 1961 (37.2%) as unremarkable. At 99.9%, 99.0%, and 98.0% sensitivity, the AI had a specificity of 24.5% (179 of 730 radiographs [95% CI: 21, 28]), 47.1% (344 of 730 radiographs [95% CI: 43, 51]), and 52.7% (385 of 730 radiographs [95% CI: 49, 56]), respectively. With the AI fixed to have a similar sensitivity as radiology reports (87.2%), the missed findings of AI and reports had 2.2% (27 of 1231 radiographs) and 1.1% (14 of 1231 radiographs) classified as critical (P = .01), 4.1% (51 of 1231 radiographs) and 3.6% (44 of 1231 radiographs) classified as clinically significant (P = .46), and 6.5% (80 of 1231) and 8.1% (100 of 1231) classified as clinically insignificant (P = .11), respectively. At sensitivities greater than or equal to 95.4%, the AI tool exhibited less than or equal to 1.1% critical misses. Conclusion A commercial AI tool used off-label could correctly exclude pathology in 24.5%-52.7% of all unremarkable chest radiographs at greater than or equal to 98% sensitivity. The AI had equal or lower rates of critical misses than radiology reports at sensitivities greater than or equal to 95.4%. These results should be confirmed in a prospective study. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Yoon and Hwang in this issue.
Collapse
Affiliation(s)
- Louis Lind Plesner
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., C.H.K., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Herlev, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., M.B., M.B.A.); Department of Radiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (M.W.B., M.B.); Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.); and Department of Cardiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (O.W.N.)
| | - Felix C Müller
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., C.H.K., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Herlev, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., M.B., M.B.A.); Department of Radiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (M.W.B., M.B.); Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.); and Department of Cardiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (O.W.N.)
| | - Mathias W Brejnebøl
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., C.H.K., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Herlev, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., M.B., M.B.A.); Department of Radiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (M.W.B., M.B.); Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.); and Department of Cardiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (O.W.N.)
| | - Christian Hedeager Krag
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., C.H.K., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Herlev, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., M.B., M.B.A.); Department of Radiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (M.W.B., M.B.); Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.); and Department of Cardiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (O.W.N.)
| | - Lene C Laustrup
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., C.H.K., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Herlev, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., M.B., M.B.A.); Department of Radiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (M.W.B., M.B.); Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.); and Department of Cardiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (O.W.N.)
| | - Finn Rasmussen
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., C.H.K., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Herlev, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., M.B., M.B.A.); Department of Radiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (M.W.B., M.B.); Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.); and Department of Cardiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (O.W.N.)
| | - Olav Wendelboe Nielsen
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., C.H.K., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Herlev, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., M.B., M.B.A.); Department of Radiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (M.W.B., M.B.); Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.); and Department of Cardiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (O.W.N.)
| | - Mikael Boesen
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., C.H.K., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Herlev, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., M.B., M.B.A.); Department of Radiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (M.W.B., M.B.); Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.); and Department of Cardiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (O.W.N.)
| | - Michael B Andersen
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., C.H.K., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Herlev, Denmark (L.L.P., F.C.M., M.W.B., C.H.K., M.B., M.B.A.); Department of Radiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (M.W.B., M.B.); Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.); and Department of Cardiology, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark (O.W.N.)
| |
Collapse
|
4
|
Katzman BD, Alabousi M, Islam N, Zha N, Patlas MN. Deep Learning for Pneumothorax Detection on Chest Radiograph: A Diagnostic Test Accuracy Systematic Review and Meta Analysis. Can Assoc Radiol J 2024; 75:525-533. [PMID: 38189265 DOI: 10.1177/08465371231220885] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2024] Open
Abstract
BACKGROUND Pneumothorax is a common acute presentation in healthcare settings. A chest radiograph (CXR) is often necessary to make the diagnosis, and minimizing the time between presentation and diagnosis is critical to deliver optimal treatment. Deep learning (DL) algorithms have been developed to rapidly identify pathologic findings on various imaging modalities. PURPOSE The purpose of this systematic review and meta-analysis was to evaluate the overall performance of studies utilizing DL algorithms to detect pneumothorax on CXR. METHODS A study protocol was created and registered a priori (PROSPERO CRD42023391375). The search strategy included studies published up until January 10, 2023. Inclusion criteria were studies that used adult patients, utilized computer-aided detection of pneumothorax on CXR, dataset was evaluated by a qualified physician, and sufficient data was present to create a 2 × 2 contingency table. Risk of bias was assessed using the QUADAS-2 tool. Bivariate random effects meta-analyses and meta-regression modeling were performed. RESULTS Twenty-three studies were selected, including 34 011 patients and 34 075 CXRs. The pooled sensitivity and specificity were 87% (95% confidence interval, 81%, 92%) and 95% (95% confidence interval, 92%, 97%), respectively. The study design, use of an institutional/public data set and risk of bias had no significant effect on the sensitivity and specificity of pneumothorax detection. CONCLUSIONS The relatively high sensitivity and specificity of pneumothorax detection by deep-learning showcases the vast potential for implementation in clinical settings to both augment the workflow of radiologists and assist in more rapid diagnoses and subsequent patient treatment.
Collapse
Affiliation(s)
- Benjamin D Katzman
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada
| | - Mostafa Alabousi
- Department of Medical Imaging, McMaster University, Hamilton, ON, Canada
| | - Nabil Islam
- Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Nanxi Zha
- Department of Medical Imaging, McMaster University, Hamilton, ON, Canada
| | - Michael N Patlas
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
5
|
Chou HY, Lin YC, Hsieh SY, Chou HH, Lai CS, Wang B, Tsai YS. Deep Learning Model for Prediction of Bronchopulmonary Dysplasia in Preterm Infants Using Chest Radiographs. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01050-9. [PMID: 38499706 DOI: 10.1007/s10278-024-01050-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Revised: 12/05/2023] [Accepted: 12/18/2023] [Indexed: 03/20/2024]
Abstract
Bronchopulmonary dysplasia (BPD) is common in preterm infants and may result in pulmonary vascular disease, compromising lung function. This study aimed to employ artificial intelligence (AI) techniques to help physicians accurately diagnose BPD in preterm infants in a timely and efficient manner. This retrospective study involves two datasets: a lung region segmentation dataset comprising 1491 chest radiographs of infants, and a BPD prediction dataset comprising 1021 chest radiographs of preterm infants. Transfer learning of a pre-trained machine learning model was employed for lung region segmentation and image fusion for BPD prediction to enhance the performance of the AI model. The lung segmentation model uses transfer learning to achieve a dice score of 0.960 for preterm infants with ≤ 168 h postnatal age. The BPD prediction model exhibited superior diagnostic performance compared to that of experts and demonstrated consistent performance for chest radiographs obtained at ≤ 24 h postnatal age, and those obtained at 25 to 168 h postnatal age. This study is the first to use deep learning on preterm chest radiographs for lung segmentation to develop a BPD prediction model with an early detection time of less than 24 h. Additionally, this study compared the model's performance according to both NICHD and Jensen criteria for BPD. Results demonstrate that the AI model surpasses the diagnostic accuracy of experts in predicting lung development in preterm infants.
Collapse
Affiliation(s)
- Hao-Yang Chou
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, 70101, Taiwan
| | - Yung-Chieh Lin
- Department of Pediatrics, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, 704, Taiwan
| | - Sun-Yuan Hsieh
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, 70101, Taiwan
- Institution of Medical Informatics, National Cheng Kung University, Tainan, 70101, Taiwan
- Institute of Manufacturing Information and Systems, National Cheng Kung University, Tainan, 70101, Taiwan
- Department of Computer Science and Information Engineering, National Chi Nan University, Nantou, 54561, Taiwan
- Institute of Information Science, Academia Sinica, Taipei, 115, Taiwan
- Research Center for Information Technology Innovation, Academia Sinica, Taipei, 115, Taiwan
| | - Hsin-Hung Chou
- Department of Computer Science and Information Engineering, National Chi Nan University, Nantou, 54561, Taiwan.
| | - Cheng-Shih Lai
- Department of Medical Imaging, National Cheng Kung University Hospital, Tainan, 701401, Taiwan
| | - Bow Wang
- Department of Medical Imaging, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, 704, Taiwan
| | - Yi-Shan Tsai
- Department of Medical Imaging, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, 704, Taiwan.
| |
Collapse
|
6
|
Bennani S, Regnard NE, Ventre J, Lassalle L, Nguyen T, Ducarouge A, Dargent L, Guillo E, Gouhier E, Zaimi SH, Canniff E, Malandrin C, Khafagy P, Koulakian H, Revel MP, Chassagnon G. Using AI to Improve Radiologist Performance in Detection of Abnormalities on Chest Radiographs. Radiology 2023; 309:e230860. [PMID: 38085079 DOI: 10.1148/radiol.230860] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Background Chest radiography remains the most common radiologic examination, and interpretation of its results can be difficult. Purpose To explore the potential benefit of artificial intelligence (AI) assistance in the detection of thoracic abnormalities on chest radiographs by evaluating the performance of radiologists with different levels of expertise, with and without AI assistance. Materials and Methods Patients who underwent both chest radiography and thoracic CT within 72 hours between January 2010 and December 2020 in a French public hospital were screened retrospectively. Radiographs were randomly included until reaching 500 radiographs, with about 50% of radiographs having abnormal findings. A senior thoracic radiologist annotated the radiographs for five abnormalities (pneumothorax, pleural effusion, consolidation, mediastinal and hilar mass, lung nodule) based on the corresponding CT results (ground truth). A total of 12 readers (four thoracic radiologists, four general radiologists, four radiology residents) read half the radiographs without AI and half the radiographs with AI (ChestView; Gleamer). Changes in sensitivity and specificity were measured using paired t tests. Results The study included 500 patients (mean age, 54 years ± 19 [SD]; 261 female, 239 male), with 522 abnormalities visible on 241 radiographs. On average, for all readers, AI use resulted in an absolute increase in sensitivity of 26% (95% CI: 20, 32), 14% (95% CI: 11, 17), 12% (95% CI: 10, 14), 8.5% (95% CI: 6, 11), and 5.9% (95% CI: 4, 8) for pneumothorax, consolidation, nodule, pleural effusion, and mediastinal and hilar mass, respectively (P < .001). Specificity increased with AI assistance (3.9% [95% CI: 3.2, 4.6], 3.7% [95% CI: 3, 4.4], 2.9% [95% CI: 2.3, 3.5], and 2.1% [95% CI: 1.6, 2.6] for pleural effusion, mediastinal and hilar mass, consolidation, and nodule, respectively), except in the diagnosis of pneumothorax (-0.2%; 95% CI: -0.36, -0.04; P = .01). The mean reading time was 81 seconds without AI versus 56 seconds with AI (31% decrease, P < .001). Conclusion AI-assisted chest radiography interpretation resulted in absolute increases in sensitivity for all radiologists of various levels of expertise and reduced the reading times; specificity increased with AI, except in the diagnosis of pneumothorax. © RSNA, 2023 Supplemental material is available for this article.
Collapse
Affiliation(s)
- Souhail Bennani
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Nor-Eddine Regnard
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Jeanne Ventre
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Louis Lassalle
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Toan Nguyen
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Alexis Ducarouge
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Lucas Dargent
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Enora Guillo
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Elodie Gouhier
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Sophie-Hélène Zaimi
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Emma Canniff
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Cécile Malandrin
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Philippe Khafagy
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Hasmik Koulakian
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Marie-Pierre Revel
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Guillaume Chassagnon
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| |
Collapse
|
7
|
Shin H, Kim T, Park J, Raj H, Jabbar MS, Abebaw ZD, Lee J, Van CC, Kim H, Shin D. Pulmonary abnormality screening on chest x-rays from different machine specifications: a generalized AI-based image manipulation pipeline. Eur Radiol Exp 2023; 7:68. [PMID: 37940797 PMCID: PMC10632317 DOI: 10.1186/s41747-023-00386-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Accepted: 09/12/2023] [Indexed: 11/10/2023] Open
Abstract
BACKGROUND Chest x-ray is commonly used for pulmonary abnormality screening. However, since the image characteristics of x-rays highly depend on the machine specifications, an artificial intelligence (AI) model developed for specific equipment usually fails when clinically applied to various machines. To overcome this problem, we propose an image manipulation pipeline. METHODS A total of 15,010 chest x-rays from systems with different generators/detectors were retrospectively collected from five institutions from May 2020 to February 2021. We developed an AI model to classify pulmonary abnormalities using x-rays from a single system. Then, we externally tested its performance on chest x-rays from various machine specifications. We compared the area under the receiver operating characteristics curve (AUC) of AI models developed using conventional image processing pipelines (histogram equalization [HE], contrast-limited histogram equalization [CLAHE], and unsharp masking [UM] with common data augmentations) with that of the proposed manipulation pipeline (XM-pipeline). RESULTS The XM-pipeline model showed the highest performance for all the datasets of different machine specifications, such as chest x-rays acquired from a computed radiography system (n = 356, AUC 0.944 for XM-pipeline versus 0.917 for HE, 0.705 for CLAHE, 0.544 for UM, p [Formula: see text] 0.001, for all) and from a mobile x-ray generator (n = 204, AUC 0.949 for XM-pipeline versus 0.933 for HE, p = 0.042, 0.932 for CLAHE (p = 0.009), 0.925 for UM (p = 0.001). CONCLUSIONS Applying the XM-pipeline to AI training increased the diagnostic performance of the AI model on the chest x-rays of different machine configurations. RELEVANCE STATEMENT The proposed training pipeline would successfully promote a wide application of the AI model for abnormality screening when chest x-rays are acquired using various x-ray machines. KEY POINTS • AI models developed using x-rays of a specific machine suffer from generalization. • We proposed a new image processing pipeline to address the generalization problem. • AI models were tested using multicenter external x-ray datasets of various machines. • AI with our pipeline achieved the highest diagnostic performance than conventional methods.
Collapse
Affiliation(s)
- Heejun Shin
- Artificial Intelligence Engineering Division, RadiSen Co., Ltd, Seoul, Korea
| | - Taehee Kim
- Artificial Intelligence Engineering Division, RadiSen Co., Ltd, Seoul, Korea
| | - Juhyung Park
- Laboratory for Imaging Science and Technology, Department of Electrical and Computer Engineering, Seoul National University, Seoul, Korea
| | - Hruthvik Raj
- Artificial Intelligence Engineering Division, RadiSen Co., Ltd, Seoul, Korea
| | | | | | - Jongho Lee
- Laboratory for Imaging Science and Technology, Department of Electrical and Computer Engineering, Seoul National University, Seoul, Korea
| | - Cong Cung Van
- Department of Radiology, National Lung Hospital, Hanoi, Vietnam
| | - Hyungjin Kim
- Department of Radiology, Seoul National University Hospital, Seoul, Korea
| | - Dongmyung Shin
- Artificial Intelligence Engineering Division, RadiSen Co., Ltd, Seoul, Korea.
| |
Collapse
|
8
|
Buschmeyer K, Hatfield S, Zenner J. Psychological assessment of AI-based decision support systems: tool development and expected benefits. Front Artif Intell 2023; 6:1249322. [PMID: 37818430 PMCID: PMC10561554 DOI: 10.3389/frai.2023.1249322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Accepted: 09/05/2023] [Indexed: 10/12/2023] Open
Abstract
This study aimed to develop an evaluation tool that assesses the use of AI-based decision support systems (DSSs) in professional practice from a human-centered perspective. Following the International Organization for Standardization, this perspective aims to ensure that the use of interactive technologies improves users' psychological load experience and behavior, e.g., in the form of reduced stress experience or increased performance. Concomitantly, this perspective attempts to proactively prevent or detect and correct the potential negative effects of these technologies on user load, such as impaired satisfaction and engagement, as early as possible. Based on this perspective, we developed and validated a questionnaire instrument, the Psychological Assessment of AI-based DSSs (PAAI), for the user-centered evaluation of the use of AI-based DSSs in practice. In particular, the instrument considers central design characteristics of AI-based DSSs and the corresponding work situation, which have a significant impact on users' psychological load. The instrument was tested in two independent studies. In Study 1, N = 223 individuals were recruited. Based on the results of item and scale analyses and an exploratory factor analysis, the newly developed instrument was refined, and the final version was tested using a confirmatory factor analysis. Findings showed acceptable-to-good fit indices, confirming the factorial validity of the PAAI. This was confirmed in a second study, which had N = 471 participants. Again, the CFA yielded acceptable-to-good fit indices. The validity was further confirmed using convergent and criterion validity analyses.
Collapse
Affiliation(s)
- Katharina Buschmeyer
- Faculty of Business, Augsburg Technical University of Applied Science, Augsburg, Germany
| | - Sarah Hatfield
- Faculty of Business, Augsburg Technical University of Applied Science, Augsburg, Germany
| | - Julie Zenner
- Faculty of Liberal Arts and Science, Augsburg Technical University of Applied Science, Augsburg, Germany
| |
Collapse
|
9
|
Lind Plesner L, Müller FC, Brejnebøl MW, Laustrup LC, Rasmussen F, Nielsen OW, Boesen M, Brun Andersen M. Commercially Available Chest Radiograph AI Tools for Detecting Airspace Disease, Pneumothorax, and Pleural Effusion. Radiology 2023; 308:e231236. [PMID: 37750768 DOI: 10.1148/radiol.231236] [Citation(s) in RCA: 16] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/27/2023]
Abstract
Background Commercially available artificial intelligence (AI) tools can assist radiologists in interpreting chest radiographs, but their real-life diagnostic accuracy remains unclear. Purpose To evaluate the diagnostic accuracy of four commercially available AI tools for detection of airspace disease, pneumothorax, and pleural effusion on chest radiographs. Materials and Methods This retrospective study included consecutive adult patients who underwent chest radiography at one of four Danish hospitals in January 2020. Two thoracic radiologists (or three, in cases of disagreement) who had access to all previous and future imaging labeled chest radiographs independently for the reference standard. Area under the receiver operating characteristic curve, sensitivity, and specificity were calculated. Sensitivity and specificity were additionally stratified according to the severity of findings, number of findings on chest radiographs, and radiographic projection. The χ2 and McNemar tests were used for comparisons. Results The data set comprised 2040 patients (median age, 72 years [IQR, 58-81 years]; 1033 female), of whom 669 (32.8%) had target findings. The AI tools demonstrated areas under the receiver operating characteristic curve ranging 0.83-0.88 for airspace disease, 0.89-0.97 for pneumothorax, and 0.94-0.97 for pleural effusion. Sensitivities ranged 72%-91% for airspace disease, 63%-90% for pneumothorax, and 62%-95% for pleural effusion. Negative predictive values ranged 92%-100% for all target findings. In airspace disease, pneumothorax, and pleural effusion, specificity was high for chest radiographs with normal or single findings (range, 85%-96%, 99%-100%, and 95%-100%, respectively) and markedly lower for chest radiographs with four or more findings (range, 27%-69%, 96%-99%, 65%-92%, respectively) (P < .001). AI sensitivity was lower for vague airspace disease (range, 33%-61%) and small pneumothorax or pleural effusion (range, 9%-94%) compared with larger findings (range, 81%-100%; P value range, > .99 to < .001). Conclusion Current-generation AI tools showed moderate to high sensitivity for detecting airspace disease, pneumothorax, and pleural effusion on chest radiographs. However, they produced more false-positive findings than radiology reports, and their performance decreased for smaller-sized target findings and when multiple findings were present. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Yanagawa and Tomiyama in this issue.
Collapse
Affiliation(s)
- Louis Lind Plesner
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Felix C Müller
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Mathias W Brejnebøl
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Lene C Laustrup
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Finn Rasmussen
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Olav W Nielsen
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Mikael Boesen
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| | - Michael Brun Andersen
- From the Department of Radiology, Herlev and Gentofte Hospital, Borgmester Ib, Juuls vej 1 Herlev, Copenhagen 2730, Denmark (L.L.P., F.C.M., M.W.B., L.C.L., M.B.A.); Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark (L.L.P., M.W.B., O.W.N., M.B., M.B.A.); Radiological Artificial Intelligence Testcenter, RAIT.dk, Capital Region of Denmark (L.L.P., F.C.M., M.W.B., M.B., M.B.A.); Departments of Radiology (M.W.B., M.B.) and Cardiology (O.W.N.), Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; and Department of Radiology, Aarhus University Hospital, Aarhus, Denmark (F.R.)
| |
Collapse
|
10
|
Junet V, Matos‐Filipe P, García‐Illarramendi JM, Ramírez E, Oliva B, Farrés J, Daura X, Mas JM, Morales R. A decision support system based on artificial intelligence and systems biology for the simulation of pancreatic cancer patient status. CPT Pharmacometrics Syst Pharmacol 2023; 12:916-928. [PMID: 37002678 PMCID: PMC10349189 DOI: 10.1002/psp4.12961] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Revised: 01/23/2023] [Accepted: 03/13/2023] [Indexed: 04/04/2023] Open
Abstract
Oncology treatments require continuous individual adjustment based on the measurement of multiple clinical parameters. Prediction tools exploiting the patterns present in the clinical data could be used to assist decision making and ease the burden associated to the interpretation of all these parameters. The goal of this study was to predict the evolution of patients with pancreatic cancer at their next visit using information routinely recorded in health records, providing a decision-support system for clinicians. We selected hematological variables as the visit's clinical outcomes, under the assumption that they can be predictive of the evolution of the patient. Multivariate models based on regression trees were generated to predict next-visit values for each of the clinical outcomes selected, based on the longitudinal clinical data as well as on molecular data sets streaming from in silico simulations of individual patient status at each visit. The models predict, with a mean prediction score (balanced accuracy) of 0.79, the evolution trends of eosinophils, leukocytes, monocytes, and platelets. Time span between visits and neutropenia were among the most common factors contributing to the predicted evolution. The inclusion of molecular variables from the systems-biology in silico simulations provided a molecular background for the observed variations in the selected outcome variables, mostly in relation to the regulation of hematopoiesis. In spite of its limitations, this study serves as a proof of concept for the application of next-visit prediction tools in real-world settings, even when available data sets are small.
Collapse
Affiliation(s)
- Valentin Junet
- Anaxomics Biotech SLBarcelonaSpain
- Institute of Biotechnology and BiomedicineUniversitat Autònoma de BarcelonaCerdanyola del VallèsSpain
| | - Pedro Matos‐Filipe
- Anaxomics Biotech SLBarcelonaSpain
- Structural Bioinformatics (GRIB‐IMIM), Departament de Ciències Experimentals i de la SalutUniversitat Pompeu FabraBarcelonaSpain
| | - Juan Manuel García‐Illarramendi
- Anaxomics Biotech SLBarcelonaSpain
- Institute of Biotechnology and BiomedicineUniversitat Autònoma de BarcelonaCerdanyola del VallèsSpain
| | | | - Baldo Oliva
- Structural Bioinformatics (GRIB‐IMIM), Departament de Ciències Experimentals i de la SalutUniversitat Pompeu FabraBarcelonaSpain
| | | | - Xavier Daura
- Institute of Biotechnology and BiomedicineUniversitat Autònoma de BarcelonaCerdanyola del VallèsSpain
- Catalan Institution for Research and Advanced Studies (ICREA)BarcelonaSpain
- Biomedical Research Networking Center in Bioengineering, Biomaterials and Nanomedicine (CIBER‐BBN)Instituto de Salud Carlos IIICerdanyola del VallèsSpain
| | | | | |
Collapse
|
11
|
Ewals LJS, van der Wulp K, van den Borne BEEM, Pluyter JR, Jacobs I, Mavroeidis D, van der Sommen F, Nederend J. The Effects of Artificial Intelligence Assistance on the Radiologists' Assessment of Lung Nodules on CT Scans: A Systematic Review. J Clin Med 2023; 12:jcm12103536. [PMID: 37240643 DOI: 10.3390/jcm12103536] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Revised: 04/19/2023] [Accepted: 05/16/2023] [Indexed: 05/28/2023] Open
Abstract
To reduce the number of missed or misdiagnosed lung nodules on CT scans by radiologists, many Artificial Intelligence (AI) algorithms have been developed. Some algorithms are currently being implemented in clinical practice, but the question is whether radiologists and patients really benefit from the use of these novel tools. This study aimed to review how AI assistance for lung nodule assessment on CT scans affects the performances of radiologists. We searched for studies that evaluated radiologists' performances in the detection or malignancy prediction of lung nodules with and without AI assistance. Concerning detection, radiologists achieved with AI assistance a higher sensitivity and AUC, while the specificity was slightly lower. Concerning malignancy prediction, radiologists achieved with AI assistance generally a higher sensitivity, specificity and AUC. The radiologists' workflows of using the AI assistance were often only described in limited detail in the papers. As recent studies showed improved performances of radiologists with AI assistance, AI assistance for lung nodule assessment holds great promise. To achieve added value of AI tools for lung nodule assessment in clinical practice, more research is required on the clinical validation of AI tools, impact on follow-up recommendations and ways of using AI tools.
Collapse
Affiliation(s)
- Lotte J S Ewals
- Department of Radiology, Catharina Cancer Institute, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands
| | - Kasper van der Wulp
- Department of Radiology, Catharina Cancer Institute, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
| | - Ben E E M van den Borne
- Department of Pulmonology, Catharina Cancer Institute, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
| | - Jon R Pluyter
- Department of Experience Design, Royal Philips, 5656 AE Eindhoven, The Netherlands
| | - Igor Jacobs
- Department of Hospital Services and Informatics, Philips Research, 5656 AE Eindhoven, The Netherlands
| | - Dimitrios Mavroeidis
- Department of Data Science, Philips Research, 5656 AE Eindhoven, The Netherlands
| | - Fons van der Sommen
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands
| | - Joost Nederend
- Department of Radiology, Catharina Cancer Institute, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
| |
Collapse
|
12
|
Liu DS, Abu-Shaban K, Halabi SS, Cook TS. Changes in Radiology Due to Artificial Intelligence That Can Attract Medical Students to the Specialty. JMIR MEDICAL EDUCATION 2023; 9:e43415. [PMID: 36939823 PMCID: PMC10131993 DOI: 10.2196/43415] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Revised: 02/19/2023] [Accepted: 02/25/2023] [Indexed: 06/18/2023]
Abstract
The role of artificial intelligence (AI) in radiology has grown exponentially in the recent years. One of the primary worries by medical students is that AI will cause the roles of a radiologist to become automated and thus obsolete. Therefore, there is a greater hesitancy by medical students to choose radiology as a specialty. However, it is in this time of change that the specialty needs new thinkers and leaders. In this succinct viewpoint, 2 medical students involved in AI and 2 radiologists specializing in AI or clinical informatics posit that not only are these fears false, but the field of radiology will be transformed in such a way due to AI that there will be novel reasons to choose radiology. These new factors include greater impact on patient care, new space for innovation, interdisciplinary collaboration, increased patient contact, becoming master diagnosticians, and greater opportunity for global health initiatives, among others. Finally, since medical students view mentorship as a critical resource when deciding their career path, medical educators must also be cognizant of these changes and not give much credence to the prevalent fearmongering. As the field and practice of radiology continue to undergo significant change due to AI, it is urgent and necessary for the conversation to expand from expert to expert to expert to student. Medical students should be encouraged to choose radiology specifically because of the changes brought on by AI rather than being deterred by it.
Collapse
Affiliation(s)
- David Shalom Liu
- University of Toledo College of Medicine and Life Sciences, Toledo, OH, United States
| | - Kamil Abu-Shaban
- University of Toledo College of Medicine and Life Sciences, Toledo, OH, United States
| | - Safwan S Halabi
- Department of Medical Imaging, Ann & Robert H Lurie Children's Hospital of Chicago, Chicago, IL, United States
| | - Tessa Sundaram Cook
- Department of Radiology, Hospital of the University of Pennsylvania, Pennsylvania, PA, United States
| |
Collapse
|
13
|
Li D, Pehrson LM, Bonnevie R, Fraccaro M, Thrane J, Tøttrup L, Lauridsen CA, Butt Balaganeshan S, Jankovic J, Andersen TT, Mayar A, Hansen KL, Carlsen JF, Darkner S, Nielsen MB. Performance and Agreement When Annotating Chest X-ray Text Reports—A Preliminary Step in the Development of a Deep Learning-Based Prioritization and Detection System. Diagnostics (Basel) 2023; 13:diagnostics13061070. [PMID: 36980376 PMCID: PMC10047142 DOI: 10.3390/diagnostics13061070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Revised: 03/06/2023] [Accepted: 03/08/2023] [Indexed: 03/18/2023] Open
Abstract
A chest X-ray report is a communicative tool and can be used as data for developing artificial intelligence-based decision support systems. For both, consistent understanding and labeling is important. Our aim was to investigate how readers would comprehend and annotate 200 chest X-ray reports. Reports written between 1 January 2015 and 11 March 2022 were selected based on search words. Annotators included three board-certified radiologists, two trained radiologists (physicians), two radiographers (radiological technicians), a non-radiological physician, and a medical student. Consensus labels by two or more of the experienced radiologists were considered “gold standard”. Matthew’s correlation coefficient (MCC) was calculated to assess annotation performance, and descriptive statistics were used to assess agreement between individual annotators and labels. The intermediate radiologist had the best correlation to “gold standard” (MCC 0.77). This was followed by the novice radiologist and medical student (MCC 0.71 for both), the novice radiographer (MCC 0.65), non-radiological physician (MCC 0.64), and experienced radiographer (MCC 0.57). Our findings showed that for developing an artificial intelligence-based support system, if trained radiologists are not available, annotations from non-radiological annotators with basic and general knowledge may be more aligned with radiologists compared to annotations from sub-specialized medical staff, if their sub-specialization is outside of diagnostic radiology.
Collapse
Affiliation(s)
- Dana Li
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
- Correspondence:
| | - Lea Marie Pehrson
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Computer Science, University of Copenhagen, 2100 Copenhagen, Denmark
| | | | | | | | | | - Carsten Ammitzbøl Lauridsen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Radiography Education, University College Copenhagen, 2200 Copenhagen, Denmark
| | - Sedrah Butt Balaganeshan
- Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Jelena Jankovic
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Tobias Thostrup Andersen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Alyas Mayar
- Department of Health Sciences, Panum Institute, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Kristoffer Lindskov Hansen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Jonathan Frederik Carlsen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Sune Darkner
- Department of Computer Science, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Michael Bachmann Nielsen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| |
Collapse
|
14
|
Chamberlin JH, Smith C, Schoepf UJ, Nance S, Elojeimy S, O'Doherty J, Baruah D, Burt JR, Varga-Szemes A, Kabakus IM. A deep convolutional neural network ensemble for composite identification of pulmonary nodules and incidental findings on routine PET/CT. Clin Radiol 2023; 78:e368-e376. [PMID: 36863883 DOI: 10.1016/j.crad.2023.01.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Revised: 10/19/2022] [Accepted: 01/30/2023] [Indexed: 02/18/2023]
Abstract
AIM To evaluate primary and secondary pathologies of interest using an artificial intelligence (AI) platform, AI-Rad Companion, on low-dose computed tomography (CT) series from integrated positron-emission tomography (PET)/CT to detect CT findings that might be overlooked. MATERIALS AND METHODS One hundred and eighty-nine sequential patients who had undergone PET/CT were included. Images were evaluated using an ensemble of convolutional neural networks (AI-Rad Companion, Siemens Healthineers, Erlangen, Germany). The primary outcome was detection of pulmonary nodules for which the accuracy, identity, and intra-rater reliability was calculated. For secondary outcomes (binary detection of coronary artery calcium, aortic ectasia, vertebral height loss), accuracy and diagnostic performance were calculated. RESULTS The overall per-nodule accuracy for detection of lung nodules was 0.847. The overall sensitivity and specificity for detection of lung nodules was 0.915 and 0.781. The overall per-patient accuracy for AI detection of coronary artery calcium, aortic ectasia, and vertebral height loss was 0.979, 0.966, and 0.840, respectively. The sensitivity and specificity for coronary artery calcium was 0.989 and 0.969. The sensitivity and specificity for aortic ectasia was 0.806 and 1. CONCLUSION The neural network ensemble accurately assessed the number of pulmonary nodules and presence of coronary artery calcium and aortic ectasia on low-dose CT series of PET/CT. The neural network was highly specific for the diagnosis of vertebral height loss, but not sensitive. The use of the AI ensemble can help radiologists and nuclear medicine physicians to catch CT findings that might be overlooked.
Collapse
Affiliation(s)
- J H Chamberlin
- Division of Thoracic Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
| | - C Smith
- Division of Thoracic Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
| | - U J Schoepf
- Division of Cardiovascular Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
| | - S Nance
- Division of Thoracic Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
| | - S Elojeimy
- Division of Nuclear Medicine, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
| | - J O'Doherty
- Division of Cardiovascular Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA; Siemens Medical Solutions, Malvern, PA, USA
| | - D Baruah
- Division of Thoracic Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
| | - J R Burt
- Division of Thoracic Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
| | - A Varga-Szemes
- Division of Cardiovascular Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
| | - I M Kabakus
- Division of Thoracic Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA; Division of Cardiovascular Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA; Division of Nuclear Medicine, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA.
| |
Collapse
|
15
|
Gambato M, Scotti N, Borsari G, Zambon Bertoja J, Gabrieli JD, De Cassai A, Cester G, Navalesi P, Quaia E, Causin F. Chest X-ray Interpretation: Detecting Devices and Device-Related Complications. Diagnostics (Basel) 2023; 13:599. [PMID: 36832087 PMCID: PMC9954842 DOI: 10.3390/diagnostics13040599] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Revised: 02/01/2023] [Accepted: 02/04/2023] [Indexed: 02/10/2023] Open
Abstract
This short review has the aim of helping the radiologist to identify medical devices when interpreting a chest X-ray, as well as looking for their most commonly detectable complications. Nowadays, many different medical devices are used, often together, especially in critical patients. It is important for the radiologist to know what to look for and to remember the technical factors that need to be considered when checking each device's positioning.
Collapse
Affiliation(s)
- Marco Gambato
- Institute of Radiology, Department of Medicine (DIMED), University of Padova, 35121 Padua, Italy
| | - Nicola Scotti
- Institute of Radiology, Department of Medicine (DIMED), University of Padova, 35121 Padua, Italy
| | - Giacomo Borsari
- Institute of Radiology, Department of Medicine (DIMED), University of Padova, 35121 Padua, Italy
| | - Jacopo Zambon Bertoja
- Institute of Radiology, Department of Medicine (DIMED), University of Padova, 35121 Padua, Italy
| | | | - Alessandro De Cassai
- Anesthesia and Intensive Care Unit, University Hospital of Padova, 35121 Padua, Italy
| | - Giacomo Cester
- Department of Neuroradiology, University Hospital of Padova, 35121 Padua, Italy
| | - Paolo Navalesi
- Anesthesia and Intensive Care Unit, Department of Medicine (DIMED), University of Padova, 35121 Padua, Italy
| | - Emilio Quaia
- Institute of Radiology, Department of Medicine (DIMED), University of Padova, 35121 Padua, Italy
- Institute of Radiology, University Hospital of Padova, 35121 Padua, Italy
| | - Francesco Causin
- Department of Neuroradiology, University Hospital of Padova, 35121 Padua, Italy
| |
Collapse
|
16
|
Gaube S, Suresh H, Raue M, Lermer E, Koch TK, Hudecek MFC, Ackery AD, Grover SC, Coughlin JF, Frey D, Kitamura FC, Ghassemi M, Colak E. Non-task expert physicians benefit from correct explainable AI advice when reviewing X-rays. Sci Rep 2023; 13:1383. [PMID: 36697450 PMCID: PMC9876883 DOI: 10.1038/s41598-023-28633-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Accepted: 01/20/2023] [Indexed: 01/26/2023] Open
Abstract
Artificial intelligence (AI)-generated clinical advice is becoming more prevalent in healthcare. However, the impact of AI-generated advice on physicians' decision-making is underexplored. In this study, physicians received X-rays with correct diagnostic advice and were asked to make a diagnosis, rate the advice's quality, and judge their own confidence. We manipulated whether the advice came with or without a visual annotation on the X-rays, and whether it was labeled as coming from an AI or a human radiologist. Overall, receiving annotated advice from an AI resulted in the highest diagnostic accuracy. Physicians rated the quality of AI advice higher than human advice. We did not find a strong effect of either manipulation on participants' confidence. The magnitude of the effects varied between task experts and non-task experts, with the latter benefiting considerably from correct explainable AI advice. These findings raise important considerations for the deployment of diagnostic advice in healthcare.
Collapse
Affiliation(s)
- Susanne Gaube
- LMU Center for Leadership and People Management, Department of Psychology, LMU Munich, Munich, Germany. .,Department of Infection Prevention and Infectious Diseases, University Hospital Regensburg, Regensburg, Germany.
| | - Harini Suresh
- MIT Computer Science & Artificial Intelligence Lab, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Martina Raue
- MIT AgeLab, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Eva Lermer
- LMU Center for Leadership and People Management, Department of Psychology, LMU Munich, Munich, Germany.,Department of Business Psychology, University of Applied Sciences Augsburg, Augsburg, Germany
| | - Timo K Koch
- LMU Center for Leadership and People Management, Department of Psychology, LMU Munich, Munich, Germany.,Department of Psychology, LMU Munich, Munich, Germany
| | - Matthias F C Hudecek
- Department of Experimental Psychology, University of Regensburg, Regensburg, Germany
| | - Alun D Ackery
- Department of Emergency Medicine, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada.,Division of Emergency Medicine, University of Toronto, Toronto, Canada
| | - Samir C Grover
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada.,Division of Gastroenterology, St. Michael's Hospital, Toronto, Canada
| | - Joseph F Coughlin
- MIT AgeLab, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Dieter Frey
- LMU Center for Leadership and People Management, Department of Psychology, LMU Munich, Munich, Germany
| | - Felipe C Kitamura
- Departamento de Diagnóstico por Imagem, Universidade Federal de São Paulo, São Paulo, Brazil.,DasaInova, Dasa, São Paulo, Brazil
| | - Marzyeh Ghassemi
- Electrical Engineering and Computer Science, Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA.,Vector Institute, Toronto, Canada
| | - Errol Colak
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada.,Department of Medical Imaging, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada.,Department of Medical Imaging, Faculty of Medicine, University of Toronto, Toronto, Canada
| |
Collapse
|
17
|
Irmici G, Cè M, Caloro E, Khenkina N, Della Pepa G, Ascenti V, Martinenghi C, Papa S, Oliva G, Cellina M. Chest X-ray in Emergency Radiology: What Artificial Intelligence Applications Are Available? Diagnostics (Basel) 2023; 13:diagnostics13020216. [PMID: 36673027 PMCID: PMC9858224 DOI: 10.3390/diagnostics13020216] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2022] [Revised: 12/28/2022] [Accepted: 01/03/2023] [Indexed: 01/11/2023] Open
Abstract
Due to its widespread availability, low cost, feasibility at the patient's bedside and accessibility even in low-resource settings, chest X-ray is one of the most requested examinations in radiology departments. Whilst it provides essential information on thoracic pathology, it can be difficult to interpret and is prone to diagnostic errors, particularly in the emergency setting. The increasing availability of large chest X-ray datasets has allowed the development of reliable Artificial Intelligence (AI) tools to help radiologists in everyday clinical practice. AI integration into the diagnostic workflow would benefit patients, radiologists, and healthcare systems in terms of improved and standardized reporting accuracy, quicker diagnosis, more efficient management, and appropriateness of the therapy. This review article aims to provide an overview of the applications of AI for chest X-rays in the emergency setting, emphasizing the detection and evaluation of pneumothorax, pneumonia, heart failure, and pleural effusion.
Collapse
Affiliation(s)
- Giovanni Irmici
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Maurizio Cè
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Elena Caloro
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Natallia Khenkina
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Gianmarco Della Pepa
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Velio Ascenti
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Carlo Martinenghi
- Radiology Department, San Raffaele Hospital, Via Olgettina 60, 20132 Milan, Italy
| | - Sergio Papa
- Unit of Diagnostic Imaging and Stereotactic Radiosurgery, Centro Diagnostico Italiano, Via Saint Bon 20, 20147 Milan, Italy
| | - Giancarlo Oliva
- Radiology Department, Fatebenefratelli Hospital, ASST Fatebenefratelli Sacco, Milano, Piazza Principessa Clotilde 3, 20121 Milan, Italy
| | - Michaela Cellina
- Radiology Department, Fatebenefratelli Hospital, ASST Fatebenefratelli Sacco, Milano, Piazza Principessa Clotilde 3, 20121 Milan, Italy
| |
Collapse
|
18
|
Impact of Artificial Intelligence Assistance on Chest CT Interpretation Times: A Prospective Randomized Study. AJR Am J Roentgenol 2022; 219:743-751. [PMID: 35703413 DOI: 10.2214/ajr.22.27598] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
BACKGROUND. Deep learning-based convolutional neural networks have enabled major advances in development of artificial intelligence (AI) software applications. Modern AI applications offer comprehensive multiorgan evaluation. OBJECTIVE. The purpose of this article was to evaluate the impact of an automated AI platform integrated into clinical workflow for chest CT interpretation on radiologists' interpretation times when evaluated in a real-world clinical setting. METHODS. In this prospective single-center study, a commercial AI software solution was integrated into clinical workflow for chest CT interpretation. The software provided automated analysis of cardiac, pulmonary, and musculoskeletal findings, including labeling, segmenting, and measuring normal structures as well as detecting, labeling, and measuring abnormalities. AI-annotated images and autogenerated summary results were stored in the PACS and available to interpreting radiologists. A total of 390 patients (204 women, 186 men; mean age, 62.8 ± 13.3 [SD] years) who underwent out-patient chest CT between January 19, 2021, and January 28, 2021, were included. Scans were randomized using 1:1 allocation between AI-assisted and non-AI-assisted arms and were clinically interpreted by one of three cardiothoracic radiologists (65 scans per arm per radiologist; total of 195 scans per arm) who recorded interpretation times using a stopwatch. Findings were categorized according to review of report impressions. Interpretation times were compared between arms. RESULTS. Mean interpretation times were significantly shorter in the AI-assisted than in the non-AI-assisted arm for all three readers (289 ± 89 vs 344 ± 129 seconds, p < .001; 449 ± 110 vs 649 ± 82 seconds, p < .001; 281 ± 114 vs 348 ± 93 seconds, p = .01) and for readers combined (328 ± 122 vs 421 ± 175 seconds, p < .001). For readers combined, the mean difference was 93 seconds (95% CI, 63-123 seconds), corresponding with a 22.1% reduction in the AI-assisted arm. Mean interpretation time was also shorter in the AI-assisted arm compared with the non-AI-assisted arm for contrast-enhanced scans (83 seconds), noncontrast scans (104 seconds), negative scans (84 seconds), positive scans without significant new findings (117 seconds), and positive scans with significant new findings (92 seconds). CONCLUSION. Cardiothoracic radiologists exhibited a 22.1% reduction in chest CT interpretations times when they had access to results from an automated AI support platform during real-world clinical practice. CLINICAL IMPACT. Integration of the AI support platform into clinical workflow improved radiologist efficiency.
Collapse
|
19
|
Rajaraman S, Antani S. Editorial on Special Issue "Artificial Intelligence in Image-Based Screening, Diagnostics, and Clinical Care of Cardiopulmonary Diseases". Diagnostics (Basel) 2022; 12:2615. [PMID: 36359459 PMCID: PMC9689170 DOI: 10.3390/diagnostics12112615] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Accepted: 10/25/2022] [Indexed: 09/08/2024] Open
Abstract
Cardiopulmonary diseases are a significant cause of mortality and morbidity worldwide [...].
Collapse
Affiliation(s)
| | - Sameer Antani
- Computational Health Research Branch, National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20894, USA
| |
Collapse
|
20
|
Culié D, Schiappa R, Contu S, Scheller B, Villarme A, Dassonville O, Poissonnet G, Bozec A, Chamorey E. Validation and Improvement of a Convolutional Neural Network to Predict the Involved Pathology in a Head and Neck Surgery Cohort. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:12200. [PMID: 36231500 PMCID: PMC9564535 DOI: 10.3390/ijerph191912200] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 09/19/2022] [Accepted: 09/22/2022] [Indexed: 06/16/2023]
Abstract
The selection of patients for the constitution of a cohort is a major issue for clinical research (prospective studies and retrospective studies in real life). Our objective was to validate in real life conditions the use of a Deep Learning process based on a neural network, for the classification of patients according to the pathology involved in a head and neck surgery department. 24,434 Electronic Health Records (EHR) from the first visit between 2000 and 2020 were extracted. More than 6000 EHR were manually classified in ten groups of interest according to the reason for consultation with a clinical relevance. A convolutional neural network (TensorFlow, previously reported by Hsu et al.) was then used to predict the group of patients based on their pathology, using two levels of classification based on clinically relevant criteria. On the first and second level of classification, macro-average performances were: 0.95, 0.83, 0.85, 0.97, 0.84 and 0.93, 0.76, 0.83, 0.96, 0.79 for accuracy, recall, precision, specificity and F1-score versus accuracy, recall and precision of 0.580, 580 and 0.582 for Hsu et al., respectively. We validated this model to predict the pathology involved and to constitute clinically relevant cohorts in a tertiary hospital. This model did not require a preprocessing stage, was used in French and showed equivalent or better performances than other already published techniques.
Collapse
Affiliation(s)
- Dorian Culié
- Head and Neck Surgery Department, Antoine Laccassagne Center, 06100 Nice, France
- Epidemiology, Biostatistics and Health Data Department, Antoine Laccassagne Center, 06100 Nice, France
| | - Renaud Schiappa
- Epidemiology, Biostatistics and Health Data Department, Antoine Laccassagne Center, 06100 Nice, France
| | - Sara Contu
- Epidemiology, Biostatistics and Health Data Department, Antoine Laccassagne Center, 06100 Nice, France
| | - Boris Scheller
- Head and Neck Surgery Department, Antoine Laccassagne Center, 06100 Nice, France
- Epidemiology, Biostatistics and Health Data Department, Antoine Laccassagne Center, 06100 Nice, France
| | - Agathe Villarme
- Head and Neck Surgery Department, Antoine Laccassagne Center, 06100 Nice, France
| | - Olivier Dassonville
- Head and Neck Surgery Department, Antoine Laccassagne Center, 06100 Nice, France
| | - Gilles Poissonnet
- Head and Neck Surgery Department, Antoine Laccassagne Center, 06100 Nice, France
| | - Alexandre Bozec
- Head and Neck Surgery Department, Antoine Laccassagne Center, 06100 Nice, France
- Epidemiology, Biostatistics and Health Data Department, Antoine Laccassagne Center, 06100 Nice, France
| | - Emmanuel Chamorey
- Epidemiology, Biostatistics and Health Data Department, Antoine Laccassagne Center, 06100 Nice, France
| |
Collapse
|