1
|
Peters S, Kellermann G, Watkinson J, Gärtner F, Huhndorf M, Stürner K, Jansen O, Larsen N. AI supported detection of cerebral multiple sclerosis lesions decreases radiologic reporting times. Eur J Radiol 2024; 178:111638. [PMID: 39067268 DOI: 10.1016/j.ejrad.2024.111638] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2024] [Revised: 06/10/2024] [Accepted: 07/16/2024] [Indexed: 07/30/2024]
Abstract
PURPOSE Multiple Sclerosis (MS) is a common autoimmune disease of the central nervous system. MRI plays a crucial role in diagnosing as well as in disease and treatment monitoring. Therefore, evaluation of cerebral MRI of MS patients is part of daily clinical routine. A growing number of companies offer commercial software to support the reporting with automated lesion detection. Aim of this study was to evaluate the effect of such a software with AI supported lesion detection to the radiologic reporting. METHOD Four radiologist each counted MS-lesions in MRI examinations of 50 patients separated by the locations periventricular, cortical/juxtacortical, infrantentorial and unspecific white matter. After at least six weeks they repeated the evaluation, this time using the AI based software mdbrain for lesion detection. In both settings the required time was documented. Further the radiologists evaluated follow-up MRI of 50 MS-patients concerning new and enlarging lesions in the same manner. RESULTS To determine the lesion-load the average reporting time decreased from 286.85 sec to 196.34 sec (p > 0.001). For the evaluation of the follow-up images the reporting time dropped from 196.17 sec to 120.87 sec (p < 0.001). The interrater reliabilities showed no significant differences for the determination of lesion-load (0.83 without vs. 0.8 with software support) and for the detection of new/enlarged lesions (0.92 without vs. 0.82 with software support). CONCLUSION For the evaluation of MR images of MS patients, an AI-based support for image-interpretation can significantly decreases reporting times.
Collapse
Affiliation(s)
- Sönke Peters
- Department of Radiology and Neuroradiology, University Hospital of Schleswig-Holstein, Campus Kiel, Germany.
| | - Gesa Kellermann
- Department of Radiology, Bundeswehr Hospital Hamburg, Germany
| | - Joe Watkinson
- Department of Radiology and Neuroradiology, University Hospital of Schleswig-Holstein, Campus Kiel, Germany
| | - Friederike Gärtner
- Department of Radiology and Neuroradiology, University Hospital of Schleswig-Holstein, Campus Kiel, Germany
| | - Monika Huhndorf
- Department of Radiology and Neuroradiology, University Hospital of Schleswig-Holstein, Campus Kiel, Germany
| | - Klarissa Stürner
- Department of Neurology, University Hospital of Schleswig-Holstein, Campus Kiel, Germany
| | - Olav Jansen
- Department of Radiology and Neuroradiology, University Hospital of Schleswig-Holstein, Campus Kiel, Germany
| | - Naomi Larsen
- Department of Radiology and Neuroradiology, University Hospital of Schleswig-Holstein, Campus Kiel, Germany
| |
Collapse
|
2
|
Shelmerdine SC. Rethinking our relationship with AI: for better or worse, richer or poorer? Eur Radiol 2024:10.1007/s00330-024-11007-9. [PMID: 39095603 DOI: 10.1007/s00330-024-11007-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2024] [Revised: 07/01/2024] [Accepted: 07/02/2024] [Indexed: 08/04/2024]
Affiliation(s)
- Susan C Shelmerdine
- Department of Clinical Radiology, Great Ormond Street Hospital for Children, London, UK.
- UCL Great Ormond Street Institute of Child Health, Great Ormond Street Hospital for Children, London, UK.
- NIHR Great Ormond Street Hospital Biomedical Research Centre, Bloomsbury, London, UK.
| |
Collapse
|
3
|
Brady AP, Allen B, Chong J, Kotter E, Kottler N, Mongan J, Oakden-Rayner L, Pinto Dos Santos D, Tang A, Wald C, Slavotinek J. Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement From the ACR, CAR, ESR, RANZCR & RSNA. J Am Coll Radiol 2024; 21:1292-1310. [PMID: 38276923 DOI: 10.1016/j.jacr.2023.12.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2024]
Abstract
Artificial intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools. KEY POINTS.
Collapse
Affiliation(s)
| | - Bibb Allen
- Department of Radiology, Grandview Medical Center, Birmingham, Alabama; American College of Radiology Data Science Institute, Reston, Virginia
| | - Jaron Chong
- Department of Medical Imaging, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Elmar Kotter
- Department of Diagnostic and Interventional Radiology, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Nina Kottler
- Radiology Partners, El Segundo, California; Stanford Center for Artificial Intelligence in Medicine & Imaging, Palo Alto, California
| | - John Mongan
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, California
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, Australia
| | - Daniel Pinto Dos Santos
- Department of Radiology, University Hospital of Cologne, Cologne, Germany; Department of Radiology, University Hospital of Frankfurt, Frankfurt, Germany
| | - An Tang
- Department of Radiology, Radiation Oncology, and Nuclear Medicine, Université de Montréal, Montréal, Québec, Canada
| | - Christoph Wald
- Department of Radiology, Lahey Hospital & Medical Center, Burlington, Massachusetts; Tufts University Medical School, Boston, Massachusetts; Commision on Informatics, and Member, Board of Chancellors, American College of Radiology, Virginia
| | - John Slavotinek
- South Australia Medical Imaging, Flinders Medical Centre Adelaide, Adelaide, Australia; College of Medicine and Public Health, Flinders University, Adelaide, Australia
| |
Collapse
|
4
|
Hwang EJ. [Clinical Application of Artificial Intelligence-Based Detection Assistance Devices for Chest X-Ray Interpretation: Current Status and Practical Considerations]. JOURNAL OF THE KOREAN SOCIETY OF RADIOLOGY 2024; 85:693-704. [PMID: 39130790 PMCID: PMC11310435 DOI: 10.3348/jksr.2024.0052] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/12/2024] [Revised: 06/14/2024] [Accepted: 07/04/2024] [Indexed: 08/13/2024]
Abstract
Artificial intelligence (AI) technology is actively being applied for the interpretation of medical imaging, such as chest X-rays. AI-based software medical devices, which automatically detect various types of abnormal findings in chest X-ray images to assist physicians in their interpretation, are actively being commercialized and clinically implemented in Korea. Several important issues need to be considered for AI-based detection assistant tools to be applied in clinical practice: the evaluation of performance and efficacy prior to implementation; the determination of the target application, range, and method of delivering results; and monitoring after implementation and legal liability issues. Appropriate decision making regarding these devices based on the situation in each institution is necessary. Radiologists must be engaged as medical assessment experts using the software for these devices as well as in medical image interpretation to ensure the safe and efficient implementation and operation of AI-based detection assistant tools.
Collapse
|
5
|
Hwang EJ, Park JE, Song KD, Yang DH, Kim KW, Lee JG, Yoon JH, Han K, Kim DH, Kim H, Park CM. 2023 Survey on User Experience of Artificial Intelligence Software in Radiology by the Korean Society of Radiology. Korean J Radiol 2024; 25:613-622. [PMID: 38942455 PMCID: PMC11214921 DOI: 10.3348/kjr.2023.1246] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Revised: 04/24/2024] [Accepted: 04/26/2024] [Indexed: 06/30/2024] Open
Abstract
OBJECTIVE In Korea, radiology has been positioned towards the early adoption of artificial intelligence-based software as medical devices (AI-SaMDs); however, little is known about the current usage, implementation, and future needs of AI-SaMDs. We surveyed the current trends and expectations for AI-SaMDs among members of the Korean Society of Radiology (KSR). MATERIALS AND METHODS An anonymous and voluntary online survey was open to all KSR members between April 17 and May 15, 2023. The survey was focused on the experiences of using AI-SaMDs, patterns of usage, levels of satisfaction, and expectations regarding the use of AI-SaMDs, including the roles of the industry, government, and KSR regarding the clinical use of AI-SaMDs. RESULTS Among the 370 respondents (response rate: 7.7% [370/4792]; 340 board-certified radiologists; 210 from academic institutions), 60.3% (223/370) had experience using AI-SaMDs. The two most common use-case of AI-SaMDs among the respondents were lesion detection (82.1%, 183/223), lesion diagnosis/classification (55.2%, 123/223), with the target imaging modalities being plain radiography (62.3%, 139/223), CT (42.6%, 95/223), mammography (29.1%, 65/223), and MRI (28.7%, 64/223). Most users were satisfied with AI-SaMDs (67.6% [115/170, for improvement of patient management] to 85.1% [189/222, for performance]). Regarding the expansion of clinical applications, most respondents expressed a preference for AI-SaMDs to assist in detection/diagnosis (77.0%, 285/370) and to perform automated measurement/quantification (63.5%, 235/370). Most respondents indicated that future development of AI-SaMDs should focus on improving practice efficiency (81.9%, 303/370) and quality (71.4%, 264/370). Overall, 91.9% of the respondents (340/370) agreed that there is a need for education or guidelines driven by the KSR regarding the use of AI-SaMDs. CONCLUSION The penetration rate of AI-SaMDs in clinical practice and the corresponding satisfaction levels were high among members of the KSR. Most AI-SaMDs have been used for lesion detection, diagnosis, and classification. Most respondents requested KSR-driven education or guidelines on the use of AI-SaMDs.
Collapse
Affiliation(s)
- Eui Jin Hwang
- Department of Radiology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Ji Eun Park
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea
| | - Kyoung Doo Song
- Department of Radiology and Center for Imaging Science, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Dong Hyun Yang
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea
| | - Kyung Won Kim
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea
| | - June-Goo Lee
- Biomedical Engineering Research Center, Asan Institute for Life Sciences, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Jung Hyun Yoon
- Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Kyunghwa Han
- Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Dong Hyun Kim
- Department of Radiology, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Hwiyoung Kim
- Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Chang Min Park
- Department of Radiology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
6
|
Waugh J, Evans J, Miocevic M, Lockie D, Aminzadeh P, Lynch A, Bell RJ. Performance of artificial intelligence in 7533 consecutive prevalent screening mammograms from the BreastScreen Australia program. Eur Radiol 2024; 34:3947-3957. [PMID: 37955669 PMCID: PMC11166844 DOI: 10.1007/s00330-023-10396-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Revised: 08/15/2023] [Accepted: 09/05/2023] [Indexed: 11/14/2023]
Abstract
OBJECTIVES To assess the performance of an artificial intelligence (AI) algorithm in the Australian mammography screening program which routinely uses two independent readers with arbitration of discordant results. METHODS A total of 7533 prevalent round mammograms from 2017 were available for analysis. The AI program classified mammograms into deciles on the basis of breast cancer (BC) risk. BC diagnoses, including invasive BC (IBC) and ductal carcinoma in situ (DCIS), included those from the prevalent round, interval cancers, and cancers identified in the subsequent screening round two years later. Performance was assessed by sensitivity, specificity, positive and negative predictive values, and the proportion of women recalled by the radiologists and identified as higher risk by AI. RESULTS Radiologists identified 54 women with IBC and 13 with DCIS with a recall rate of 9.7%. In contrast, 51 of 54 of the IBCs and 12/13 cases of DCIS were within the higher AI score group (score 10), a recall equivalent of 10.6% (a difference of 0.9% (CI -0.03 to 1.89%, p = 0.06). When IBCs were identified in the 2017 round, interval cancers classified as false negatives or with minimal signs in 2017, and cancers from the 2019 round were combined, the radiologists identified 54/67 and 59/67 were in the highest risk AI category (sensitivity 80.6% and 88.06 % respectively, a difference that was not different statistically). CONCLUSIONS As the performance of AI was comparable to that of expert radiologists, future AI roles in screening could include replacing one reader and supporting arbitration, reducing workload and false positive results. CLINICAL RELEVANCE STATEMENT AI analysis of consecutive prevalent screening mammograms from the Australian BreastScreen program demonstrated the algorithm's ability to match the cancer detection of experienced radiologists, additionally identifying five interval cancers (false negatives), and the majority of the false positive recalls. KEY POINTS • The AI program was almost as sensitive as the radiologists in terms of identifying prevalent lesions (51/54 for invasive breast cancer, 63/67 when including ductal carcinoma in situ). • If selected interval cancers and cancers identified in the subsequent screening round were included, the AI program identified more cancers than the radiologists (59/67 compared with 54/67, sensitivity 88.06 % and 80.6% respectively p = 0.24). • The high negative predictive value of a score of 1-9 would indicate a role for AI as a triage tool to reduce the recall rate (specifically false positives).
Collapse
Affiliation(s)
- John Waugh
- Monash BreastScreen, Monash Cancer Centre, Moorabbin Hospital, 823-865 Centre Road, Bentleigh East, Victoria, 3165, Australia.
| | - Jill Evans
- Monash BreastScreen, Monash Cancer Centre, Moorabbin Hospital, 823-865 Centre Road, Bentleigh East, Victoria, 3165, Australia
| | - Miranda Miocevic
- Monash BreastScreen, Monash Cancer Centre, Moorabbin Hospital, 823-865 Centre Road, Bentleigh East, Victoria, 3165, Australia
| | - Darren Lockie
- Monash BreastScreen, Monash Cancer Centre, Moorabbin Hospital, 823-865 Centre Road, Bentleigh East, Victoria, 3165, Australia
| | - Parisa Aminzadeh
- Monash BreastScreen, Monash Cancer Centre, Moorabbin Hospital, 823-865 Centre Road, Bentleigh East, Victoria, 3165, Australia
| | - Anne Lynch
- Monash BreastScreen, Monash Cancer Centre, Moorabbin Hospital, 823-865 Centre Road, Bentleigh East, Victoria, 3165, Australia
| | - Robin J Bell
- Women's Health Research Program, School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, 3004, Australia
| |
Collapse
|
7
|
Trieu PDY, Barron ML, Jiang Z, Tavakoli Taba S, Gandomkar Z, Lewis SJ. Familiarity, confidence and preference of artificial intelligence feedback and prompts by Australian breast cancer screening readers. AUST HEALTH REV 2024; 48:299-311. [PMID: 38692648 DOI: 10.1071/ah23275] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Accepted: 04/05/2024] [Indexed: 05/03/2024]
Abstract
Objectives This study explored the familiarity, perceptions and confidence of Australian radiology clinicians involved in reading screening mammograms, regarding artificial intelligence (AI) applications in breast cancer detection. Methods Sixty-five radiologists, breast physicians and radiology trainees participated in an online survey that consisted of 23 multiple choice questions asking about their experience and familiarity with AI products. Furthermore, the survey asked about their confidence in using AI outputs and their preference for AI modes applied in a breast screening context. Participants' responses to questions were compared using Pearson's χ 2 test. Bonferroni-adjusted significance tests were used for pairwise comparisons. Results Fifty-five percent of respondents had experience with AI in their workplaces, with automatic density measurement powered by machine learning being the most familiar AI product (69.4%). The top AI outputs with the highest ranks of perceived confidence were 'Displaying suspicious areas on mammograms with the percentage of cancer possibility' (67.8%) and 'Automatic mammogram classification (normal, benign, cancer, uncertain)' (64.6%). Radiology and breast physicians preferred using AI as second-reader mode (75.4% saying 'somewhat happy' to 'extremely happy') over triage (47.7%), pre-screening and first-reader modes (both with 26.2%) (P < 0.001). Conclusion The majority of screen readers expressed increased confidence in utilising AI for highlighting suspicious areas on mammograms and for automatically classifying mammograms. They considered AI as an optimal second-reader mode being the most ideal use in a screening program. The findings provide valuable insights into the familiarities and expectations of radiologists and breast clinicians for the AI products that can enhance the effectiveness of the breast cancer screening programs, benefitting both healthcare professionals and patients alike.
Collapse
Affiliation(s)
- Phuong Dung Yun Trieu
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, D18- Level 7 - Susan Wakil Health Building, Camperdown, NSW 2006, Australia
| | - Melissa L Barron
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, D18- Level 7 - Susan Wakil Health Building, Camperdown, NSW 2006, Australia
| | - Zhengqiang Jiang
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, D18- Level 7 - Susan Wakil Health Building, Camperdown, NSW 2006, Australia
| | - Seyedamir Tavakoli Taba
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, D18- Level 7 - Susan Wakil Health Building, Camperdown, NSW 2006, Australia
| | - Ziba Gandomkar
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, D18- Level 7 - Susan Wakil Health Building, Camperdown, NSW 2006, Australia
| | - Sarah J Lewis
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, D18- Level 7 - Susan Wakil Health Building, Camperdown, NSW 2006, Australia; and School of Health Sciences, Western Sydney University, University Drive, Campbelltown, Locked Bag 1797, Penrith, NSW 2751, Australia
| |
Collapse
|
8
|
Stogiannos N, Litosseliti L, O'Regan T, Scurr E, Barnes A, Kumar A, Malik R, Pogose M, Harvey H, McEntee MF, Malamateniou C. Black box no more: A cross-sectional multi-disciplinary survey for exploring governance and guiding adoption of AI in medical imaging and radiotherapy in the UK. Int J Med Inform 2024; 186:105423. [PMID: 38531254 DOI: 10.1016/j.ijmedinf.2024.105423] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Revised: 03/12/2024] [Accepted: 03/20/2024] [Indexed: 03/28/2024]
Abstract
BACKGROUND Medical Imaging and radiotherapy (MIRT) are at the forefront of artificial intelligence applications. The exponential increase of these applications has made governance frameworks necessary to uphold safe and effective clinical adoption. There is little information about how healthcare practitioners in MIRT in the UK use AI tools, their governance and associated challenges, opportunities and priorities for the future. METHODS This cross-sectional survey was open from November to December 2022 to MIRT professionals who had knowledge or made use of AI tools, as an attempt to map out current policy and practice and to identify future needs. The survey was electronically distributed to the participants. Statistical analysis included descriptive statistics and inferential statistics on the SPSS statistical software. Content analysis was employed for the open-ended questions. RESULTS Among the 245 responses, the following were emphasised as central to AI adoption: governance frameworks, practitioner training, leadership, and teamwork within the AI ecosystem. Prior training was strongly correlated with increased knowledge about AI tools and frameworks. However, knowledge of related frameworks remained low, with different professionals showing different affinity to certain frameworks related to their respective roles. Common challenges and opportunities of AI adoption were also highlighted, with recommendations for future practice.
Collapse
Affiliation(s)
- Nikolaos Stogiannos
- Department of Radiography, City, University of London, UK; Magnitiki Tomografia Kerkyras, Greece.
| | - Lia Litosseliti
- School of Health & Psychological Sciences, City, University of London, UK.
| | - Tracy O'Regan
- The Society and College of Radiographers, London, UK.
| | | | - Anna Barnes
- King's Technology Evaluation Centre (KiTEC), School of Biomedical Engineering & Imaging Science, King's College London, UK.
| | | | | | | | | | - Mark F McEntee
- Discipline of Medical Imaging and Radiation Therapy, University College Cork, Ireland.
| | - Christina Malamateniou
- Department of Radiography, City, University of London, UK; European Society of Medical Imaging Informatics, Vienna, Austria.
| |
Collapse
|
9
|
Hoppe BF, Rueckel J, Dikhtyar Y, Heimer M, Fink N, Sabel BO, Ricke J, Rudolph J, Cyran CC. Implementing Artificial Intelligence for Emergency Radiology Impacts Physicians' Knowledge and Perception: A Prospective Pre- and Post-Analysis. Invest Radiol 2024; 59:404-412. [PMID: 37843828 DOI: 10.1097/rli.0000000000001034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2023]
Abstract
PURPOSE The aim of this study was to evaluate the impact of implementing an artificial intelligence (AI) solution for emergency radiology into clinical routine on physicians' perception and knowledge. MATERIALS AND METHODS A prospective interventional survey was performed pre-implementation and 3 months post-implementation of an AI algorithm for fracture detection on radiographs in late 2022. Radiologists and traumatologists were asked about their knowledge and perception of AI on a 7-point Likert scale (-3, "strongly disagree"; +3, "strongly agree"). Self-generated identification codes allowed matching the same individuals pre-intervention and post-intervention, and using Wilcoxon signed rank test for paired data. RESULTS A total of 47/71 matched participants completed both surveys (66% follow-up rate) and were eligible for analysis (34 radiologists [72%], 13 traumatologists [28%], 15 women [32%]; mean age, 34.8 ± 7.8 years). Postintervention, there was an increase that AI "reduced missed findings" (1.28 [pre] vs 1.94 [post], P = 0.003) and made readers "safer" (1.21 vs 1.64, P = 0.048), but not "faster" (0.98 vs 1.21, P = 0.261). There was a rising disagreement that AI could "replace the radiological report" (-2.04 vs -2.34, P = 0.038), as well as an increase in self-reported knowledge about "clinical AI," its "chances," and its "risks" (0.40 vs 1.00, 1.21 vs 1.70, and 0.96 vs 1.34; all P 's ≤ 0.028). Radiologists used AI results more frequently than traumatologists ( P < 0.001) and rated benefits higher (all P 's ≤ 0.038), whereas senior physicians were less likely to use AI or endorse its benefits (negative correlation with age, -0.35 to 0.30; all P 's ≤ 0.046). CONCLUSIONS Implementing AI for emergency radiology into clinical routine has an educative aspect and underlines the concept of AI as a "second reader," to support and not replace physicians.
Collapse
Affiliation(s)
- Boj Friedrich Hoppe
- From the Department of Radiology, University Hospital, LMU Munich, Munich, Germany (B.F.J., J.Rueckel, Y.D., M.H., N.F., B.O.S., J.Ricke, J.Rudolph, C.C.C.); and Institute of Neuroradiology, University Hospital, LMU Munich, Munich, Germany (J.R.)
| | | | | | | | | | | | | | | | | |
Collapse
|
10
|
Brady AP, Allen B, Chong J, Kotter E, Kottler N, Mongan J, Oakden-Rayner L, Dos Santos DP, Tang A, Wald C, Slavotinek J. Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement From the ACR, CAR, ESR, RANZCR & RSNA. Can Assoc Radiol J 2024; 75:226-244. [PMID: 38251882 DOI: 10.1177/08465371231222229] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2024] Open
Abstract
Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever‑growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi‑society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.
Collapse
Affiliation(s)
| | - Bibb Allen
- Department of Radiology, Grandview Medical Center, Birmingham, AL, USA
- Data Science Institute, American College of Radiology, Reston, VA, USA
| | - Jaron Chong
- Department of Medical Imaging, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Elmar Kotter
- Department of Diagnostic and Interventional Radiology, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Nina Kottler
- Radiology Partners, El Segundo, CA, USA
- Stanford Center for Artificial Intelligence in Medicine & Imaging, Palo Alto, CA, USA
| | - John Mongan
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, SA, Australia
| | - Daniel Pinto Dos Santos
- Department of Radiology, University Hospital of Cologne, Cologne, Germany
- Department of Radiology, University Hospital of Frankfurt, Frankfurt, Germany
| | - An Tang
- Department of Radiology, Radiation Oncology, and Nuclear Medicine, Université de Montréal, Montréal, QC, Canada
| | - Christoph Wald
- Department of Radiology, Lahey Hospital & Medical Center, Burlington, MA, USA
- Tufts University Medical School, Boston, MA, USA
- American College of Radiology, Reston, VA, USA
| | - John Slavotinek
- South Australia Medical Imaging, Flinders Medical Centre Adelaide, SA, Australia
- College of Medicine and Public Health, Flinders University, Adelaide, SA, Australia
| |
Collapse
|
11
|
Ngan KH, Mansouri-Benssassi E, Phelan J, Townsend J, Garcez AD. From explanation to intervention: Interactive knowledge extraction from Convolutional Neural Networks used in radiology. PLoS One 2024; 19:e0293967. [PMID: 38598468 PMCID: PMC11006149 DOI: 10.1371/journal.pone.0293967] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2023] [Accepted: 10/23/2023] [Indexed: 04/12/2024] Open
Abstract
Deep Learning models such as Convolutional Neural Networks (CNNs) are very effective at extracting complex image features from medical X-rays. However, the limited interpretability of CNNs has hampered their deployment in medical settings as they failed to gain trust among clinicians. In this work, we propose an interactive framework to allow clinicians to ask what-if questions and intervene in the decisions of a CNN, with the aim of increasing trust in the system. The framework translates a layer of a trained CNN into a measurable and compact set of symbolic rules. Expert interactions with visualizations of the rules promote the use of clinically-relevant CNN kernels and attach meaning to the rules. The definition and relevance of the kernels are supported by radiomics analyses and permutation evaluations, respectively. CNN kernels that do not have a clinically-meaningful interpretation are removed without affecting model performance. By allowing clinicians to evaluate the impact of adding or removing kernels from the rule set, our approach produces an interpretable refinement of the data-driven CNN in alignment with medical best practice.
Collapse
Affiliation(s)
- Kwun Ho Ngan
- Data Science Institute, City, University of London, London, United Kingdom
- Fujitsu Research of Europe Ltd, Slough, United Kingdom
| | | | - James Phelan
- Data Science Institute, City, University of London, London, United Kingdom
| | | | | |
Collapse
|
12
|
Al Mohammad B, Aldaradkeh A, Gharaibeh M, Reed W. Assessing radiologists' and radiographers' perceptions on artificial intelligence integration: opportunities and challenges. Br J Radiol 2024; 97:763-769. [PMID: 38273675 PMCID: PMC11027289 DOI: 10.1093/bjr/tqae022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Revised: 09/30/2023] [Accepted: 01/21/2024] [Indexed: 01/27/2024] Open
Abstract
OBJECTIVES The objective of this study was to evaluate radiologists' and radiographers' opinions and perspectives on artificial intelligence (AI) and its integration into the radiology department. Additionally, we investigated the most common challenges and barriers that radiologists and radiographers face when learning about AI. METHODS A nationwide, online descriptive cross-sectional survey was distributed to radiologists and radiographers working in hospitals and medical centres from May 29, 2023 to July 30, 2023. The questionnaire examined the participants' opinions, feelings, and predictions regarding AI and its applications in the radiology department. Descriptive statistics were used to report the participants' demographics and responses. Five-points Likert-scale data were reported using divergent stacked bar graphs to highlight any central tendencies. RESULTS Responses were collected from 258 participants, revealing a positive attitude towards implementing AI. Both radiologists and radiographers predicted breast imaging would be the subspecialty most impacted by the AI revolution. MRI, mammography, and CT were identified as the primary modalities with significant importance in the field of AI application. The major barrier encountered by radiologists and radiographers when learning about AI was the lack of mentorship, guidance, and support from experts. CONCLUSION Participants demonstrated a positive attitude towards learning about AI and implementing it in the radiology practice. However, radiologists and radiographers encounter several barriers when learning about AI, such as the absence of experienced professionals support and direction. ADVANCES IN KNOWLEDGE Radiologists and radiographers reported several barriers to AI learning, with the most significant being the lack of mentorship and guidance from experts, followed by the lack of funding and investment in new technologies.
Collapse
Affiliation(s)
- Badera Al Mohammad
- Department of Allied Medical Sciences, Faculty of Applied Medical Sciences, Jordan University of Science and Technology, Irbid 22110, Jordan
| | - Afnan Aldaradkeh
- Department of Allied Medical Sciences, Faculty of Applied Medical Sciences, Jordan University of Science and Technology, Irbid 22110, Jordan
| | - Monther Gharaibeh
- Department of Special Surgery, Faculty of Medicine, The Hashemite University, Zarqa 13133, Jordan
| | - Warren Reed
- Discipline of Medical Imaging Science, Faculty of Medicine and Health, University of Sydney 2006, Sydney, NSW, Australia
| |
Collapse
|
13
|
Brady AP, Allen B, Chong J, Kotter E, Kottler N, Mongan J, Oakden-Rayner L, Pinto Dos Santos D, Tang A, Wald C, Slavotinek J. Developing, purchasing, implementing and monitoring AI tools in radiology: Practical considerations. A multi-society statement from the ACR, CAR, ESR, RANZCR & RSNA. J Med Imaging Radiat Oncol 2024; 68:7-26. [PMID: 38259140 DOI: 10.1111/1754-9485.13612] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Accepted: 11/23/2023] [Indexed: 01/24/2024]
Abstract
Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.
Collapse
Affiliation(s)
| | - Bibb Allen
- Department of Radiology, Grandview Medical Center, Birmingham, Alabama, USA
- American College of Radiology Data Science Institute, Reston, Virginia, USA
| | - Jaron Chong
- Department of Medical Imaging, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| | - Elmar Kotter
- Department of Diagnostic and Interventional Radiology, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Nina Kottler
- Radiology Partners, El Segundo, California, USA
- Stanford Center for Artificial Intelligence in Medicine & Imaging, Palo Alto, California, USA
| | - John Mongan
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, California, USA
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, South Australia, Australia
| | - Daniel Pinto Dos Santos
- Department of Radiology, University Hospital of Cologne, Cologne, Germany
- Department of Radiology, University Hospital of Frankfurt, Frankfurt, Germany
| | - An Tang
- Department of Radiology, Radiation Oncology, and Nuclear Medicine, Université de Montréal, Montreal, Quebec, Canada
| | - Christoph Wald
- Department of Radiology, Lahey Hospital & Medical Center, Burlington, Massachusetts, USA
- Tufts University Medical School, Boston, Massachusetts, USA
- Commision On Informatics, and Member, Board of Chancellors, American College of Radiology, Reston, Virginia, USA
| | - John Slavotinek
- South Australia Medical Imaging, Flinders Medical Centre Adelaide, Adelaide, South Australia, Australia
- College of Medicine and Public Health, Flinders University, Adelaide, South Australia, Australia
| |
Collapse
|
14
|
Brady AP, Allen B, Chong J, Kotter E, Kottler N, Mongan J, Oakden-Rayner L, Dos Santos DP, Tang A, Wald C, Slavotinek J. Developing, purchasing, implementing and monitoring AI tools in radiology: practical considerations. A multi-society statement from the ACR, CAR, ESR, RANZCR & RSNA. Insights Imaging 2024; 15:16. [PMID: 38246898 PMCID: PMC10800328 DOI: 10.1186/s13244-023-01541-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2024] Open
Abstract
Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones.This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.Key points • The incorporation of artificial intelligence (AI) in radiological practice demands increased monitoring of its utility and safety.• Cooperation between developers, clinicians, and regulators will allow all involved to address ethical issues and monitor AI performance.• AI can fulfil its promise to advance patient well-being if all steps from development to integration in healthcare are rigorously evaluated.
Collapse
Affiliation(s)
| | - Bibb Allen
- Department of Radiology, Grandview Medical Center, Birmingham, AL, USA
- American College of Radiology Data Science Institute, Reston, VA, USA
| | - Jaron Chong
- Department of Medical Imaging, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Elmar Kotter
- Department of Diagnostic and Interventional Radiology, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Nina Kottler
- Radiology Partners, El Segundo, CA, USA
- Stanford Center for Artificial Intelligence in Medicine & Imaging, Palo Alto, CA, USA
| | - John Mongan
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, USA
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, Australia
| | - Daniel Pinto Dos Santos
- Department of Radiology, University Hospital of Cologne, Cologne, Germany
- Department of Radiology, University Hospital of Frankfurt, Frankfurt, Germany
| | - An Tang
- Department of Radiology, Radiation Oncology, and Nuclear Medicine, Université de Montréal, Montréal, Québec, Canada
| | - Christoph Wald
- Department of Radiology, Lahey Hospital & Medical Center, Burlington, MA, USA
- Tufts University Medical School, Boston, MA, USA
- Commision On Informatics, and Member, Board of Chancellors, American College of Radiology, Virginia, USA
| | - John Slavotinek
- South Australia Medical Imaging, Flinders Medical Centre Adelaide, Adelaide, Australia
- College of Medicine and Public Health, Flinders University, Adelaide, Australia
| |
Collapse
|
15
|
Brady AP, Allen B, Chong J, Kotter E, Kottler N, Mongan J, Oakden-Rayner L, dos Santos DP, Tang A, Wald C, Slavotinek J. Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement from the ACR, CAR, ESR, RANZCR and RSNA. Radiol Artif Intell 2024; 6:e230513. [PMID: 38251899 PMCID: PMC10831521 DOI: 10.1148/ryai.230513] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2024]
Abstract
Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools. This article is simultaneously published in Insights into Imaging (DOI 10.1186/s13244-023-01541-3), Journal of Medical Imaging and Radiation Oncology (DOI 10.1111/1754-9485.13612), Canadian Association of Radiologists Journal (DOI 10.1177/08465371231222229), Journal of the American College of Radiology (DOI 10.1016/j.jacr.2023.12.005), and Radiology: Artificial Intelligence (DOI 10.1148/ryai.230513). Keywords: Artificial Intelligence, Radiology, Automation, Machine Learning Published under a CC BY 4.0 license. ©The Author(s) 2024. Editor's Note: The RSNA Board of Directors has endorsed this article. It has not undergone review or editing by this journal.
Collapse
Affiliation(s)
| | - Bibb Allen
- Department of Radiology, Grandview Medical
Center, Birmingham, AL, USA
- American College of Radiology Data Science
Institute, Reston, VA, USA
| | - Jaron Chong
- Department of Medical Imaging, Schulich
School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Elmar Kotter
- Department of Diagnostic and
Interventional Radiology, Medical Center, Faculty of Medicine, University of
Freiburg, Freiburg, Germany
| | - Nina Kottler
- Radiology Partners, El Segundo, CA,
USA
- Stanford Center for Artificial
Intelligence in Medicine & Imaging, Palo Alto, CA, USA
| | - John Mongan
- Department of Radiology and Biomedical
Imaging, University of California, San Francisco, USA
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning,
University of Adelaide, Adelaide, Australia
| | - Daniel Pinto dos Santos
- Department of Radiology, University
Hospital of Cologne, Cologne, Germany
- Department of Radiology, University
Hospital of Frankfurt, Frankfurt, Germany
| | - An Tang
- Department of Radiology, Radiation
Oncology, and Nuclear Medicine, Université de Montréal,
Montréal, Québec, Canada
| | - Christoph Wald
- Department of Radiology, Lahey Hospital
& Medical Center, Burlington, MA, USA
- Tufts University Medical School, Boston,
MA, USA
- Commission On Informatics, and Member,
Board of Chancellors, American College of Radiology, Virginia, USA
| | - John Slavotinek
- South Australia Medical Imaging,
Flinders Medical Centre Adelaide, Adelaide, Australia
- College of Medicine and Public Health,
Flinders University, Adelaide, Australia
| |
Collapse
|
16
|
van Leeuwen KG, de Rooij M, Schalekamp S, van Ginneken B, Rutten MJCM. Clinical use of artificial intelligence products for radiology in the Netherlands between 2020 and 2022. Eur Radiol 2024; 34:348-354. [PMID: 37515632 PMCID: PMC10791748 DOI: 10.1007/s00330-023-09991-5] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 04/14/2023] [Accepted: 05/23/2023] [Indexed: 07/31/2023]
Abstract
OBJECTIVES To map the clinical use of CE-marked artificial intelligence (AI)-based software in radiology departments in the Netherlands (n = 69) between 2020 and 2022. MATERIALS AND METHODS Our AI network (one radiologist or AI representative per Dutch hospital organization) received a questionnaire each spring from 2020 to 2022 about AI product usage, financing, and obstacles to adoption. Products that were not listed on www.AIforRadiology.com by July 2022 were excluded from the analysis. RESULTS The number of respondents was 43 in 2020, 36 in 2021, and 33 in 2022. The number of departments using AI has been growing steadily (2020: 14, 2021: 19, 2022: 23). The diversity (2020: 7, 2021: 18, 2022: 34) and the number of total implementations (2020: 19, 2021: 38, 2022: 68) has rapidly increased. Seven implementations were discontinued in 2022. Four hospital organizations said to use an AI platform or marketplace for the deployment of AI solutions. AI is mostly used to support chest CT (17), neuro CT (17), and musculoskeletal radiograph (12) analysis. The budget for AI was reserved in 13 of the responding centers in both 2021 and 2022. The most important obstacles to the adoption of AI remained costs and IT integration. Of the respondents, 28% stated that the implemented AI products realized health improvement and 32% assumed both health improvement and cost savings. CONCLUSION The adoption of AI products in radiology departments in the Netherlands is showing common signs of a developing market. The major obstacles to reaching widespread adoption are a lack of financial resources and IT integration difficulties. CLINICAL RELEVANCE STATEMENT The clinical impact of AI starts with its adoption in daily clinical practice. Increased transparency around AI products being adopted, implementation obstacles, and impact may inspire increased collaboration and improved decision-making around the implementation and financing of AI products. KEY POINTS • The adoption of artificial intelligence products for radiology has steadily increased since 2020 to at least a third of the centers using AI in clinical practice in the Netherlands in 2022. • The main areas in which artificial intelligence products are used are lung nodule detection on CT, aided stroke diagnosis, and bone age prediction. • The majority of respondents experienced added value (decreased costs and/or improved outcomes) from using artificial intelligence-based software; however, major obstacles to adoption remain the costs and IT-related difficulties.
Collapse
Affiliation(s)
- Kicky G van Leeuwen
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Maarten de Rooij
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Steven Schalekamp
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Bram van Ginneken
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Matthieu J C M Rutten
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands
- Department of Radiology, Jeroen Bosch Hospital, 's-Hertogenbosch, The Netherlands
| |
Collapse
|
17
|
Chen Y, Wu Z, Wang P, Xie L, Yan M, Jiang M, Yang Z, Zheng J, Zhang J, Zhu J. Radiology Residents' Perceptions of Artificial Intelligence: Nationwide Cross-Sectional Survey Study. J Med Internet Res 2023; 25:e48249. [PMID: 37856181 PMCID: PMC10623237 DOI: 10.2196/48249] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2023] [Revised: 07/07/2023] [Accepted: 09/01/2023] [Indexed: 10/20/2023] Open
Abstract
BACKGROUND Artificial intelligence (AI) is transforming various fields, with health care, especially diagnostic specialties such as radiology, being a key but controversial battleground. However, there is limited research systematically examining the response of "human intelligence" to AI. OBJECTIVE This study aims to comprehend radiologists' perceptions regarding AI, including their views on its potential to replace them, its usefulness, and their willingness to accept it. We examine the influence of various factors, encompassing demographic characteristics, working status, psychosocial aspects, personal experience, and contextual factors. METHODS Between December 1, 2020, and April 30, 2021, a cross-sectional survey was completed by 3666 radiology residents in China. We used multivariable logistic regression models to examine factors and associations, reporting odds ratios (ORs) and 95% CIs. RESULTS In summary, radiology residents generally hold a positive attitude toward AI, with 29.90% (1096/3666) agreeing that AI may reduce the demand for radiologists, 72.80% (2669/3666) believing AI improves disease diagnosis, and 78.18% (2866/3666) feeling that radiologists should embrace AI. Several associated factors, including age, gender, education, region, eye strain, working hours, time spent on medical images, resilience, burnout, AI experience, and perceptions of residency support and stress, significantly influence AI attitudes. For instance, burnout symptoms were associated with greater concerns about AI replacement (OR 1.89; P<.001), less favorable views on AI usefulness (OR 0.77; P=.005), and reduced willingness to use AI (OR 0.71; P<.001). Moreover, after adjusting for all other factors, perceived AI replacement (OR 0.81; P<.001) and AI usefulness (OR 5.97; P<.001) were shown to significantly impact the intention to use AI. CONCLUSIONS This study profiles radiology residents who are accepting of AI. Our comprehensive findings provide insights for a multidimensional approach to help physicians adapt to AI. Targeted policies, such as digital health care initiatives and medical education, can be developed accordingly.
Collapse
Affiliation(s)
- Yanhua Chen
- Vanke School of Public Health, Tsinghua University, Beijing, China
- School of Medicine, Tsinghua University, Beijing, China
| | - Ziye Wu
- Vanke School of Public Health, Tsinghua University, Beijing, China
| | - Peicheng Wang
- Vanke School of Public Health, Tsinghua University, Beijing, China
- School of Medicine, Tsinghua University, Beijing, China
| | - Linbo Xie
- Vanke School of Public Health, Tsinghua University, Beijing, China
- School of Medicine, Tsinghua University, Beijing, China
| | - Mengsha Yan
- Vanke School of Public Health, Tsinghua University, Beijing, China
| | - Maoqing Jiang
- Department of Radiology, Ningbo No. 2 Hospital, Ningbo, China
| | - Zhenghan Yang
- Department of Radiology, Beijing Friendship Hospital, Capital Medical University, Beijing, China
| | - Jianjun Zheng
- Department of Radiology, Ningbo No. 2 Hospital, Ningbo, China
| | - Jingfeng Zhang
- Department of Radiology, Ningbo No. 2 Hospital, Ningbo, China
| | - Jiming Zhu
- Vanke School of Public Health, Tsinghua University, Beijing, China
- Institute for Healthy China, Tsinghua University, Beijing, China
| |
Collapse
|
18
|
Becker M. How to prepare for a bright future of radiology in Europe. Insights Imaging 2023; 14:168. [PMID: 37816908 PMCID: PMC10564684 DOI: 10.1186/s13244-023-01525-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 09/16/2023] [Indexed: 10/12/2023] Open
Abstract
Because artificial intelligence (AI)-powered algorithms allow automated image analysis in a growing number of diagnostic scenarios, some healthcare stakeholders have raised doubts about the future of the entire radiologic profession. Their view disregards not only the role of radiologists in the diagnostic service chain beyond reporting, but also the many multidisciplinary and patient-related consulting tasks for which radiologists are solicited. The time commitment for these non-reporting tasks is considerable but difficult to quantify and often impossible to fulfil considering the current mismatch between workload and workforce in many countries. Nonetheless, multidisciplinary, and patient-centred consulting activities could move up on radiologists' agendas as soon as AI-based tools can save time in daily routine. Although there are many reasons why AI will assist and not replace radiologists as imaging experts in the future, it is important to position the next generation of European radiologists in view of this expected trend. To ensure radiologists' personal professional recognition and fulfilment in multidisciplinary environments, the focus of training should go beyond diagnostic reporting, concentrating on clinical backgrounds, specific communication skills with referrers and patients, and integration of imaging findings with those of other disciplines. Close collaboration between the European Society of Radiology (ESR) and European national radiologic societies can help to achieve these goals. Although each adequate treatment begins with a correct diagnosis, many health politicians see radiologic procedures mainly as a cost factor. Radiologic research should, therefore, increasingly investigate the imaging impact on treatment and outcome rather than focusing mainly on technical improvements and diagnostic accuracy alone.Critical relevance statement Strategies are presented to prepare for a successful future of the radiologic profession in Europe, if AI-powered tools can alleviate the current reporting overload: engaging in multidisciplinary activities (clinical and integrative diagnostics), enhancing the value and recognition of radiologists' role through clinical expertise, focusing radiological research on the impact on diagnosis and outcome, and promoting patient-centred radiology by enhancing communication skills.Key points • AI-powered tools will not replace radiologists but hold promise to reduce the current reporting burden, enabling them to reinvest liberated time in multidisciplinary clinical and patient-related tasks.• The skills and resources for these tasks should be considered when recruiting and teaching the next generation of radiologists, when organising departments and planning staffing.• Communication skills will play an increasing role in both multidisciplinary activities and patient-centred radiology.• The value and importance of a correct and integrative diagnosis and the cost of an incorrect imaging diagnosis should be emphasised when discussing with non-medical stakeholders in healthcare.• The radiologic community in Europe should start now to prepare for a bright future of the profession for the benefit of patients and medical colleagues alike.
Collapse
Affiliation(s)
- Minerva Becker
- Unit of Head and Neck and Maxilofacial Radiology, Division of Radiology, Diagnostic Department, Geneva University Hospitals, University of Geneva, Rue Gabrielle Perret Gentil 4, Geneva 14, CH 1211, Switzerland.
| |
Collapse
|
19
|
Mello-Thoms C, Mello CAB. Clinical applications of artificial intelligence in radiology. Br J Radiol 2023; 96:20221031. [PMID: 37099398 PMCID: PMC10546456 DOI: 10.1259/bjr.20221031] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 03/28/2023] [Accepted: 03/28/2023] [Indexed: 04/27/2023] Open
Abstract
The rapid growth of medical imaging has placed increasing demands on radiologists. In this scenario, artificial intelligence (AI) has become an attractive partner, one that may complement case interpretation and may aid in various non-interpretive aspects of the work in the radiological clinic. In this review, we discuss interpretative and non-interpretative uses of AI in the clinical practice, as well as report on the barriers to AI's adoption in the clinic. We show that AI currently has a modest to moderate penetration in the clinical practice, with many radiologists still being unconvinced of its value and the return on its investment. Moreover, we discuss the radiologists' liabilities regarding the AI decisions, and explain how we currently do not have regulation to guide the implementation of explainable AI or of self-learning algorithms.
Collapse
Affiliation(s)
| | - Carlos A B Mello
- Centro de Informática, Universidade Federal de Pernambuco, Recife, Brazil
| |
Collapse
|
20
|
Hong W, Hwang EJ, Park CM, Goo JM. Effects of Implementing Artificial Intelligence-Based Computer-Aided Detection for Chest Radiographs in Daily Practice on the Rate of Referral to Chest Computed Tomography in Pulmonology Outpatient Clinic. Korean J Radiol 2023; 24:890-902. [PMID: 37634643 PMCID: PMC10462895 DOI: 10.3348/kjr.2023.0255] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2023] [Revised: 03/19/2023] [Accepted: 06/26/2023] [Indexed: 08/29/2023] Open
Abstract
OBJECTIVE The clinical impact of artificial intelligence-based computer-aided detection (AI-CAD) beyond diagnostic accuracy remains uncertain. We aimed to investigate the influence of the clinical implementation of AI-CAD for chest radiograph (CR) interpretation in daily practice on the rate of referral for chest computed tomography (CT). MATERIALS AND METHODS AI-CAD was implemented in clinical practice at the Seoul National University Hospital. CRs obtained from patients who visited the pulmonology outpatient clinics before (January-December 2019) and after (January-December 2020) implementation were included in this study. After implementation, the referring pulmonologist requested CRs with or without AI-CAD analysis. We conducted multivariable logistic regression analyses to evaluate the associations between using AI-CAD and the following study outcomes: the rate of chest CT referral, defined as request and actual acquisition of chest CT within 30 days after CR acquisition, and the CT referral rates separately for subsequent positive and negative CT results. Multivariable analyses included various covariates such as patient age and sex, time of CR acquisition (before versus after AI-CAD implementation), referring pulmonologist, nature of the CR examination (baseline versus follow-up examination), and radiology reports presence at the time of the pulmonology visit. RESULTS A total of 28546 CRs from 14565 patients (mean age: 67 years; 7130 males) and 25888 CRs from 12929 patients (mean age: 67 years; 6435 males) before and after AI-CAD implementation were included. The use of AI-CAD was independently associated with increased chest CT referrals (odds ratio [OR], 1.33; P = 0.008) and referrals with subsequent negative chest CT results (OR, 1.46; P = 0.005). Meanwhile, referrals with positive chest CT results were not significantly associated with AI-CAD use (OR, 1.08; P = 0.647). CONCLUSION The use of AI-CAD for CR interpretation in pulmonology outpatients was independently associated with an increased frequency of overall referrals for chest CT scans and referrals with subsequent negative results.
Collapse
Affiliation(s)
- Wonju Hong
- Department of Radiology, Hallym University Sacred Heart Hospital, Anyang, Republic of Korea
| | - Eui Jin Hwang
- Department of Radiology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Republic of Korea.
| | - Chang Min Park
- Department of Radiology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Republic of Korea
- Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Republic of Korea
- Cancer Research Institute, Seoul National University, Seoul, Republic of Korea
| | - Jin Mo Goo
- Department of Radiology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Republic of Korea
- Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Republic of Korea
- Cancer Research Institute, Seoul National University, Seoul, Republic of Korea
| |
Collapse
|
21
|
Petsiou DP, Martinos A, Spinos D. Applications of Artificial Intelligence in Temporal Bone Imaging: Advances and Future Challenges. Cureus 2023; 15:e44591. [PMID: 37795060 PMCID: PMC10545916 DOI: 10.7759/cureus.44591] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/02/2023] [Indexed: 10/06/2023] Open
Abstract
The applications of artificial intelligence (AI) in temporal bone (TB) imaging have gained significant attention in recent years, revolutionizing the field of otolaryngology and radiology. Accurate interpretation of imaging features of TB conditions plays a crucial role in diagnosing and treating a range of ear-related pathologies, including middle and inner ear diseases, otosclerosis, and vestibular schwannomas. According to multiple clinical studies published in the literature, AI-powered algorithms have demonstrated exceptional proficiency in interpreting imaging findings, not only saving time for physicians but also enhancing diagnostic accuracy by reducing human error. Although several challenges remain in routinely relying on AI applications, the collaboration between AI and healthcare professionals holds the key to better patient outcomes and significantly improved patient care. This overview delivers a comprehensive update on the advances of AI in the field of TB imaging, summarizes recent evidence provided by clinical studies, and discusses future insights and challenges in the widespread integration of AI in clinical practice.
Collapse
Affiliation(s)
- Dioni-Pinelopi Petsiou
- Otolaryngology-Head and Neck Surgery, National and Kapodistrian University of Athens, School of Medicine, Athens, GRC
| | - Anastasios Martinos
- Otolaryngology-Head and Neck Surgery, National and Kapodistrian University of Athens, School of Medicine, Athens, GRC
| | - Dimitrios Spinos
- Otolaryngology-Head and Neck Surgery, Gloucestershire Hospitals NHS Foundation Trust, Gloucester, GBR
| |
Collapse
|
22
|
Hathaway QA, Hogg JP, Lakhani DA. Need for Medical Student Education in Emerging Technologies and Artificial Intelligence: Fostering Enthusiasm, Rather Than Flight, From Specialties Most Affected by Emerging Technologies. Acad Radiol 2023; 30:1770-1771. [PMID: 36464546 DOI: 10.1016/j.acra.2022.11.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Accepted: 11/14/2022] [Indexed: 12/03/2022]
Affiliation(s)
- Quincy A Hathaway
- School of Medicine, West Virginia University, 1 Medical Center Drive, Morgantown, WV, USA
| | - Jeffery P Hogg
- School of Medicine, West Virginia University, 1 Medical Center Drive, Morgantown, WV, USA; Department of Radiology, West Virginia University, 1 Medical Center Drive, Morgantown, WV, USA
| | - Dhairya A Lakhani
- Department of Radiology, West Virginia University, 1 Medical Center Drive, Morgantown, WV, USA.
| |
Collapse
|
23
|
Rajpurkar P, Lungren MP. The Current and Future State of AI Interpretation of Medical Images. N Engl J Med 2023; 388:1981-1990. [PMID: 37224199 DOI: 10.1056/nejmra2301725] [Citation(s) in RCA: 80] [Impact Index Per Article: 80.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Affiliation(s)
- Pranav Rajpurkar
- From the Department of Biomedical Informatics, Harvard Medical School, Boston (P.R.); the Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Stanford, and the Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco - both in California (M.P.L.); and Microsoft, Redmond, Washington (M.P.L.)
| | - Matthew P Lungren
- From the Department of Biomedical Informatics, Harvard Medical School, Boston (P.R.); the Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Stanford, and the Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco - both in California (M.P.L.); and Microsoft, Redmond, Washington (M.P.L.)
| |
Collapse
|
24
|
Hong S, Hwang EJ, Kim S, Song J, Lee T, Jo GD, Choi Y, Park CM, Goo JM. Methods of Visualizing the Results of an Artificial-Intelligence-Based Computer-Aided Detection System for Chest Radiographs: Effect on the Diagnostic Performance of Radiologists. Diagnostics (Basel) 2023; 13:diagnostics13061089. [PMID: 36980397 PMCID: PMC10046978 DOI: 10.3390/diagnostics13061089] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2023] [Revised: 03/02/2023] [Accepted: 03/12/2023] [Indexed: 03/16/2023] Open
Abstract
It is unclear whether the visualization methods for artificial-intelligence-based computer-aided detection (AI-CAD) of chest radiographs influence the accuracy of readers’ interpretation. We aimed to evaluate the accuracy of radiologists’ interpretations of chest radiographs using different visualization methods for the same AI-CAD. Initial chest radiographs of patients with acute respiratory symptoms were retrospectively collected. A commercialized AI-CAD using three different methods of visualizing was applied: (a) closed-line method, (b) heat map method, and (c) combined method. A reader test was conducted with five trainee radiologists over three interpretation sessions. In each session, the chest radiographs were interpreted using AI-CAD with one of the three visualization methods in random order. Examination-level sensitivity and accuracy, and lesion-level detection rates for clinically significant abnormalities were evaluated for the three visualization methods. The sensitivity (p = 0.007) and accuracy (p = 0.037) of the combined method are significantly higher than that of the closed-line method. Detection rates using the heat map method (p = 0.043) and the combined method (p = 0.004) are significantly higher than those using the closed-line method. The methods for visualizing AI-CAD results for chest radiographs influenced the performance of radiologists’ interpretations. Combining the closed-line and heat map methods for visualizing AI-CAD results led to the highest sensitivity and accuracy of radiologists.
Collapse
Affiliation(s)
- Sungho Hong
- Department of Radiology, Seoul National University Hospital, Seoul 03082, Republic of Korea
| | - Eui Jin Hwang
- Department of Radiology, Seoul National University Hospital, Seoul 03082, Republic of Korea
- Department of Radiology, Seoul National University College of Medicine, Seoul 03082, Republic of Korea
- Correspondence: ; Tel.: +82-2-2072-2057
| | - Soojin Kim
- Department of Radiology, Seoul National University Hospital, Seoul 03082, Republic of Korea
| | - Jiyoung Song
- Department of Radiology, Seoul National University Hospital, Seoul 03082, Republic of Korea
| | - Taehee Lee
- Department of Radiology, Seoul National University Hospital, Seoul 03082, Republic of Korea
| | - Gyeong Deok Jo
- Department of Radiology, Seoul National University Hospital, Seoul 03082, Republic of Korea
| | - Yelim Choi
- Department of Radiology, Seoul National University Hospital, Seoul 03082, Republic of Korea
| | - Chang Min Park
- Department of Radiology, Seoul National University Hospital, Seoul 03082, Republic of Korea
- Department of Radiology, Seoul National University College of Medicine, Seoul 03082, Republic of Korea
- Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul 03082, Republic of Korea
| | - Jin Mo Goo
- Department of Radiology, Seoul National University Hospital, Seoul 03082, Republic of Korea
- Department of Radiology, Seoul National University College of Medicine, Seoul 03082, Republic of Korea
- Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul 03082, Republic of Korea
| |
Collapse
|
25
|
Berbís MA, Paulano Godino F, Royuela del Val J, Alcalá Mata L, Luna A. Clinical impact of artificial intelligence-based solutions on imaging of the pancreas and liver. World J Gastroenterol 2023; 29:1427-1445. [PMID: 36998424 PMCID: PMC10044858 DOI: 10.3748/wjg.v29.i9.1427] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 01/13/2023] [Accepted: 02/27/2023] [Indexed: 03/07/2023] Open
Abstract
Artificial intelligence (AI) has experienced substantial progress over the last ten years in many fields of application, including healthcare. In hepatology and pancreatology, major attention to date has been paid to its application to the assisted or even automated interpretation of radiological images, where AI can generate accurate and reproducible imaging diagnosis, reducing the physicians’ workload. AI can provide automatic or semi-automatic segmentation and registration of the liver and pancreatic glands and lesions. Furthermore, using radiomics, AI can introduce new quantitative information which is not visible to the human eye to radiological reports. AI has been applied in the detection and characterization of focal lesions and diffuse diseases of the liver and pancreas, such as neoplasms, chronic hepatic disease, or acute or chronic pancreatitis, among others. These solutions have been applied to different imaging techniques commonly used to diagnose liver and pancreatic diseases, such as ultrasound, endoscopic ultrasonography, computerized tomography (CT), magnetic resonance imaging, and positron emission tomography/CT. However, AI is also applied in this context to many other relevant steps involved in a comprehensive clinical scenario to manage a gastroenterological patient. AI can also be applied to choose the most convenient test prescription, to improve image quality or accelerate its acquisition, and to predict patient prognosis and treatment response. In this review, we summarize the current evidence on the application of AI to hepatic and pancreatic radiology, not only in regard to the interpretation of images, but also to all the steps involved in the radiological workflow in a broader sense. Lastly, we discuss the challenges and future directions of the clinical application of AI methods.
Collapse
Affiliation(s)
- M Alvaro Berbís
- Department of Radiology, HT Médica, San Juan de Dios Hospital, Córdoba 14960, Spain
- Faculty of Medicine, Autonomous University of Madrid, Madrid 28049, Spain
| | | | | | - Lidia Alcalá Mata
- Department of Radiology, HT Médica, Clínica las Nieves, Jaén 23007, Spain
| | - Antonio Luna
- Department of Radiology, HT Médica, Clínica las Nieves, Jaén 23007, Spain
| |
Collapse
|
26
|
Giganti F, Panebianco V, Tempany CM, Purysko AS. Is Artificial Intelligence Replacing Our Radiology Stars in Prostate Magnetic Resonance Imaging? The Stars Do Not Look Big, But They Can Look Brighter. EUR UROL SUPPL 2022; 48:12-13. [PMID: 36578461 PMCID: PMC9791603 DOI: 10.1016/j.euros.2022.11.021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/21/2022] [Indexed: 12/23/2022] Open
Affiliation(s)
- Francesco Giganti
- Department of Radiology, University College London Hospital NHS Foundation Trust, London, UK,Division of Surgery & Interventional Science, University College London, London, UK,Corresponding author. Division of Surgery and Interventional Science, University College London, Charles Bell House, 43–45 Foley Street, London W1W 7TY, UK.
| | - Valeria Panebianco
- Department of Radiological Sciences, Oncology and Pathology, Sapienza University/Policlinico Umberto I, Rome, Italy
| | - Clare M. Tempany
- Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
| | - Andrei S. Purysko
- Abdominal Imaging Section and Nuclear Radiology Department, Imaging Institute, Cleveland Clinic, Cleveland, OH, USA,Glickman Urological and Kidney Institute, Cleveland Clinic, Cleveland, OH, USA
| |
Collapse
|
27
|
Nagy E, Marterer R, Hržić F, Sorantin E, Tschauner S. Learning rate of students detecting and annotating pediatric wrist fractures in supervised artificial intelligence dataset preparations. PLoS One 2022; 17:e0276503. [PMID: 36264961 PMCID: PMC9584407 DOI: 10.1371/journal.pone.0276503] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 09/13/2022] [Indexed: 11/06/2022] Open
Abstract
The use of artificial intelligence (AI) in image analysis is an intensively debated topic in the radiology community these days. AI computer vision algorithms typically rely on large-scale image databases, annotated by specialists. Developing and maintaining them is time-consuming, thus, the involvement of non-experts into the workflow of annotation should be considered. We assessed the learning rate of inexperienced evaluators regarding correct labeling of pediatric wrist fractures on digital radiographs. Students with and without a medical background labeled wrist fractures with bounding boxes in 7,000 radiographs over ten days. Pediatric radiologists regularly discussed their mistakes. We found F1 scores-as a measure for detection rate-to increase substantially under specialist feedback (mean 0.61±0.19 at day 1 to 0.97±0.02 at day 10, p<0.001), but not the Intersection over Union as a parameter for labeling precision (mean 0.27±0.29 at day 1 to 0.53±0.25 at day 10, p<0.001). The times needed to correct the students decreased significantly (mean 22.7±6.3 seconds per image at day 1 to 8.9±1.2 seconds at day 10, p<0.001) and were substantially lower as annotated by the radiologists alone. In conclusion our data showed, that the involvement of undergraduated students into annotation of pediatric wrist radiographs enables a substantial time saving for specialists, therefore, it should be considered.
Collapse
Affiliation(s)
- Eszter Nagy
- Division of Pediatric Radiology, Department of Radiology, Medical University of Graz, Graz, Austria
- * E-mail:
| | - Robert Marterer
- Division of Pediatric Radiology, Department of Radiology, Medical University of Graz, Graz, Austria
| | - Franko Hržić
- Faculty of Engineering, University of Rijeka, Rijeka, Croatia
| | - Erich Sorantin
- Division of Pediatric Radiology, Department of Radiology, Medical University of Graz, Graz, Austria
| | - Sebastian Tschauner
- Division of Pediatric Radiology, Department of Radiology, Medical University of Graz, Graz, Austria
| |
Collapse
|
28
|
Murugesan A, Patel S, Viswanathan VS, Bhargava P, Faraji N. Dear Medical Students - Artificial Intelligence is NOT taking away a Radiologist's Job. Curr Probl Diagn Radiol 2022; 52:1-5. [DOI: 10.1067/j.cpradiol.2022.08.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2022] [Accepted: 08/22/2022] [Indexed: 11/22/2022]
|
29
|
Time Is Money: Considerations for Measuring the Radiological Reading Time. J Imaging 2022; 8:jimaging8080208. [PMID: 35893086 PMCID: PMC9394242 DOI: 10.3390/jimaging8080208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Revised: 07/13/2022] [Accepted: 07/22/2022] [Indexed: 11/17/2022] Open
Abstract
Timestamps in the Radiology Information System (RIS) are a readily available and valuable source of information with increasing significance, among others, due to the current focus on the clinical impact of artificial intelligence applications. We aimed to evaluate timestamp-based radiological dictation time, introduce timestamp modeling techniques, and compare those with prospective measured reporting. Dictation time was calculated from RIS timestamps between 05/2010 and 01/2021 at our institution (n = 108,310). We minimized contextual outliers by simulating the raw data by iteration (1000, vector size (µ/sd/λ) = 100/loop), assuming normally distributed reporting times. In addition, 329 reporting times were prospectively measured by two radiologists (1 and 4 years of experience). Altogether, 106,127 of 108,310 exams were included after simulation, with a mean dictation time of 16.62 min. Mean dictation time was 16.05 min head CT (44,743/45,596), 15.84 min for chest CT (32,797/33,381), 17.92 min for abdominal CT (n = 22,805/23,483), 10.96 min for CT foot (n = 937/958), 9.14 min for lumbar spine (881/892), 8.83 min for shoulder (409/436), 8.83 min for CT wrist (1201/1322), and 39.20 min for a polytrauma patient (2127/2242), without a significant difference to the prospective reporting times. In conclusion, timestamp analysis is useful to measure current reporting practice, whereas body-region and radiological experience are confounders. This could aid in cost–benefit assessments of workflow changes (e.g., AI implementation).
Collapse
|