1
|
Wongveerasin P, Tongdee T, Saiviroonporn P. Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residents. Eur J Radiol Open 2024; 13:100593. [PMID: 39175597 PMCID: PMC11338948 DOI: 10.1016/j.ejro.2024.100593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2024] [Revised: 07/12/2024] [Accepted: 07/23/2024] [Indexed: 08/24/2024] Open
Abstract
Background Artificial intelligence (AI) has been proven useful for the assessment of tubes and lines on chest radiographs of general patients. However, validation on intensive care unit (ICU) patients remains imperative. Methods This retrospective case-control study evaluated the performance of deep learning (DL) models for tubes and lines classification on both an external public dataset and a local dataset comprising 303 films randomly sampled from the ICU database. The endotracheal tubes (ETTs), central venous catheters (CVCs), and nasogastric tubes (NGTs) were classified into "Normal," "Abnormal," or "Borderline" positions by DL models with and without rule-based modification. Their performance was evaluated using an experienced radiologist as the standard reference. Results The algorithm showed decreased performance on the local ICU dataset, compared to that of the external dataset, decreasing from the Area Under the Curve of Receiver (AUC) of 0.967 (95 % CI 0.965-0.973) to the AUC of 0.70 (95 % CI 0.68-0.77). Significant improvement in the ETT classification task was observed after modifications were made to the model to allow the use of the spatial relationship between line tips and reference anatomy with the improvement of the AUC, increasing from 0.71 (95 % CI 0.70 - 0.75) to 0.86 (95 % CI 0.83 - 0.94). Conclusions The externally trained model exhibited limited generalizability on the local ICU dataset. Therefore, evaluating the performance of externally trained AI before integrating it into critical care routine is crucial. Rule-based algorithm may be used in combination with DL to improve results.
Collapse
Affiliation(s)
- Pootipong Wongveerasin
- Department of Radiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
| | - Trongtum Tongdee
- Department of Radiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
| | - Pairash Saiviroonporn
- Department of Radiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
| |
Collapse
|
2
|
Warren BE, Bilbily A, Gichoya JW, Conway A, Li B, Fawzy A, Barragán C, Jaberi A, Mafeld S. An Introductory Guide to Artificial Intelligence in Interventional Radiology: Part 1 Foundational Knowledge. Can Assoc Radiol J 2024; 75:558-567. [PMID: 38445497 DOI: 10.1177/08465371241236376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/07/2024] Open
Abstract
Artificial intelligence (AI) is rapidly evolving and has transformative potential for interventional radiology (IR) clinical practice. However, formal training in AI may be limited for many clinicians and therefore presents a challenge for initial implementation and trust in AI. An understanding of the foundational concepts in AI may help familiarize the interventional radiologist with the field of AI, thus facilitating understanding and participation in the development and deployment of AI. A pragmatic classification system of AI based on the complexity of the model may guide clinicians in the assessment of AI. Finally, the current state of AI in IR and the patterns of implementation are explored (pre-procedural, intra-procedural, and post-procedural).
Collapse
Affiliation(s)
- Blair Edward Warren
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| | - Alexander Bilbily
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- 16 Bit Inc., Toronto, ON, Canada
- Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | | | - Aaron Conway
- Prince Charles Hospital, Queensland University of Technology, Brisbane, QLD, Australia
| | - Ben Li
- Division of Vascular Surgery, Department of Surgery, University of Toronto, Toronto, ON, Canada
| | - Aly Fawzy
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
| | - Camilo Barragán
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| | - Arash Jaberi
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| | - Sebastian Mafeld
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| |
Collapse
|
3
|
Warren BE, Bilbily A, Gichoya JW, Chartier LB, Fawzy A, Barragán C, Jaberi A, Mafeld S. An Introductory Guide to Artificial Intelligence in Interventional Radiology: Part 2: Implementation Considerations and Harms. Can Assoc Radiol J 2024; 75:568-574. [PMID: 38445517 DOI: 10.1177/08465371241236377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/07/2024] Open
Abstract
The introduction of artificial intelligence (AI) in interventional radiology (IR) will bring about new challenges and opportunities for patients and clinicians. AI may comprise software as a medical device or AI-integrated hardware and will require a rigorous evaluation that should be guided based on the level of risk of the implementation. A hierarchy of risk of harm and possible harms are described herein. A checklist to guide deployment of an AI in a clinical IR environment is provided. As AI continues to evolve, regulation and evaluation of the AI medical devices will need to continue to evolve to keep pace and ensure patient safety.
Collapse
Affiliation(s)
- Blair Edward Warren
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| | - Alexander Bilbily
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- 16 Bit Inc., Toronto, ON, Canada
- Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | | | - Lucas B Chartier
- Division of Emergency Medicine, Department of Medicine, University of Toronto, Toronto, ON, Canada
- Department of Emergency Medicine, University Health Network, Toronto, ON, Canada
| | - Aly Fawzy
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
| | - Camilo Barragán
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| | - Arash Jaberi
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| | - Sebastian Mafeld
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| |
Collapse
|
4
|
Stogiannos N, Jennings M, George CS, Culbertson J, Salehi H, Furterer S, Pergola M, Culp MP, Malamateniou C. The American Society of Radiologic Technologists (ASRT) AI educator survey: A cross-sectional study to explore knowledge, experience, and use of AI within education. J Med Imaging Radiat Sci 2024; 55:101449. [PMID: 39004006 DOI: 10.1016/j.jmir.2024.101449] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2024] [Revised: 05/09/2024] [Accepted: 06/04/2024] [Indexed: 07/16/2024]
Abstract
INTRODUCTION Artificial Intelligence (AI) is revolutionizing medical imaging and radiation therapy. AI-powered applications are being deployed to aid Medical Radiation Technologists (MRTs) in clinical workflows, decision-making, dose optimisation, and a wide range of other tasks. Exploring the levels of AI education provided across the United States is crucial to prepare future graduates to deliver the digital future. This study aims to assess educators' levels of AI knowledge, the current state of AI educational provisions, the perceived challenges around AI education, and important factors for future advancements. METHODS An online survey was electronically administered to all radiologic technologists in the American Society of Radiologic Technologists (ASRT) database who indicated that they had an educator role in the United States. This was distributed through the membership of the ASRT, from February to April 2023. All quantitative data was analysed using frequency and descriptive statistics. The survey's open-ended questions were analysed using a conceptual content analysis approach. RESULTS Out of 5,066 educators in the ASRT database, 373 valid responses were received, resulting in a response rate of 7.4%. Despite 84.5% of educators expressing the importance of teaching AI, 23.7% currently included AI in academic curricula. Of the 76.3% that did not include AI in their curricula, lack of AI knowledge among educators was the top reason for not integrating AI in education (59.1%). Similarly, AI-enabled tools were utilised by only 11.1% of the programs to assist teaching. The levels of trust in AI varied among educators. CONCLUSION The study found that although US educators of MRTs have a good baseline knowledge of general concepts regarding AI, they could improve on the teaching and use of AI in their curricula. AI training and guidance, adequate time to develop educational resources, and funding and support from higher education institutions were key priorities as highlighted by educators.
Collapse
Affiliation(s)
- Nikolaos Stogiannos
- Department of Midwifery & Radiography, School of Health and Psychological Sciences, City, University of London, UK; Magnitiki Tomografia Kerkiras, Corfu, Greece.
| | - Michael Jennings
- Senior Research Analyst, American Society of Radiologic Technologists, New Mexico, USA
| | - Craig St George
- Director of Education, American Society of Radiologic Technologists, New Mexico, USA
| | - John Culbertson
- Director of Research, American Society of Radiologic Technologists, New Mexico, USA
| | - Hugh Salehi
- Department of Biomedical Industrial & Human Factor Engineering, Wright State University, Ohio, USA
| | - Sandra Furterer
- Department of Integrated Systems Engineering, The Ohio State University, Ohio, USA
| | - Melissa Pergola
- Chief Executive Officer, American Society of Radiologic Technologists, New Mexico, USA
| | - Melissa P Culp
- Executive Vice President of Member Engagement, American Society of Radiologic Technologists, New Mexico, USA.
| | - Christina Malamateniou
- Department of Midwifery & Radiography, School of Health and Psychological Sciences, City, University of London, UK; Discipline of Medical Imaging and Radiation Therapy, College of Medicine and Health, University College Cork, Ireland; European Society of Medical Imaging Informatics, Vienna, Austria
| |
Collapse
|
5
|
Linguraru MG, Bakas S, Aboian M, Chang PD, Flanders AE, Kalpathy-Cramer J, Kitamura FC, Lungren MP, Mongan J, Prevedello LM, Summers RM, Wu CC, Adewole M, Kahn CE. Clinical, Cultural, Computational, and Regulatory Considerations to Deploy AI in Radiology: Perspectives of RSNA and MICCAI Experts. Radiol Artif Intell 2024; 6:e240225. [PMID: 38984986 PMCID: PMC11294958 DOI: 10.1148/ryai.240225] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2024] [Revised: 04/13/2024] [Accepted: 04/25/2024] [Indexed: 07/11/2024]
Abstract
The Radiological Society of North of America (RSNA) and the Medical Image Computing and Computer Assisted Intervention (MICCAI) Society have led a series of joint panels and seminars focused on the present impact and future directions of artificial intelligence (AI) in radiology. These conversations have collected viewpoints from multidisciplinary experts in radiology, medical imaging, and machine learning on the current clinical penetration of AI technology in radiology and how it is impacted by trust, reproducibility, explainability, and accountability. The collective points-both practical and philosophical-define the cultural changes for radiologists and AI scientists working together and describe the challenges ahead for AI technologies to meet broad approval. This article presents the perspectives of experts from MICCAI and RSNA on the clinical, cultural, computational, and regulatory considerations-coupled with recommended reading materials-essential to adopt AI technology successfully in radiology and, more generally, in clinical practice. The report emphasizes the importance of collaboration to improve clinical deployment, highlights the need to integrate clinical and medical imaging data, and introduces strategies to ensure smooth and incentivized integration. Keywords: Adults and Pediatrics, Computer Applications-General (Informatics), Diagnosis, Prognosis © RSNA, 2024.
Collapse
Affiliation(s)
- Marius George Linguraru
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - Spyridon Bakas
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - Mariam Aboian
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - Peter D. Chang
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - Adam E. Flanders
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - Jayashree Kalpathy-Cramer
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - Felipe C. Kitamura
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - Matthew P. Lungren
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - John Mongan
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - Luciano M. Prevedello
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - Ronald M. Summers
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - Carol C. Wu
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - Maruf Adewole
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| | - Charles E. Kahn
- From the Sheikh Zayed Institute for Pediatric Surgical Innovation,
Children’s National Hospital, Washington, DC (M.G.L.); Divisions of
Radiology and Pediatrics, George Washington University School of Medicine and
Health Sciences, Washington, DC (M.G.L.); Division of Computational Pathology,
Department of Pathology & Laboratory Medicine, School of Medicine,
Indiana University, Indianapolis, Ind (S.B.); Department of Radiology,
Children’s Hospital of Philadelphia, Philadelphia, Pa (M.A.); Department
of Radiological Sciences, University of California Irvine, Irvine, Calif
(P.D.C.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa
(A.E.F.); Department of Ophthalmology, University of Colorado Anschutz Medical
Campus, Aurora, Colo (J.K.C.); Department of Applied Innovation and AI,
Diagnósticos da América SA (DasaInova), São Paulo, Brazil
(F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São
Paulo, São Paulo, Brazil (F.C.K.); Microsoft, Nuance, Burlington, Mass
(M.P.L.); Department of Radiology and Biomedical Imaging and Center for
Intelligent Imaging, University of California San Francisco, San Francisco,
Calif (J.M.); Department of Radiology, The Ohio State University Wexner Medical
Center, Columbus, Ohio (L.M.P.); Department of Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Bethesda, Md (R.M.S.); Division
of Diagnostic Imaging, University of Texas MD Anderson Cancer Center, Houston,
Tex (C.C.W.); Medical Artificial Intelligence Laboratory, University of Lagos
College of Medicine, Lagos, Nigeria (M.A.); and Department of Radiology,
University of Pennsylvania, 3400 Spruce St, 1 Silverstein, Philadelphia, PA
19104-6243 (C.E.K.)
| |
Collapse
|
6
|
Barat M, Milot L. Is robotic assistance the future of percutaneous interventional radiology? Diagn Interv Imaging 2024; 105:209-210. [PMID: 38403506 DOI: 10.1016/j.diii.2024.01.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Accepted: 01/23/2024] [Indexed: 02/27/2024]
Affiliation(s)
- Maxime Barat
- Department of Radiology, Hôpital Cochin, Assistance Publique-Hôpitaux de Paris, 75014 Paris, France; Génomique et Signalisation des Tumeurs Endocrines, Institut Cochin, INSERM U 1016, CNRS UMR8104, 75014 Paris, France; Université Paris Cité, Faculté de Médecine, 75006 Paris, France.
| | - Laurent Milot
- Department of Diagnostic and Interventional Radiology, Hôpital Edouard Herriot, Hospices Civils de Lyon, 69005 Lyon, France; LabTAU, INSERM U1032, 69003 Lyon, France; Université Claude Bernard Lyon 1, 69003 Lyon, France
| |
Collapse
|
7
|
Brady AP, Allen B, Chong J, Kotter E, Kottler N, Mongan J, Oakden-Rayner L, Dos Santos DP, Tang A, Wald C, Slavotinek J. Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement From the ACR, CAR, ESR, RANZCR & RSNA. Can Assoc Radiol J 2024; 75:226-244. [PMID: 38251882 DOI: 10.1177/08465371231222229] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2024] Open
Abstract
Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever‑growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi‑society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.
Collapse
Affiliation(s)
| | - Bibb Allen
- Department of Radiology, Grandview Medical Center, Birmingham, AL, USA
- Data Science Institute, American College of Radiology, Reston, VA, USA
| | - Jaron Chong
- Department of Medical Imaging, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Elmar Kotter
- Department of Diagnostic and Interventional Radiology, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Nina Kottler
- Radiology Partners, El Segundo, CA, USA
- Stanford Center for Artificial Intelligence in Medicine & Imaging, Palo Alto, CA, USA
| | - John Mongan
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, SA, Australia
| | - Daniel Pinto Dos Santos
- Department of Radiology, University Hospital of Cologne, Cologne, Germany
- Department of Radiology, University Hospital of Frankfurt, Frankfurt, Germany
| | - An Tang
- Department of Radiology, Radiation Oncology, and Nuclear Medicine, Université de Montréal, Montréal, QC, Canada
| | - Christoph Wald
- Department of Radiology, Lahey Hospital & Medical Center, Burlington, MA, USA
- Tufts University Medical School, Boston, MA, USA
- American College of Radiology, Reston, VA, USA
| | - John Slavotinek
- South Australia Medical Imaging, Flinders Medical Centre Adelaide, SA, Australia
- College of Medicine and Public Health, Flinders University, Adelaide, SA, Australia
| |
Collapse
|
8
|
Boverhof BJ, Redekop WK, Bos D, Starmans MPA, Birch J, Rockall A, Visser JJ. Radiology AI Deployment and Assessment Rubric (RADAR) to bring value-based AI into radiological practice. Insights Imaging 2024; 15:34. [PMID: 38315288 PMCID: PMC10844175 DOI: 10.1186/s13244-023-01599-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Accepted: 11/14/2023] [Indexed: 02/07/2024] Open
Abstract
OBJECTIVE To provide a comprehensive framework for value assessment of artificial intelligence (AI) in radiology. METHODS This paper presents the RADAR framework, which has been adapted from Fryback and Thornbury's imaging efficacy framework to facilitate the valuation of radiology AI from conception to local implementation. Local efficacy has been newly introduced to underscore the importance of appraising an AI technology within its local environment. Furthermore, the RADAR framework is illustrated through a myriad of study designs that help assess value. RESULTS RADAR presents a seven-level hierarchy, providing radiologists, researchers, and policymakers with a structured approach to the comprehensive assessment of value in radiology AI. RADAR is designed to be dynamic and meet the different valuation needs throughout the AI's lifecycle. Initial phases like technical and diagnostic efficacy (RADAR-1 and RADAR-2) are assessed pre-clinical deployment via in silico clinical trials and cross-sectional studies. Subsequent stages, spanning from diagnostic thinking to patient outcome efficacy (RADAR-3 to RADAR-5), require clinical integration and are explored via randomized controlled trials and cohort studies. Cost-effectiveness efficacy (RADAR-6) takes a societal perspective on financial feasibility, addressed via health-economic evaluations. The final level, RADAR-7, determines how prior valuations translate locally, evaluated through budget impact analysis, multi-criteria decision analyses, and prospective monitoring. CONCLUSION The RADAR framework offers a comprehensive framework for valuing radiology AI. Its layered, hierarchical structure, combined with a focus on local relevance, aligns RADAR seamlessly with the principles of value-based radiology. CRITICAL RELEVANCE STATEMENT The RADAR framework advances artificial intelligence in radiology by delineating a much-needed framework for comprehensive valuation. KEYPOINTS • Radiology artificial intelligence lacks a comprehensive approach to value assessment. • The RADAR framework provides a dynamic, hierarchical method for thorough valuation of radiology AI. • RADAR advances clinical radiology by bridging the artificial intelligence implementation gap.
Collapse
Affiliation(s)
- Bart-Jan Boverhof
- Erasmus School of Health Policy and Management, Erasmus University Rotterdam, Rotterdam, The Netherlands
| | - W Ken Redekop
- Erasmus School of Health Policy and Management, Erasmus University Rotterdam, Rotterdam, The Netherlands
| | - Daniel Bos
- Department of Epidemiology, Erasmus University Medical Centre, Rotterdam, The Netherlands
- Department of Radiology & Nuclear Medicine, Erasmus University Medical Centre, Rotterdam, The Netherlands
| | - Martijn P A Starmans
- Department of Radiology & Nuclear Medicine, Erasmus University Medical Centre, Rotterdam, The Netherlands
| | | | - Andrea Rockall
- Department of Surgery & Cancer, Imperial College London, London, UK
| | - Jacob J Visser
- Department of Radiology & Nuclear Medicine, Erasmus University Medical Centre, Rotterdam, The Netherlands.
| |
Collapse
|
9
|
Cai Y, Cai YQ, Tang LY, Wang YH, Gong M, Jing TC, Li HJ, Li-Ling J, Hu W, Yin Z, Gong DX, Zhang GW. Artificial intelligence in the risk prediction models of cardiovascular disease and development of an independent validation screening tool: a systematic review. BMC Med 2024; 22:56. [PMID: 38317226 PMCID: PMC10845808 DOI: 10.1186/s12916-024-03273-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/16/2023] [Accepted: 01/23/2024] [Indexed: 02/07/2024] Open
Abstract
BACKGROUND A comprehensive overview of artificial intelligence (AI) for cardiovascular disease (CVD) prediction and a screening tool of AI models (AI-Ms) for independent external validation are lacking. This systematic review aims to identify, describe, and appraise AI-Ms of CVD prediction in the general and special populations and develop a new independent validation score (IVS) for AI-Ms replicability evaluation. METHODS PubMed, Web of Science, Embase, and IEEE library were searched up to July 2021. Data extraction and analysis were performed for the populations, distribution, predictors, algorithms, etc. The risk of bias was evaluated with the prediction risk of bias assessment tool (PROBAST). Subsequently, we designed IVS for model replicability evaluation with five steps in five items, including transparency of algorithms, performance of models, feasibility of reproduction, risk of reproduction, and clinical implication, respectively. The review is registered in PROSPERO (No. CRD42021271789). RESULTS In 20,887 screened references, 79 articles (82.5% in 2017-2021) were included, which contained 114 datasets (67 in Europe and North America, but 0 in Africa). We identified 486 AI-Ms, of which the majority were in development (n = 380), but none of them had undergone independent external validation. A total of 66 idiographic algorithms were found; however, 36.4% were used only once and only 39.4% over three times. A large number of different predictors (range 5-52,000, median 21) and large-span sample size (range 80-3,660,000, median 4466) were observed. All models were at high risk of bias according to PROBAST, primarily due to the incorrect use of statistical methods. IVS analysis confirmed only 10 models as "recommended"; however, 281 and 187 were "not recommended" and "warning," respectively. CONCLUSION AI has led the digital revolution in the field of CVD prediction, but is still in the early stage of development as the defects of research design, report, and evaluation systems. The IVS we developed may contribute to independent external validation and the development of this field.
Collapse
Affiliation(s)
- Yue Cai
- China Medical University, Shenyang, 110122, China
| | - Yu-Qing Cai
- China Medical University, Shenyang, 110122, China
| | - Li-Ying Tang
- China Medical University, Shenyang, 110122, China
| | - Yi-Han Wang
- China Medical University, Shenyang, 110122, China
| | - Mengchun Gong
- Digital Health China Co. Ltd, Beijing, 100089, China
| | - Tian-Ci Jing
- Smart Hospital Management Department, the First Hospital of China Medical University, Shenyang, 110001, China
| | - Hui-Jun Li
- Shenyang Medical & Film Science and Technology Co. Ltd., Shenyang, 110001, China
- Enduring Medicine Smart Innovation Research Institute, Shenyang, 110001, China
| | - Jesse Li-Ling
- Institute of Genetic Medicine, School of Life Science, State Key Laboratory of Biotherapy, Sichuan University, Chengdu, 610065, China
| | - Wei Hu
- Bayi Orthopedic Hospital, Chengdu, 610017, China
| | - Zhihua Yin
- Department of Epidemiology, School of Public Health, China Medical University, Shenyang, 110122, China.
| | - Da-Xin Gong
- Smart Hospital Management Department, the First Hospital of China Medical University, Shenyang, 110001, China.
- The Internet Hospital Branch of the Chinese Research Hospital Association, Beijing, 100006, China.
| | - Guang-Wei Zhang
- Smart Hospital Management Department, the First Hospital of China Medical University, Shenyang, 110001, China.
- The Internet Hospital Branch of the Chinese Research Hospital Association, Beijing, 100006, China.
| |
Collapse
|
10
|
Gong B, Rowe SP, Duron L. The AI "Grid": A French national initiative as a product of radiology and industry collaboration. Diagn Interv Imaging 2024; 105:43-44. [PMID: 37880006 DOI: 10.1016/j.diii.2023.10.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2023] [Accepted: 10/09/2023] [Indexed: 10/27/2023]
Affiliation(s)
- Bo Gong
- Department of Radiology, University of British Columbia, Vancouver, BC, V6T 1M9, Canada.
| | - Steven P Rowe
- Molecular Imaging and Therapeutics, Department of Radiology, University of North Carolina School of Medicine, Chapel Hill, NC, 27599, USA
| | - Loic Duron
- Department of Neuroradiology, Hôpital Fondation Adolphe de Rothschild, Paris 75019, France
| |
Collapse
|
11
|
Brady AP, Allen B, Chong J, Kotter E, Kottler N, Mongan J, Oakden-Rayner L, Pinto Dos Santos D, Tang A, Wald C, Slavotinek J. Developing, purchasing, implementing and monitoring AI tools in radiology: Practical considerations. A multi-society statement from the ACR, CAR, ESR, RANZCR & RSNA. J Med Imaging Radiat Oncol 2024; 68:7-26. [PMID: 38259140 DOI: 10.1111/1754-9485.13612] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Accepted: 11/23/2023] [Indexed: 01/24/2024]
Abstract
Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.
Collapse
Affiliation(s)
| | - Bibb Allen
- Department of Radiology, Grandview Medical Center, Birmingham, Alabama, USA
- American College of Radiology Data Science Institute, Reston, Virginia, USA
| | - Jaron Chong
- Department of Medical Imaging, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| | - Elmar Kotter
- Department of Diagnostic and Interventional Radiology, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Nina Kottler
- Radiology Partners, El Segundo, California, USA
- Stanford Center for Artificial Intelligence in Medicine & Imaging, Palo Alto, California, USA
| | - John Mongan
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, California, USA
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, South Australia, Australia
| | - Daniel Pinto Dos Santos
- Department of Radiology, University Hospital of Cologne, Cologne, Germany
- Department of Radiology, University Hospital of Frankfurt, Frankfurt, Germany
| | - An Tang
- Department of Radiology, Radiation Oncology, and Nuclear Medicine, Université de Montréal, Montreal, Quebec, Canada
| | - Christoph Wald
- Department of Radiology, Lahey Hospital & Medical Center, Burlington, Massachusetts, USA
- Tufts University Medical School, Boston, Massachusetts, USA
- Commision On Informatics, and Member, Board of Chancellors, American College of Radiology, Reston, Virginia, USA
| | - John Slavotinek
- South Australia Medical Imaging, Flinders Medical Centre Adelaide, Adelaide, South Australia, Australia
- College of Medicine and Public Health, Flinders University, Adelaide, South Australia, Australia
| |
Collapse
|
12
|
Brady AP, Allen B, Chong J, Kotter E, Kottler N, Mongan J, Oakden-Rayner L, Dos Santos DP, Tang A, Wald C, Slavotinek J. Developing, purchasing, implementing and monitoring AI tools in radiology: practical considerations. A multi-society statement from the ACR, CAR, ESR, RANZCR & RSNA. Insights Imaging 2024; 15:16. [PMID: 38246898 PMCID: PMC10800328 DOI: 10.1186/s13244-023-01541-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2024] Open
Abstract
Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones.This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.Key points • The incorporation of artificial intelligence (AI) in radiological practice demands increased monitoring of its utility and safety.• Cooperation between developers, clinicians, and regulators will allow all involved to address ethical issues and monitor AI performance.• AI can fulfil its promise to advance patient well-being if all steps from development to integration in healthcare are rigorously evaluated.
Collapse
Affiliation(s)
| | - Bibb Allen
- Department of Radiology, Grandview Medical Center, Birmingham, AL, USA
- American College of Radiology Data Science Institute, Reston, VA, USA
| | - Jaron Chong
- Department of Medical Imaging, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Elmar Kotter
- Department of Diagnostic and Interventional Radiology, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Nina Kottler
- Radiology Partners, El Segundo, CA, USA
- Stanford Center for Artificial Intelligence in Medicine & Imaging, Palo Alto, CA, USA
| | - John Mongan
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, USA
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, Australia
| | - Daniel Pinto Dos Santos
- Department of Radiology, University Hospital of Cologne, Cologne, Germany
- Department of Radiology, University Hospital of Frankfurt, Frankfurt, Germany
| | - An Tang
- Department of Radiology, Radiation Oncology, and Nuclear Medicine, Université de Montréal, Montréal, Québec, Canada
| | - Christoph Wald
- Department of Radiology, Lahey Hospital & Medical Center, Burlington, MA, USA
- Tufts University Medical School, Boston, MA, USA
- Commision On Informatics, and Member, Board of Chancellors, American College of Radiology, Virginia, USA
| | - John Slavotinek
- South Australia Medical Imaging, Flinders Medical Centre Adelaide, Adelaide, Australia
- College of Medicine and Public Health, Flinders University, Adelaide, Australia
| |
Collapse
|
13
|
Barat M, Soyer P, Dohan A. Appropriateness of Recommendations Provided by ChatGPT to Interventional Radiologists. Can Assoc Radiol J 2023; 74:758-763. [PMID: 37052880 DOI: 10.1177/08465371231170133] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/14/2023] Open
Affiliation(s)
- Maxime Barat
- Department of Radiology, Hopital Cochin, AP-HP, Université Paris Cité, Paris, France
| | - Philippe Soyer
- Department of Radiology, Hopital Cochin, AP-HP, Université Paris Cité, Paris, France
| | - Anthony Dohan
- Department of Radiology, Hopital Cochin, AP-HP, Université Paris Cité, Paris, France
| |
Collapse
|
14
|
Khader F, Müller-Franzes G, Wang T, Han T, Tayebi Arasteh S, Haarburger C, Stegmaier J, Bressem K, Kuhl C, Nebelung S, Kather JN, Truhn D. Multimodal Deep Learning for Integrating Chest Radiographs and Clinical Parameters: A Case for Transformers. Radiology 2023; 309:e230806. [PMID: 37787671 DOI: 10.1148/radiol.230806] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/04/2023]
Abstract
Background Clinicians consider both imaging and nonimaging data when diagnosing diseases; however, current machine learning approaches primarily consider data from a single modality. Purpose To develop a neural network architecture capable of integrating multimodal patient data and compare its performance to models incorporating a single modality for diagnosing up to 25 pathologic conditions. Materials and Methods In this retrospective study, imaging and nonimaging patient data were extracted from the Medical Information Mart for Intensive Care (MIMIC) database and an internal database comprised of chest radiographs and clinical parameters inpatients in the intensive care unit (ICU) (January 2008 to December 2020). The MIMIC and internal data sets were each split into training (n = 33 893, n = 28 809), validation (n = 740, n = 7203), and test (n = 1909, n = 9004) sets. A novel transformer-based neural network architecture was trained to diagnose up to 25 conditions using nonimaging data alone, imaging data alone, or multimodal data. Diagnostic performance was assessed using area under the receiver operating characteristic curve (AUC) analysis. Results The MIMIC and internal data sets included 36 542 patients (mean age, 63 years ± 17 [SD]; 20 567 male patients) and 45 016 patients (mean age, 66 years ± 16; 27 577 male patients), respectively. The multimodal model showed improved diagnostic performance for all pathologic conditions. For the MIMIC data set, the mean AUC was 0.77 (95% CI: 0.77, 0.78) when both chest radiographs and clinical parameters were used, compared with 0.70 (95% CI: 0.69, 0.71; P < .001) for only chest radiographs and 0.72 (95% CI: 0.72, 0.73; P < .001) for only clinical parameters. These findings were confirmed on the internal data set. Conclusion A model trained on imaging and nonimaging data outperformed models trained on only one type of data for diagnosing multiple diseases in patients in an ICU setting. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Kitamura and Topol in this issue.
Collapse
Affiliation(s)
- Firas Khader
- From the Department of Diagnostic and Interventional Radiology (F.K., G.M.F., T.W., S.T.A., C.K., S.N., D.T.) and Department of Medicine III (J.N.K.), University Hospital Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Physics of Molecular Imaging Systems, Institute of Experimental Molecular Imaging (T.H.), and Institute of Imaging and Computer Vision (J.S.), RWTH Aachen University, Aachen, Germany; Ocumeda, Munich, Germany (C.H.); Department of Diagnostic and Interventional Radiology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (K.B.); Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany (J.N.K.); Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK (J.N.K.); and Department of Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany (J.N.K.)
| | - Gustav Müller-Franzes
- From the Department of Diagnostic and Interventional Radiology (F.K., G.M.F., T.W., S.T.A., C.K., S.N., D.T.) and Department of Medicine III (J.N.K.), University Hospital Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Physics of Molecular Imaging Systems, Institute of Experimental Molecular Imaging (T.H.), and Institute of Imaging and Computer Vision (J.S.), RWTH Aachen University, Aachen, Germany; Ocumeda, Munich, Germany (C.H.); Department of Diagnostic and Interventional Radiology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (K.B.); Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany (J.N.K.); Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK (J.N.K.); and Department of Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany (J.N.K.)
| | - Tianci Wang
- From the Department of Diagnostic and Interventional Radiology (F.K., G.M.F., T.W., S.T.A., C.K., S.N., D.T.) and Department of Medicine III (J.N.K.), University Hospital Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Physics of Molecular Imaging Systems, Institute of Experimental Molecular Imaging (T.H.), and Institute of Imaging and Computer Vision (J.S.), RWTH Aachen University, Aachen, Germany; Ocumeda, Munich, Germany (C.H.); Department of Diagnostic and Interventional Radiology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (K.B.); Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany (J.N.K.); Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK (J.N.K.); and Department of Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany (J.N.K.)
| | - Tianyu Han
- From the Department of Diagnostic and Interventional Radiology (F.K., G.M.F., T.W., S.T.A., C.K., S.N., D.T.) and Department of Medicine III (J.N.K.), University Hospital Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Physics of Molecular Imaging Systems, Institute of Experimental Molecular Imaging (T.H.), and Institute of Imaging and Computer Vision (J.S.), RWTH Aachen University, Aachen, Germany; Ocumeda, Munich, Germany (C.H.); Department of Diagnostic and Interventional Radiology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (K.B.); Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany (J.N.K.); Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK (J.N.K.); and Department of Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany (J.N.K.)
| | - Soroosh Tayebi Arasteh
- From the Department of Diagnostic and Interventional Radiology (F.K., G.M.F., T.W., S.T.A., C.K., S.N., D.T.) and Department of Medicine III (J.N.K.), University Hospital Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Physics of Molecular Imaging Systems, Institute of Experimental Molecular Imaging (T.H.), and Institute of Imaging and Computer Vision (J.S.), RWTH Aachen University, Aachen, Germany; Ocumeda, Munich, Germany (C.H.); Department of Diagnostic and Interventional Radiology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (K.B.); Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany (J.N.K.); Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK (J.N.K.); and Department of Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany (J.N.K.)
| | - Christoph Haarburger
- From the Department of Diagnostic and Interventional Radiology (F.K., G.M.F., T.W., S.T.A., C.K., S.N., D.T.) and Department of Medicine III (J.N.K.), University Hospital Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Physics of Molecular Imaging Systems, Institute of Experimental Molecular Imaging (T.H.), and Institute of Imaging and Computer Vision (J.S.), RWTH Aachen University, Aachen, Germany; Ocumeda, Munich, Germany (C.H.); Department of Diagnostic and Interventional Radiology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (K.B.); Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany (J.N.K.); Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK (J.N.K.); and Department of Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany (J.N.K.)
| | - Johannes Stegmaier
- From the Department of Diagnostic and Interventional Radiology (F.K., G.M.F., T.W., S.T.A., C.K., S.N., D.T.) and Department of Medicine III (J.N.K.), University Hospital Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Physics of Molecular Imaging Systems, Institute of Experimental Molecular Imaging (T.H.), and Institute of Imaging and Computer Vision (J.S.), RWTH Aachen University, Aachen, Germany; Ocumeda, Munich, Germany (C.H.); Department of Diagnostic and Interventional Radiology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (K.B.); Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany (J.N.K.); Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK (J.N.K.); and Department of Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany (J.N.K.)
| | - Keno Bressem
- From the Department of Diagnostic and Interventional Radiology (F.K., G.M.F., T.W., S.T.A., C.K., S.N., D.T.) and Department of Medicine III (J.N.K.), University Hospital Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Physics of Molecular Imaging Systems, Institute of Experimental Molecular Imaging (T.H.), and Institute of Imaging and Computer Vision (J.S.), RWTH Aachen University, Aachen, Germany; Ocumeda, Munich, Germany (C.H.); Department of Diagnostic and Interventional Radiology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (K.B.); Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany (J.N.K.); Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK (J.N.K.); and Department of Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany (J.N.K.)
| | - Christiane Kuhl
- From the Department of Diagnostic and Interventional Radiology (F.K., G.M.F., T.W., S.T.A., C.K., S.N., D.T.) and Department of Medicine III (J.N.K.), University Hospital Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Physics of Molecular Imaging Systems, Institute of Experimental Molecular Imaging (T.H.), and Institute of Imaging and Computer Vision (J.S.), RWTH Aachen University, Aachen, Germany; Ocumeda, Munich, Germany (C.H.); Department of Diagnostic and Interventional Radiology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (K.B.); Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany (J.N.K.); Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK (J.N.K.); and Department of Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany (J.N.K.)
| | - Sven Nebelung
- From the Department of Diagnostic and Interventional Radiology (F.K., G.M.F., T.W., S.T.A., C.K., S.N., D.T.) and Department of Medicine III (J.N.K.), University Hospital Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Physics of Molecular Imaging Systems, Institute of Experimental Molecular Imaging (T.H.), and Institute of Imaging and Computer Vision (J.S.), RWTH Aachen University, Aachen, Germany; Ocumeda, Munich, Germany (C.H.); Department of Diagnostic and Interventional Radiology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (K.B.); Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany (J.N.K.); Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK (J.N.K.); and Department of Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany (J.N.K.)
| | - Jakob Nikolas Kather
- From the Department of Diagnostic and Interventional Radiology (F.K., G.M.F., T.W., S.T.A., C.K., S.N., D.T.) and Department of Medicine III (J.N.K.), University Hospital Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Physics of Molecular Imaging Systems, Institute of Experimental Molecular Imaging (T.H.), and Institute of Imaging and Computer Vision (J.S.), RWTH Aachen University, Aachen, Germany; Ocumeda, Munich, Germany (C.H.); Department of Diagnostic and Interventional Radiology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (K.B.); Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany (J.N.K.); Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK (J.N.K.); and Department of Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany (J.N.K.)
| | - Daniel Truhn
- From the Department of Diagnostic and Interventional Radiology (F.K., G.M.F., T.W., S.T.A., C.K., S.N., D.T.) and Department of Medicine III (J.N.K.), University Hospital Aachen, Pauwelsstraße 30, 52074 Aachen, Germany; Physics of Molecular Imaging Systems, Institute of Experimental Molecular Imaging (T.H.), and Institute of Imaging and Computer Vision (J.S.), RWTH Aachen University, Aachen, Germany; Ocumeda, Munich, Germany (C.H.); Department of Diagnostic and Interventional Radiology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany (K.B.); Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany (J.N.K.); Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK (J.N.K.); and Department of Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany (J.N.K.)
| |
Collapse
|
15
|
Offersen CM, Sørensen J, Sheng K, Carlsen JF, Langkilde AR, Pai A, Truelsen TC, Nielsen MB. Artificial Intelligence for Automated DWI/FLAIR Mismatch Assessment on Magnetic Resonance Imaging in Stroke: A Systematic Review. Diagnostics (Basel) 2023; 13:2111. [PMID: 37371006 DOI: 10.3390/diagnostics13122111] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Revised: 06/15/2023] [Accepted: 06/16/2023] [Indexed: 06/29/2023] Open
Abstract
We conducted this Systematic Review to create an overview of the currently existing Artificial Intelligence (AI) methods for Magnetic Resonance Diffusion-Weighted Imaging (DWI)/Fluid-Attenuated Inversion Recovery (FLAIR)-mismatch assessment and to determine how well DWI/FLAIR mismatch algorithms perform compared to domain experts. We searched PubMed Medline, Ovid Embase, Scopus, Web of Science, Cochrane, and IEEE Xplore literature databases for relevant studies published between 1 January 2017 and 20 November 2022, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. We assessed the included studies using the Quality Assessment of Diagnostic Accuracy Studies 2 tool. Five studies fit the scope of this review. The area under the curve ranged from 0.74 to 0.90. The sensitivity and specificity ranged from 0.70 to 0.85 and 0.74 to 0.84, respectively. Negative predictive value, positive predictive value, and accuracy ranged from 0.55 to 0.82, 0.74 to 0.91, and 0.73 to 0.83, respectively. In a binary classification of ±4.5 h from stroke onset, the surveyed AI methods performed equivalent to or even better than domain experts. However, using the relation between time since stroke onset (TSS) and increasing visibility of FLAIR hyperintensity lesions is not recommended for the determination of TSS within the first 4.5 h. An AI algorithm on DWI/FLAIR mismatch assessment focused on treatment eligibility, outcome prediction, and consideration of patient-specific data could potentially increase the proportion of stroke patients with unknown onset who could be treated with thrombolysis.
Collapse
Affiliation(s)
- Cecilie Mørck Offersen
- Department of Radiology, Copenhagen University Hospital Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2200 Copenhagen, Denmark
| | - Jens Sørensen
- Department of Radiology, Copenhagen University Hospital Rigshospitalet, 2100 Copenhagen, Denmark
| | - Kaining Sheng
- Department of Radiology, Copenhagen University Hospital Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2200 Copenhagen, Denmark
| | - Jonathan Frederik Carlsen
- Department of Radiology, Copenhagen University Hospital Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2200 Copenhagen, Denmark
| | - Annika Reynberg Langkilde
- Department of Radiology, Copenhagen University Hospital Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2200 Copenhagen, Denmark
| | - Akshay Pai
- Department of Radiology, Copenhagen University Hospital Rigshospitalet, 2100 Copenhagen, Denmark
- Cerebriu A/S, 1127 Copenhagen, Denmark
| | - Thomas Clement Truelsen
- Department of Clinical Medicine, University of Copenhagen, 2200 Copenhagen, Denmark
- Department of Neurology, Copenhagen University Hospital Rigshospitalet, 2100 Copenhagen, Denmark
| | - Michael Bachmann Nielsen
- Department of Radiology, Copenhagen University Hospital Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2200 Copenhagen, Denmark
| |
Collapse
|
16
|
Ranjan A, Parpaleix A, Cardoso J, Adeleke S. AI vs FRCR: What it means for the future. Eur J Radiol 2023; 165:110918. [PMID: 37311341 DOI: 10.1016/j.ejrad.2023.110918] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2023] [Revised: 05/22/2023] [Accepted: 05/31/2023] [Indexed: 06/15/2023]
Abstract
A recent work by Shelmerdine et al. was published in the Christmas edition of the BMJ. The authors were inspired by George Hinton's statement that artificial intelligence (AI) would supersede radiologists, and ventured to investigate whether the AI software Milvue Suite which had been trained on a few hundred thousand chest and musculoskeletal x-rays, could pass the rapid reporting section of the FRCR - an exam which must be passed in order to practice as a consultant radiologist in the UK. This brief comment sums up the company's opinions and perspective from the practical AI developmental angle and also its translation into a commercially viable and clinically useful tool. Hoping this will provide a fair and balanced view of the role of AI in radiology.
Collapse
Affiliation(s)
- Aditi Ranjan
- Royal Berkshire Hospital NHS Foundation Trust, Reading, United Kingdom
| | | | - Jorge Cardoso
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom
| | - Sola Adeleke
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom.
| |
Collapse
|