|
1
|
Stogiannos N, Gillan C, Precht H, Reis CSD, Kumar A, O'Regan T, Ellis V, Barnes A, Meades R, Pogose M, Greggio J, Scurr E, Kumar S, King G, Rosewarne D, Jones C, van Leeuwen KG, Hyde E, Beardmore C, Alliende JG, El-Farra S, Papathanasiou S, Beger J, Nash J, van Ooijen P, Zelenyanszki C, Koch B, Langmack KA, Tucker R, Goh V, Turmezei T, Lip G, Reyes-Aldasoro CC, Alonso E, Dean G, Hirani SP, Torre S, Akudjedu TN, Ohene-Botwe B, Khine R, O'Sullivan C, Kyratsis Y, McEntee M, Wheatstone P, Thackray Y, Cairns J, Jerome D, Scarsbrook A, Malamateniou C. A multidisciplinary team and multiagency approach for AI implementation: A commentary for medical imaging and radiotherapy key stakeholders. J Med Imaging Radiat Sci 2024; 55:101717. [PMID: 39067309 DOI: 10.1016/j.jmir.2024.101717] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2024] [Accepted: 06/27/2024] [Indexed: 07/30/2024]
Affiliation(s)
- Nikolaos Stogiannos
- Division of Midwifery & Radiography, City, University of London, United Kingdom; Magnitiki Tomografia Kerkiras, Corfu, Greece.
| | - Caitlin Gillan
- Joint Department of Medical Imaging, University Health Network, Canada; Departments of Radiation Oncology & Medical Imaging, University of Toronto, Toronto, Canada
| | - Helle Precht
- Health Sciences Research Centre, UCL University College, Radiology Department, Lillebelt Hospital, University Hospitals of Southern Denmark, Institute of Regional Health Research, University of Southern Denmark, Discipline of Medical Imaging and Radiation Therapy, University College Cork, Ireland
| | - Claudia Sa Dos Reis
- School of Health Sciences (HESAV), University of Applied Sciences and Arts Western Switzerland (HES-SO), Lausanne, Switzerland
| | - Amrita Kumar
- Frimley Health NHS Foundation Trust, British Institute of Radiology, United Kingdom
| | - Tracy O'Regan
- The Society and College of Radiographers, London, United Kingdom
| | | | - Anna Barnes
- King's Technology Evaluation Centre, School of biomedical engineering and imaging sciences, King's College London, United Kingdom
| | - Richard Meades
- Department of Nuclear Medicine, Royal Free London NHS Foundation, London, United Kingdom
| | | | - Julien Greggio
- Division of Midwifery & Radiography, City, University of London, United Kingdom; Italian Association of MR Radiographers, Cagliari, Italy
| | - Erica Scurr
- Department of Radiology, Royal Marsden Hospital, London, United Kingdom
| | | | - Graham King
- Annalise.ai Pty Ltd, Sydney, Australia; AI Special Focus Group, AXREM Association of Healthcare Technology Providers for Imaging Radiotherapy and Care, London, United Kingdom
| | | | - Catherine Jones
- Royal Brisbane and Womens' Hospital, Brisbane, Australia; I-MED Radiology, Brisbane, Australia; Faculty of Medicine and Health, University of Sydney, Sydney, Australia
| | - Kicky G van Leeuwen
- Romion Health, Utrecht, the Netherlands; Health AI Register, Utrecht, the Netherlands
| | - Emma Hyde
- University of Derby, Derby, United Kingdom
| | | | | | - Samar El-Farra
- Emirates Medical Society - The Radiographers Society of Emirates (RASE), United Arab Emirates
| | | | - Jan Beger
- Science and Technology Organisation, GE HealthCare, United States
| | - Jonathan Nash
- University Hospitals Sussex, United Kingdom; Kheiron Medical Technologies, London, United Kingdom; British Society of Breast Radiology, the Netherlands
| | - Peter van Ooijen
- Dept of Radiotherapy and Data Science Center in Health (DASH), University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
| | - Christiane Zelenyanszki
- Community Diagnostics, Barking, Havering and Redbridge University Hospitals NHS Trust, United Kingdom
| | - Barbara Koch
- Jheronimus Academy of Data Science, the Netherlands; Tilburg University, the Netherlands
| | | | | | - Vicky Goh
- School of Biomedical Engineering and Imaging Sciences, King's College London. Department of Radiology, Guy's & St Thomas' NHS Foundation Trust, London, United Kingdom
| | - Tom Turmezei
- Norwich Medical School, University of East Anglia, United Kingdom; Department of Radiology, Norfolk and Norwich University Hospitals NHS Foundation Trust, Norwich, United Kingdom
| | | | | | - Eduardo Alonso
- Artificial Intelligence Research Centre, City, University of London, United Kingdom
| | - Geraldine Dean
- ESTH NHS Trust, United Kingdom; NHS SW London Imaging Network, United Kingdom
| | - Shashivadan P Hirani
- Centre for Healthcare Innovation Research, City, University of London, London, United Kingdom
| | - Sofia Torre
- Frimley Health Foundation Trust, United Kingdom
| | - Theophilus N Akudjedu
- Institute of Medical Imaging & Visualisation, Department of Medical Science & Public Health, Faculty of Health & Social Sciences, Bournemouth University, United Kingdom
| | - Benard Ohene-Botwe
- Department of Midwifery & Radiography, City, University of London, United Kingdom
| | - Ricardo Khine
- Institute of Health Sciences Education, Faculty of Medicine and Dentistry, Queen Mary, University of London, United Kingdom
| | - Chris O'Sullivan
- Department of Midwifery & Radiography, School of Health & Psychological Sciences, City, University of London, United Kingdom
| | - Yiannis Kyratsis
- Erasmus School of Health Policy and Management, Erasmus University Rotterdam, the Netherlands
| | - Mark McEntee
- Discipline of Medical Imaging and Radiation Therapy, University College Cork, Ireland; Institute of Regional Health Research, University of Southern Denmark, Denmark; Faculty of Health Sciences, The University of Sydney, Australia
| | | | | | - James Cairns
- Department of Radiology, Leeds Teaching Hospitals NHS Trust, Leeds, United Kingdom; Faculty of Medicine & Health, University of Leeds, Leeds, United Kingdom
| | | | - Andrew Scarsbrook
- Department of Radiology, Leeds Teaching Hospitals NHS Trust, Leeds, United Kingdom; Faculty of Medicine & Health, University of Leeds, Leeds, United Kingdom
| | - Christina Malamateniou
- Department of Midwifery & Radiography, City, University of London, United Kingdom; European Society of Medical Imaging Informatics, Vienna, Austria
| |
Collapse
|
|
2
|
Payne VL, Sattar U, Wright M, Hill E, Butler JM, Macpherson B, Jeppesen A, Del Fiol G, Madaras-Kelly K. Clinician perspectives on how situational context and augmented intelligence design features impact perceived usefulness of sepsis prediction scores embedded within a simulated electronic health record. J Am Med Inform Assoc 2024; 31:1331-1340. [PMID: 38661564 PMCID: PMC11105126 DOI: 10.1093/jamia/ocae089] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 03/04/2024] [Accepted: 04/05/2024] [Indexed: 04/26/2024] Open
Abstract
OBJECTIVE Obtain clinicians' perspectives on early warning scores (EWS) use within context of clinical cases. MATERIAL AND METHODS We developed cases mimicking sepsis situations. De-identified data, synthesized physician notes, and EWS representing deterioration risk were displayed in a simulated EHR for analysis. Twelve clinicians participated in semi-structured interviews to ascertain perspectives across four domains: (1) Familiarity with and understanding of artificial intelligence (AI), prediction models and risk scores; (2) Clinical reasoning processes; (3) Impression and response to EWS; and (4) Interface design. Transcripts were coded and analyzed using content and thematic analysis. RESULTS Analysis revealed clinicians have experience but limited AI and prediction/risk modeling understanding. Case assessments were primarily based on clinical data. EWS went unmentioned during initial case analysis; although when prompted to comment on it, they discussed it in subsequent cases. Clinicians were unsure how to interpret or apply the EWS, and desired evidence on its derivation and validation. Design recommendations centered around EWS display in multi-patient lists for triage, and EWS trends within the patient record. Themes included a "Trust but Verify" approach to AI and early warning information, dichotomy that EWS is helpful for triage yet has disproportional signal-to-high noise ratio, and action driven by clinical judgment, not the EWS. CONCLUSIONS Clinicians were unsure of how to apply EWS, acted on clinical data, desired score composition and validation information, and felt EWS was most useful when embedded in multi-patient views. Systems providing interactive visualization may facilitate EWS transparency and increase confidence in AI-generated information.
Collapse
Affiliation(s)
- Velma L Payne
- Kasiska Division of Health Sciences, College of Health, Idaho State University, Pocatello, ID 83209, United States
| | - Usman Sattar
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT 84108, United States
| | - Melanie Wright
- Tunnell Government Services, Inc., Bethesda, MD 20817, United States
| | - Elijah Hill
- Kasiska Division of Health Sciences, College of Pharmacy, Idaho State University, Pocatello, ID 83209, United States
| | - Jorie M Butler
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT 84108, United States
| | - Brekk Macpherson
- Virginia Commonwealth University Health System, Richmond, VA 83298, United States
| | - Amanda Jeppesen
- Kasiska Division of Health Sciences, College of Pharmacy, Idaho State University, Pocatello, ID 83209, United States
| | - Guilherme Del Fiol
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT 84108, United States
| | - Karl Madaras-Kelly
- Kasiska Division of Health Sciences, College of Pharmacy, Idaho State University, Pocatello, ID 83209, United States
| |
Collapse
|
|
4
|
Livorsi DJ, Branch-Elliman W, Drekonja D, Echevarria KL, Fitzpatrick MA, Goetz MB, Graber CJ, Jones MM, Kelly AA, Madaras-Kelly K, Morgan DJ, Stevens VW, Suda K, Trautner BW, Ward MJ, Jump RLP. Research agenda for antibiotic stewardship within the Veterans' Health Administration, 2024-2028. Infect Control Hosp Epidemiol 2024:1-7. [PMID: 38305034 PMCID: PMC11294492 DOI: 10.1017/ice.2024.6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2024]
Affiliation(s)
- Daniel J Livorsi
- Center for Access and Delivery Research and Evaluation, Iowa City Veterans' Affairs (VA) Health Care System, Iowa City, Iowa
- Division of Infectious Diseases, University of Iowa Carver College of Medicine, Iowa City, Iowa
| | - Westyn Branch-Elliman
- VA Boston Healthcare System, Department of Medicine, Section of Infectious Diseases. Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
| | - Dimitri Drekonja
- Center for Care Delivery and Outcomes Research, Minneapolis VA Health Care System, Minneapolis, Minnesota
- Department of Medicine, University of Minnesota, Minneapolis, Minnesota
| | - Kelly L Echevarria
- VHA Pharmacy Benefits and Antimicrobial Stewardship Task Force, Department of Veterans' Affairs, Washington, DC
| | - Margaret A Fitzpatrick
- Center of Innovation for Veteran-Centered and Value-Driven Care, VA Eastern Colorado Healthcare System, Aurora, Colorado
- University of Colorado Anschutz Medical Campus, Aurora, Colorado
| | - Matthew Bidwell Goetz
- VA Greater Los Angeles Healthcare System, Los Angeles, California
- David Geffen School of Medicine at the University of California, Los Angeles, California
| | - Christopher J Graber
- VA Greater Los Angeles Healthcare System, Los Angeles, California
- David Geffen School of Medicine at the University of California, Los Angeles, California
| | - Makoto M Jones
- Informatics, Decision Enhancement, and Analytic Sciences (IDEAS) Center of Innovation, VA Salt Lake City Health Care System, Salt Lake City, Utah
- Division of Epidemiology, University of Utah School of Medicine, Salt Lake City, Utah
| | - Allison A Kelly
- VHA Pharmacy Benefits and Antimicrobial Stewardship Task Force, Department of Veterans' Affairs, Washington, DC
- Cincinnati Veterans' Affairs Medical Center, Cincinnati, Ohio
- University of Cincinnati College of Medicine, Cincinnati, Ohio
| | - Karl Madaras-Kelly
- Boise Veterans' Affairs Medical Center, Boise, Idaho
- Idaho State University, College of Pharmacy, Meridian, Idaho
| | - Daniel J Morgan
- Department of Medicine, VA Maryland Healthcare System, Baltimore, Maryland
- Center for Innovation in Diagnosis, University of Maryland School of Medicine, Baltimore, Maryland
| | - Vanessa W Stevens
- Informatics, Decision Enhancement, and Analytic Sciences (IDEAS) Center of Innovation, VA Salt Lake City Health Care System, Salt Lake City, Utah
- Division of Epidemiology, University of Utah School of Medicine, Salt Lake City, Utah
| | - Katie Suda
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania
- Division of General Internal Medicine, Department of Medicine, School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Barbara W Trautner
- Center for Innovations in Quality, Effectiveness, and Safety (IQuESt), Michael E. DeBakey Veterans' Affairs Medical Center, Houston, Texas
- Section of Health Services Research, Baylor College of Medicine, Houston, Texas
| | - Michael J Ward
- Geriatric Research, Education, and Clinical Center (GRECC), VA Tennessee Valley Healthcare System, Nashville, Tennessee
- Department of Emergency Medicine and Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Robin L P Jump
- Technology Enhancing Cognition and Health Geriatric Research Education and Clinical Center (TECH-GRECC) at the VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania
- Division of Geriatric Medicine, Department of Medicine, School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania
| |
Collapse
|