1
|
Payne VL, Sattar U, Wright M, Hill E, Butler JM, Macpherson B, Jeppesen A, Del Fiol G, Madaras-Kelly K. Clinician perspectives on how situational context and augmented intelligence design features impact perceived usefulness of sepsis prediction scores embedded within a simulated electronic health record. J Am Med Inform Assoc 2024; 31:1331-1340. [PMID: 38661564 PMCID: PMC11105126 DOI: 10.1093/jamia/ocae089] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 03/04/2024] [Accepted: 04/05/2024] [Indexed: 04/26/2024] Open
Abstract
OBJECTIVE Obtain clinicians' perspectives on early warning scores (EWS) use within context of clinical cases. MATERIAL AND METHODS We developed cases mimicking sepsis situations. De-identified data, synthesized physician notes, and EWS representing deterioration risk were displayed in a simulated EHR for analysis. Twelve clinicians participated in semi-structured interviews to ascertain perspectives across four domains: (1) Familiarity with and understanding of artificial intelligence (AI), prediction models and risk scores; (2) Clinical reasoning processes; (3) Impression and response to EWS; and (4) Interface design. Transcripts were coded and analyzed using content and thematic analysis. RESULTS Analysis revealed clinicians have experience but limited AI and prediction/risk modeling understanding. Case assessments were primarily based on clinical data. EWS went unmentioned during initial case analysis; although when prompted to comment on it, they discussed it in subsequent cases. Clinicians were unsure how to interpret or apply the EWS, and desired evidence on its derivation and validation. Design recommendations centered around EWS display in multi-patient lists for triage, and EWS trends within the patient record. Themes included a "Trust but Verify" approach to AI and early warning information, dichotomy that EWS is helpful for triage yet has disproportional signal-to-high noise ratio, and action driven by clinical judgment, not the EWS. CONCLUSIONS Clinicians were unsure of how to apply EWS, acted on clinical data, desired score composition and validation information, and felt EWS was most useful when embedded in multi-patient views. Systems providing interactive visualization may facilitate EWS transparency and increase confidence in AI-generated information.
Collapse
Affiliation(s)
- Velma L Payne
- Kasiska Division of Health Sciences, College of Health, Idaho State University, Pocatello, ID 83209, United States
| | - Usman Sattar
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT 84108, United States
| | - Melanie Wright
- Tunnell Government Services, Inc., Bethesda, MD 20817, United States
| | - Elijah Hill
- Kasiska Division of Health Sciences, College of Pharmacy, Idaho State University, Pocatello, ID 83209, United States
| | - Jorie M Butler
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT 84108, United States
| | - Brekk Macpherson
- Virginia Commonwealth University Health System, Richmond, VA 83298, United States
| | - Amanda Jeppesen
- Kasiska Division of Health Sciences, College of Pharmacy, Idaho State University, Pocatello, ID 83209, United States
| | - Guilherme Del Fiol
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT 84108, United States
| | - Karl Madaras-Kelly
- Kasiska Division of Health Sciences, College of Pharmacy, Idaho State University, Pocatello, ID 83209, United States
| |
Collapse
|
2
|
Munro CL, Swamy L. Documentation, Data, and Decision-Making. Am J Crit Care 2024; 33:162-165. [PMID: 38688848 DOI: 10.4037/ajcc2024617] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/02/2024]
Affiliation(s)
- Cindy L Munro
- Cindy L. Munro is coeditor in chief of the American Journal of Critical Care. She is dean and professor, School of Nursing and Health Studies, University of Miami, Coral Gables, Florida
| | - Lakshman Swamy
- Lakshman Swamy is coeditor in chief of the American Journal of Critical Care. He is an instructor in Medicine at Harvard Medical School, Boston, Massachusetts, and a practicing physician in Pulmonary and Critical Care Medicine at Cambridge Health Alliance, Cambridge, Massachusetts
| |
Collapse
|
3
|
Livorsi DJ, Branch-Elliman W, Drekonja D, Echevarria KL, Fitzpatrick MA, Goetz MB, Graber CJ, Jones MM, Kelly AA, Madaras-Kelly K, Morgan DJ, Stevens VW, Suda K, Trautner BW, Ward MJ, Jump RLP. Research agenda for antibiotic stewardship within the Veterans' Health Administration, 2024-2028. Infect Control Hosp Epidemiol 2024:1-7. [PMID: 38305034 DOI: 10.1017/ice.2024.6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2024]
Affiliation(s)
- Daniel J Livorsi
- Center for Access and Delivery Research and Evaluation, Iowa City Veterans' Affairs (VA) Health Care System, Iowa City, Iowa
- Division of Infectious Diseases, University of Iowa Carver College of Medicine, Iowa City, Iowa
| | - Westyn Branch-Elliman
- VA Boston Healthcare System, Department of Medicine, Section of Infectious Diseases. Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
| | - Dimitri Drekonja
- Center for Care Delivery and Outcomes Research, Minneapolis VA Health Care System, Minneapolis, Minnesota
- Department of Medicine, University of Minnesota, Minneapolis, Minnesota
| | - Kelly L Echevarria
- VHA Pharmacy Benefits and Antimicrobial Stewardship Task Force, Department of Veterans' Affairs, Washington, DC
| | - Margaret A Fitzpatrick
- Center of Innovation for Veteran-Centered and Value-Driven Care, VA Eastern Colorado Healthcare System, Aurora, Colorado
- University of Colorado Anschutz Medical Campus, Aurora, Colorado
| | - Matthew Bidwell Goetz
- VA Greater Los Angeles Healthcare System, Los Angeles, California
- David Geffen School of Medicine at the University of California, Los Angeles, California
| | - Christopher J Graber
- VA Greater Los Angeles Healthcare System, Los Angeles, California
- David Geffen School of Medicine at the University of California, Los Angeles, California
| | - Makoto M Jones
- Informatics, Decision Enhancement, and Analytic Sciences (IDEAS) Center of Innovation, VA Salt Lake City Health Care System, Salt Lake City, Utah
- Division of Epidemiology, University of Utah School of Medicine, Salt Lake City, Utah
| | - Allison A Kelly
- VHA Pharmacy Benefits and Antimicrobial Stewardship Task Force, Department of Veterans' Affairs, Washington, DC
- Cincinnati Veterans' Affairs Medical Center, Cincinnati, Ohio
- University of Cincinnati College of Medicine, Cincinnati, Ohio
| | - Karl Madaras-Kelly
- Boise Veterans' Affairs Medical Center, Boise, Idaho
- Idaho State University, College of Pharmacy, Meridian, Idaho
| | - Daniel J Morgan
- Department of Medicine, VA Maryland Healthcare System, Baltimore, Maryland
- Center for Innovation in Diagnosis, University of Maryland School of Medicine, Baltimore, Maryland
| | - Vanessa W Stevens
- Informatics, Decision Enhancement, and Analytic Sciences (IDEAS) Center of Innovation, VA Salt Lake City Health Care System, Salt Lake City, Utah
- Division of Epidemiology, University of Utah School of Medicine, Salt Lake City, Utah
| | - Katie Suda
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania
- Division of General Internal Medicine, Department of Medicine, School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Barbara W Trautner
- Center for Innovations in Quality, Effectiveness, and Safety (IQuESt), Michael E. DeBakey Veterans' Affairs Medical Center, Houston, Texas
- Section of Health Services Research, Baylor College of Medicine, Houston, Texas
| | - Michael J Ward
- Geriatric Research, Education, and Clinical Center (GRECC), VA Tennessee Valley Healthcare System, Nashville, Tennessee
- Department of Emergency Medicine and Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Robin L P Jump
- Technology Enhancing Cognition and Health Geriatric Research Education and Clinical Center (TECH-GRECC) at the VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania
- Division of Geriatric Medicine, Department of Medicine, School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania
| |
Collapse
|