The Impact of Clinical Decision Support on Health Disparities and the Digital Divide

Why do users override alerts? Utilizing large language model to summarize comments and optimize clinical decision support

OBJECTIVES

To evaluate the capability of using generative artificial intelligence (AI) in summarizing alert comments and to determine if the AI-generated summary could be used to improve clinical decision support (CDS) alerts.

MATERIALS AND METHODS

We extracted user comments to alerts generated from September 1, 2022 to September 1, 2023 at Vanderbilt University Medical Center. For a subset of 8 alerts, comment summaries were generated independently by 2 physicians and then separately by GPT-4. We surveyed 5 CDS experts to rate the human-generated and AI-generated summaries on a scale from 1 (strongly disagree) to 5 (strongly agree) for the 4 metrics: clarity, completeness, accuracy, and usefulness.

RESULTS

Five CDS experts participated in the survey. A total of 16 human-generated summaries and 8 AI-generated summaries were assessed. Among the top 8 rated summaries, five were generated by GPT-4. AI-generated summaries demonstrated high levels of clarity, accuracy, and usefulness, similar to the human-generated summaries. Moreover, AI-generated summaries exhibited significantly higher completeness and usefulness compared to the human-generated summaries (AI: 3.4 ± 1.2, human: 2.7 ± 1.2, P = .001).

CONCLUSION

End-user comments provide clinicians' immediate feedback to CDS alerts and can serve as a direct and valuable data resource for improving CDS delivery. Traditionally, these comments may not be considered in the CDS review process due to their unstructured nature, large volume, and the presence of redundant or irrelevant content. Our study demonstrates that GPT-4 is capable of distilling these comments into summaries characterized by high clarity, accuracy, and completeness. AI-generated summaries are equivalent and potentially better than human-generated summaries. These AI-generated summaries could provide CDS experts with a novel means of reviewing user comments to rapidly optimize CDS alerts both online and offline.

Advances in Clinical Decision Support Systems: Contributions from the 2022 Literature

OBJECTIVE

To summarize significant research contributions published in 2022 in the field of clinical decision support (CDS) systems and select the best papers for the Decision Support section of the International Medical Informatics Association (IMIA) Yearbook 2023.

METHODS

A renewed search query for identifying CDS scholarship was developed using Medical Subject Headings (MeSH) terms and related keywords. The query was executed in PubMed in January 2023. The search results were reviewed in three stages by two reviewers: title-based triaging, followed by abstract screening, and then full text review. The resulting articles were sent for external review to identity best paper candidates.

RESULTS

A total of 1,939 articles related to CDS were retrieved. Of these, 11 articles were selected as candidates for best papers. The general themes of the final three best papers are (1) reducing documentation burden through in-line guidance for clinical notes, (2) clinician engagement for continuous improvement of CDS, and (3) mitigating healthcare-related carbon emissions using scalable and accessible CDS, respectively.

CONCLUSION

The field of clinical decision support remains highly active and dynamic, with innovative contributions to a range of clinical domains from primary to acute care. Interoperability issues, documentation burden, clinician acceptance, and the need for effective integration into existing healthcare workflows are among the prominent challenges and areas of interest faced by CDS implementation efforts.

Informatics for One Health

OBJECTIVES

To introduce the 2023 International Medical Informatics Association (IMIA) Yearbook by the editors.

METHODS

The editorial provides an introduction and overview to the 2023 IMIA Yearbook where the special topic is "Informatics for One Health". The special topic, survey papers and some best papers are discussed. The section changes in the Yearbook editorial committee are also described.

RESULTS

IMIA Yearbook 2023 provides many perspectives on a relatively new topic called "One Digital Health". The subject is vast, and includes the use of digital technologies to promote the well-being of people and animals, but also of the environment in which they evolve. Many sections produced new work in the topic including One Health and all sections included the latest themes in many specialties in medical informatics.

CONCLUSIONS

The theme of "Informatics for One Health" is relatively new but the editors of the IMIA Yearbook have presented excellent and thought-provoking work for biomedical informatics in 2023.