Primary Care Physicians' Perspectives on High-Quality Discharge Summaries

How to write a good discharge summary: a primer for junior physicians

A discharge summary is an important clinical document that summarizes a patient's clinical information and relevant events that occurred during hospitalization. It serves as a detailed handover of the patient's most recent and updated medical case records to general practitioners, who continue longitudinal follow-up with patients in the community and future medical care providers. A copy of the redacted/abbreviated form of the discharge summary is also usually given to patients and their caregivers so that important information, such as diagnoses, medication changes, return advice, and follow-up plans, is clearly documented. However, in reality, as discharge summaries are often written by junior physicians who may be inexperienced or have lacked medical training in this area, clinical audits often reveal poorly written discharge summaries that are unclear, inaccurate, or lack important details. Therefore, in this article, we sought to develop a simple "DISCHARGED" framework that outlines the important components of the discharge summary that we derived from a systematic search of relevant literature and further discuss several pedagogical strategies for training and assessing discharge summary writing.

Evaluating Large Language Models for Drafting Emergency Department Discharge Summaries

Importance

Large language models (LLMs) possess a range of capabilities which may be applied to the clinical domain, including text summarization. As ambient artificial intelligence scribes and other LLM-based tools begin to be deployed within healthcare settings, rigorous evaluations of the accuracy of these technologies are urgently needed.

Objective

To investigate the performance of GPT-4 and GPT-3.5-turbo in generating Emergency Department (ED) discharge summaries and evaluate the prevalence and type of errors across each section of the discharge summary.

Design

Cross-sectional study.

Setting

University of California, San Francisco ED.

Participants

We identified all adult ED visits from 2012 to 2023 with an ED clinician note. We randomly selected a sample of 100 ED visits for GPT-summarization.

Exposure

We investigate the potential of two state-of-the-art LLMs, GPT-4 and GPT-3.5-turbo, to summarize the full ED clinician note into a discharge summary.

Main Outcomes and Measures

GPT-3.5-turbo and GPT-4-generated discharge summaries were evaluated by two independent Emergency Medicine physician reviewers across three evaluation criteria: 1) Inaccuracy of GPT-summarized information; 2) Hallucination of information; 3) Omission of relevant clinical information. On identifying each error, reviewers were additionally asked to provide a brief explanation for their reasoning, which was manually classified into subgroups of errors.

Results

From 202,059 eligible ED visits, we randomly sampled 100 for GPT-generated summarization and then expert-driven evaluation. In total, 33% of summaries generated by GPT-4 and 10% of those generated by GPT-3.5-turbo were entirely error-free across all evaluated domains. Summaries generated by GPT-4 were mostly accurate, with inaccuracies found in only 10% of cases, however, 42% of the summaries exhibited hallucinations and 47% omitted clinically relevant information. Inaccuracies and hallucinations were most commonly found in the Plan sections of GPT-generated summaries, while clinical omissions were concentrated in text describing patients' Physical Examination findings or History of Presenting Complaint.

Conclusions and Relevance

In this cross-sectional study of 100 ED encounters, we found that LLMs could generate accurate discharge summaries, but were liable to hallucination and omission of clinically relevant information. A comprehensive understanding of the location and type of errors found in GPT-generated clinical text is important to facilitate clinician review of such content and prevent patient harm.

Thrown in at the deep end: a qualitative study with physicians on the purpose and challenges of discharge interviews

OBJECTIVES

Against the backdrop of poor discharge communication in hospitals, this study explores the purpose of discharge interviews from the physicians' perspective and the challenges they are confronted with. Discharge interviews are legally required in Germany as part of the discharge management. Led by the ward physician, the discharge interview should summarize relevant information about the hospital stay, medication, lifestyle interventions and follow-up treatment.

METHODS

Semi-structured interviews with n = 12 physicians were conducted at Heidelberg University Hospital between February and April 2020. Qualitative content analysis was carried out using MAXQDA.

RESULTS

Physicians reported gaining information, providing information, and answering open-ended questions as the purpose of the discharge interview. Challenges in conducting discharge interviews were related to finding a common language, patient-related challenges, conditions of everyday ward life, and lack of training. Physicians reported receiving no explicit training on discharge interviews. While professional experience seems to mitigate the lack of training, some physicians expressed a prevailing sense of insecurity.

CONCLUSION

The lack of preparation for discharge interviews in medical school makes it particularly challenging for physicians to translate their theoretical knowledge into patient-centered discharge communication. Medical training on discharge interviews should be expanded in terms of theoretical input on the ideal content, its purpose and potential (e.g. in reducing readmissions), as well as practical exercises.