Look Who's Talking: Application of a Theory-Based Taxonomy to Patient-Clinician E-mail Messages

Differences in Patient Secure Message Volume Among Otolaryngologists: A Retrospective Cohort Study

OBJECTIVE

To identify differences in inbox and secure message burden among otolaryngologists based on demographics and subspecialty over 4 years.

METHODS

Inbox data were queried from January 2019 until December 2022. Otolaryngologists were categorized into cohorts by area of practice and gender. All inbox tasks, secure messages, and clinical encounters were collected and compared by gender, practice type, and years in practice. Means were compared using t-tests and chi-squared tests.

RESULTS

Of the 128 physicians, 45.7% were comprehensive otolaryngologists and 61.3% were male. The most common subspecialties were facial plastics (15.6%), oncology (8.6%), and otology (7.8%). Otolaryngologists had an average of 143.5 inbox tasks per month, with 97.2 (67.7%) of them being secure messages, resulting in an average of 1.14 inbox tasks and 0.80 secure messages per clinical encounter. The ratio of secure messages per clinical encounter was consistent across all specialties except oncology (1.10, P = .003). Otology (0.86, P = .032) and facial plastics (0.95, P = .028) had significantly lower ratios of inbox tasks to clinical encounters when compared to their colleagues, while oncology had a higher ratio (1.70, P < .001). No significant differences in inbox burden were observed between genders, years in practice, or languages spoken. Secure messages steadily increased over the study period.

CONCLUSION

Inbox burden for otolaryngologists primarily stems from patient secure messages and varies across subspecialties. Considerations should be made to the inbox burden of head and neck oncologists. The implementation of support systems for inbox management could improve the imbalance between clinical and non-clinical responsibilities in otolaryngology.

LEVEL OF EVIDENCE

Level III, Retrospective Cohort Study.

Leveraging large language models for generating responses to patient messages-a subjective analysis

OBJECTIVE

This study aimed to develop and assess the performance of fine-tuned large language models for generating responses to patient messages sent via an electronic health record patient portal.

MATERIALS AND METHODS

Utilizing a dataset of messages and responses extracted from the patient portal at a large academic medical center, we developed a model (CLAIR-Short) based on a pre-trained large language model (LLaMA-65B). In addition, we used the OpenAI API to update physician responses from an open-source dataset into a format with informative paragraphs that offered patient education while emphasizing empathy and professionalism. By combining with this dataset, we further fine-tuned our model (CLAIR-Long). To evaluate fine-tuned models, we used 10 representative patient portal questions in primary care to generate responses. We asked primary care physicians to review generated responses from our models and ChatGPT and rated them for empathy, responsiveness, accuracy, and usefulness.

RESULTS

The dataset consisted of 499 794 pairs of patient messages and corresponding responses from the patient portal, with 5000 patient messages and ChatGPT-updated responses from an online platform. Four primary care physicians participated in the survey. CLAIR-Short exhibited the ability to generate concise responses similar to provider's responses. CLAIR-Long responses provided increased patient educational content compared to CLAIR-Short and were rated similarly to ChatGPT's responses, receiving positive evaluations for responsiveness, empathy, and accuracy, while receiving a neutral rating for usefulness.

CONCLUSION

This subjective analysis suggests that leveraging large language models to generate responses to patient messages demonstrates significant potential in facilitating communication between patients and healthcare providers.

Approach to the Patient With Headache

OBJECTIVE

The evaluation of patients with headache relies heavily on the history. This article reviews key questions for diagnosing primary and secondary headache disorders with a rationale for each and phrasing to optimize the information obtained and the patient's experience.

LATEST DEVELOPMENTS

The availability of online resources for clinicians and patients continues to increase, including sites that use artificial intelligence to generate a diagnosis and report based on patient responses online. Patient-friendly headache apps include calendars that help track treatment response, identify triggers, and provide educational information.

ESSENTIAL POINTS

A structured approach to taking the history, incorporating online resources and other technologies when needed, facilitates making an accurate diagnosis and often eliminates the need for unnecessary testing. A detailed yet empathetic approach incorporating interpersonal skills enhances relationship building and trust, both of which are integral to successful treatment.

Leveraging Large Language Models for Generating Responses to Patient Messages

Objective

This study aimed to develop and assess the performance of fine-tuned large language models for generating responses to patient messages sent via an electronic health record patient portal.

Methods

Utilizing a dataset of messages and responses extracted from the patient portal at a large academic medical center, we developed a model (CLAIR-Short) based on a pre-trained large language model (LLaMA-65B). In addition, we used the OpenAI API to update physician responses from an open-source dataset into a format with informative paragraphs that offered patient education while emphasizing empathy and professionalism. By combining with this dataset, we further fine-tuned our model (CLAIR-Long). To evaluate the fine-tuned models, we used ten representative patient portal questions in primary care to generate responses. We asked primary care physicians to review generated responses from our models and ChatGPT and rated them for empathy, responsiveness, accuracy, and usefulness.

Results

The dataset consisted of a total of 499,794 pairs of patient messages and corresponding responses from the patient portal, with 5,000 patient messages and ChatGPT-updated responses from an online platform. Four primary care physicians participated in the survey. CLAIR-Short exhibited the ability to generate concise responses similar to provider's responses. CLAIR-Long responses provided increased patient educational content compared to CLAIR-Short and were rated similarly to ChatGPT's responses, receiving positive evaluations for responsiveness, empathy, and accuracy, while receiving a neutral rating for usefulness.

Conclusion

Leveraging large language models to generate responses to patient messages demonstrates significant potential in facilitating communication between patients and primary care providers.

Clinicians' Attitudes and Behaviors Towards Communicating Electronically with Patients: A Grounded Practical Theory Approach

Secure messaging (SM), asynchronous communication between patients and clinicians, is an increasingly popular tool among patients to contact clinicians about their care. Despite patients' enthusiasm, clinicians have been hesitant to embrace the technology to communicate with patients. Using the theoretical and methodological framework of Grounded Practical Theory (GPT), we analyzed and interpreted clinicians' perceptions, attitudes, and approaches toward SM to communicate with patients. Twenty clinicians in medical oncology and radiation oncology participated in audio-recorded, semi-structured interviews. Findings revealed the problems with using SM, such as difficulty interpreting low-quality messages, the amount of time needed to devote to responding, and its potential to negatively affect the patient-clinician relationship. Techniques employed to manage such problems consisted of using different forms of communication and utilizing messaging to expedite workloads. The philosophical rationale of clinicians toward SM was that it can improve patient care and this form of communication is already embedded within existing patient care. Overall, this article clarifies how clinicians can re-conceptualize how they think about SM so that it becomes a productive, informative, and useful aspect of patient care.

Associations Between Patient-Provider Secure Message Content and Patients' Health Care Visits

Background: Between-visit communications can play a vital role in improving intermediate patient outcomes such as access to care and satisfaction. Secure messaging is a growing modality for these communications, but research is limited about the influence of message content on those intermediate outcomes. We examined associations between secure message content and patients' number of health care visits. Methods: Our study included 2,111 adult patients with hypertension and/or diabetes and 18,309 patient- and staff-generated messages. We estimated incident rate ratios (IRRs) for associations between taxonomic codes assigned to message content, and the number of office, emergency department, and inpatient visits. Results: Patients who initiated message threads in 2017 had higher numbers of outpatient visits (p < 0.001) compared with patients who did not initiate threads. Among patients who initiated threads, we identified an inverse relationship between outpatient visits and preventive care scheduling requests (IRR = 0.92; 95% confidence interval [CI]: 0.86-0.98) and requests for appointments for new conditions (IRR = 0.95; 95% CI: 0.92-0.99). Patients with higher proportions of request denials or more follow-up appointment requests had more emergency department visits compared with patients who received or sent other content (IRR = 1.18; 95% CI: 1.03-1.34 and IRR = 1.14; 95% CI: 1.07-1.23, respectively). We identified a positive association between outpatient visits and the proportion of threads that lacked a clinic response (IRR = 1.02; 95% CI: 1.00-1.03). Discussion: We report on the first analyses to examine associations between message content and health care visits. Conclusions: Our findings are relevant to understanding how to better use secure messaging to support patients and their care.

Classification of patient- and clinician-generated secure messages using a theory-based taxonomy

BACKGROUND

As secure electronic message exchange increases between patients and clinicians, we must explore and understand how patients and clinicians use those messages to communicate between clinical visits.

OBJECTIVE

To present the application of a taxonomy developed specifically to code secure message content in a way that allows for identification of patient and clinician communication functions demonstrated to be associated with patients' intermediate and health outcomes.

METHOD

We randomly sampled 1031 patients who sent and received 18 309 messages and coded those messages with codes from our taxonomy. We present the prevalence of each taxon (ie, code) within the sample.

RESULTS

The most common taxon among initial patient-generated messages were Information seeking (29.09%), followed by Scheduling requests (27.91%), and Prescription requests (23.09%). Over half of subsequent patient-generated messages included responses to clinic staffs' questions (58.31%). Six in 10 clinic staff responses included some form of Information sharing with process-based responses being most common (32.81%). A third of all clinician-generated messages (36.28%) included acknowledgement or some level of fulfilment of a patient's task-oriented request. Clinic staff sought information from patients in 20.54% of their messages.

CONCLUSION

This taxonomy is the first step toward examining whether secure messaging communication can be associated with patients' health outcomes. Knowing which content is positively associated with outcomes can support training of, and targeted responses from, clinicians with the goal of generating message content designed to improve outcomes.

PATIENT CONTRIBUTION

This study is based on analyses of patient-initiated secure message threads.

Associations Between Patient Health Outcomes and Secure Message Content Exchanged Between Patients and Clinicians: Retrospective Cohort Study

Background

The number of electronic messages securely exchanged between clinic staff and patients has risen dramatically over the last decade. A variety of studies explored whether the volume of messages sent by patients was associated with outcomes. None of these studies, however, examined whether message content itself was associated with outcomes. Because secure messaging is a significant form of communication between patients and clinic staff, it is critical to evaluate the context of the communication to best understand its impact on patient health outcomes.

Objective

To examine associations between patients’ and clinicians’ message content and changes in patients’ health outcomes.

Methods

We applied a taxonomy developed specifically for secure messages to 14,394 patient- and clinic staff–generated messages derived from patient-initiated message threads. Our study population included 1602 patients, 50.94% (n=816) of whom initiated message threads. We conducted linear regression analyses to determine whether message codes were associated with changes in glycemic (A1C) levels in patients with diabetes and changes in systolic (SBP) and diastolic (DBP) blood pressure in patients with hypertension.

Results

Patients who initiated threads had larger declines in A1Cs (P=.01) compared to patients who did not initiate threads. Clinic nonresponse was associated with decreased SBP (β=–.30; 95% CI –0.56 to –0.04), as were staffs’ action responses (β=–30; 95% CI –0.58 to –0.02). Increased DBP, SBP, and A1C levels were associated with patient-generated appreciation and praise messages and staff encouragement with effect sizes ranging from 0.51 (A1C) to 5.80 (SBP). We found improvements in SBP associated with patients’ complaints (β=–4.03; 95% CI –7.94 to –0.12). Deferred information sharing by clinic staff was associated with increased SBP (β=1.29; 95% CI 0.4 to 2.19).

Conclusions

This is the first research to find associations between message content and patients’ health outcomes. Our findings indicate mixed associations between patient message content and patient outcomes. Further research is needed to understand the implications of this work; in the meantime, health care providers should be aware that their message content may influence patient health outcomes.