Tethered to the EHR: Primary Care Physician Workload Assessment Using EHR Event Log Data and Time-Motion Observations

The contextual interview: a cross-cutting patient-interviewing approach for social context

Patient interviewing pedagogy in medical education has not evolved to comprehensively capture the biopsychosocial model of healthcare delivery. While gathering a patient's social history targets important aspects of social context it does not adequately capture and account for the real-time reassessment required to understand evolving factors that influence exposure to drivers of health inequities, social determinants of health, and access to supports that promote health. The authors offer a patient interviewing approach called the Contextual Interview (CI) that specifically targets dynamic and ever-changing social context information. To substantiate the use of the CI in medical education, the authors conducted a qualitative review of the Accreditation Council for Graduate Medical Education Milestones for primary care specialties (Family Medicine, Internal Medicine, and Pediatrics). Milestones were coded to the extent to which they reflected the learner's need to acknowledge, assess, synthesize and/or apply patient contextual data in real-time patient encounters. Approximately 1 in 5 milestones met the context-related and patient-facing criteria. This milestone review further highlights the need for more intentional training in eliciting meaningful social context data during patient interviewing. The CI as a cross-cutting, practical, time-conscious, and semi-structured patient interviewing approach that deliberately elicits information to improve the clinician's sense and understanding of a patient's social context. The authors reviewed future directions in researching adapted versions of the CI for undergraduate and graduate medical education.

Primary care clinician engagement in implementing a machine-learning algorithm for targeted screening of familial hypercholesterolemia

Objective

To assess the impact of a multi-pronged educational approach on the knowledge, attitudes, and behaviors regarding Familial Hypercholesterolemia (FH) management at a large academic medical center with the aim of empowering primary care clinicians (PCC) to diagnose and treat FH.

Methods

A comprehensive educational program for PCCs on FH management was developed and piloted from July 2022 to March 2024. Components of our intervention included: 1. Implementation of a novel clinical decision support tool in the electronic medical record for FH management, 2. Development and dissemination of an interactive educational website focused on FH and its management, 3. Delivery of virtual instructional sessions to increase awareness of the tool, provide education on its use, and obtain support from institutional leadership, and 4. Direct outreach to a pilot subset of PCCs whose patients had been detected using the validated FIND FH® machine learning algorithm. Participating clinicians were surveyed at baseline before the intervention and after the educational session.

Results

70 PCC consented to participate in the study with a survey completion rate of 79 % (n = 55) and 42 % (n = 23) for the baseline and follow-up surveys, respectively. Objective PCC knowledge scores improved from 40 to 65 % of responders correctly responding to at least 2/3rds of survey questions. Despite the fact that 87 % identified PCC's as most effective for early detection of FH, 100 % of PCCs who received direct outreach chose to defer care to an outpatient cardiologist over pursuing workup in the primary care setting.

Conclusion

Empowering PCCs in management of FH serves as a key strategy in addressing this underdiagnosed and undertreated potentially life-threatening condition. A systems-based approach to addressing these aims may include leveraging EMR-based clinical decision support models and cross-disciplinary educational partnerships with medical specialists.

Billing for Electronic Patient-Physician Communications: An Ethical Analysis

This review paper analyzes the ethical implications of billing patients for electronic communication with physicians through electronic health records, a practice already adopted by medical institutions such as the Cleveland Clinic. The analysis assesses how billing aligns with pillars of medical ethics which include beneficence, respect for persons, and justice. Although billing may enhance communication, improve patient care, and alleviate physician burnout, concerns arise over potential consequences on patient autonomy, trust, and health care disparities. The review delves into the intricate balance of these ethical principles by first considering the potential benefits of incentivizing concise questions and improving physician workload management through billing. By reducing messages, this approach can potentially mitigate burnout and enhance care. It also acknowledges potential drawbacks such as deterring patients because of financial constraints and eroding trust in physicians and the medical team. It emphasizes the necessity of thoroughly examining all aspects of this intricate ethical dilemma to formulate a nuanced solution that protects patient well-being while respecting physicians. We propose a middle-ground approach involving nominal and transparent billing on the basis of the question's complexity, urgency, and level of expertise required in the response. Transparent billing policies, up-front communication of costs, and potential fee waivers on the basis of socioeconomic status can address equity concerns and maintain patient trust. Striking a balance between the potential benefits and drawbacks of billing for patient questions is crucial in maintaining ethical patient-physician interactions and equitable health care provision. The analysis underscores the importance of aligning online patient-physician communication with ethical principles within the evolving digital health care landscape.

Impact of response bias in three surveys on primary care providers' experiences with electronic health records

OBJECTIVE

To identify impacts of different survey methodologies assessing primary care physicians' (PCPs') experiences with electronic health records (EHRs), we compared three surveys: the 2022 Continuous Certification Questionnaire (CCQ) from the American Board of Family Medicine, the 2022 University of California San Francisco (UCSF) Physician Health IT Survey, and the 2021 National Electronic Health Records Survey (NEHRS).

MATERIALS AND METHODS

We evaluated differences between survey pairs using Rao-Scott corrected chi-square tests, which account for weighting.

RESULTS

CCQ received 3991 responses from PCPs (100% response rate), UCSF received 1375 (3.6% response rate), and NEHRS received 858 (18.2% response rate). Substantial, statistically significant differences in demographics were detected across the surveys. CCQ respondents were younger and more likely to work in a health system; NEHRS respondents were more likely to work in private practice; and UCSF respondents disproportionately practiced in larger academic settings. Many EHR experience indicators were similar between CCQ and NEHRS, but CCQ respondents reported higher documentation burden.

DISCUSSION

The UCSF approach is unlikely to supply reliable data. Significant demographic differences between CCQ and NEHRS raise response bias concerns, and while there were similarities in some reported EHR experiences, there were important, significant differences.

CONCLUSION

Federal EHR policy monitoring and maintenance require reliable data. This test of existing and alternative sources suggest that diversified data sources are necessary to understand physicians' experiences with EHRs and interoperability. Comprehensive surveys administered by specialty boards have the potential to contribute to these efforts, since they are likely to be free of response bias.

Primary Care in the United States: Past, Present and Future

Even though a well-functioning primary care system is widely acknowledged as critical to population health, the number of primary care physicians (PCPs) practicing in the United States has steadily declined, and PCPs are in short supply. The reasons are multiple and include inadequate income relative to other specialties, excessive administrative demands on PCPs and the lack of respect given to primary care specialties during medical school and residency. Advanced practice nurses can augment the services of primary care physicians but cannot substitute for them. To change this situation, we need action on several fronts. Medical schools should give preference to students who are more likely to enter the primary care specialties. The income gap between primary care and other specialties should be narrowed. The administrative load placed on PCPs, including cumbersome electronic medical records, must be lessened. Insurers, including Medicare and Medicaid, must provide the resources to allow primary care physicians to act as leaders of multidisciplinary teams.

Consistency is key: documentation distribution and efficiency in primary care

OBJECTIVES

We analyzed the degree to which daily documentation patterns in primary care varied and whether specific patterns, consistency over time, and deviations from clinicians' usual patterns were associated with note-writing efficiency.

MATERIALS AND METHODS

We used electronic health record (EHR) active use data from the Oracle Cerner Advance platform capturing hourly active documentation time for 498 physicians and advance practice clinicians (eg, nurse practitioners) for 65 152 clinic days. We used k-means clustering to identify distinct daily patterns of active documentation time and analyzed the relationship between these patterns and active documentation time per note. We determined each primary care clinician's (PCC) modal documentation pattern and analyzed how consistency and deviations were related to documentation efficiency.

RESULTS

We identified 8 distinct daily documentation patterns; the 3 most common patterns accounted for 80.6% of PCC-days and differed primarily in average volume of documentation time (78.1 minutes per day; 35.4 minutes per day; 144.6 minutes per day); associations with note efficiency were mixed. PCCs with >80% of days attributable to a single pattern demonstrated significantly more efficient documentation than PCCs with lower consistency; for high-consistency PCCs, days that deviated from their usual patterns were associated with less efficient documentation.

DISCUSSION

We found substantial variation in efficiency across daily documentation patterns, suggesting that PCC-level factors like EHR facility and consistency may be more important than when documentation occurs. There were substantial efficiency returns to consistency, and deviations from consistent patterns were costly.

CONCLUSION

Organizational leaders aiming to reduce documentation burden should pay specific attention to the ability for PCCs to execute consistent documentation patterns day-to-day.

Effect of Ambient Voice Technology, Natural Language Processing, and Artificial Intelligence on the Patient-Physician Relationship

BACKGROUND

 The method of documentation during a clinical encounter may affect the patient-physician relationship.

OBJECTIVES

 Evaluate how the use of ambient voice recognition, coupled with natural language processing and artificial intelligence (DAX), affects the patient-physician relationship.

METHODS

 This was a prospective observational study with a primary aim of evaluating any difference in patient satisfaction on the Patient-Doctor Relationship Questionnaire-9 (PDRQ-9) scale between primary care encounters in which DAX was utilized for documentation as compared to another method. A single-arm open-label phase was also performed to query direct feedback from patients.

RESULTS

 A total of 288 patients were include in the open-label arm and 304 patients were included in the masked phase of the study comparing encounters with and without DAX use. In the open-label phase, patients strongly agreed that the provider was more focused on them, spent less time typing, and made the encounter feel more personable. In the masked phase of the study, no difference was seen in the total PDRQ-9 score between patients whose encounters used DAX (median: 45, interquartile range [IQR]: 8) and those who did not (median: 45 [IQR: 3.5]; p = 0.31). The adjusted odds ratio for DAX use was 0.8 (95% confidence interval: 0.48-1.34) for the patient reporting complete satisfaction on how well their clinician listened to them during their encounter.

CONCLUSION

 Patients strongly agreed with the use of ambient voice recognition, coupled with natural language processing and artificial intelligence (DAX) for documentation in primary care. However, no difference was detected in the patient-physician relationship on the PDRQ-9 scale.

Measuring Documentation Burden in Healthcare

BACKGROUND

The enactment of the Health Information Technology for Economic and Clinical Health Act and the wide adoption of electronic health record (EHR) systems have ushered in increasing documentation burden, frequently cited as a key factor affecting the work experience of healthcare professionals and a contributor to burnout. This systematic review aims to identify and characterize measures of documentation burden.

METHODS

We integrated discussions with Key Informants and a comprehensive search of the literature, including MEDLINE, Embase, Scopus, and gray literature published between 2010 and 2023. Data were narratively and thematically synthesized.

RESULTS

We identified 135 articles about measuring documentation burden. We classified measures into 11 categories: overall time spent in EHR, activities related to clinical documentation, inbox management, time spent in clinical review, time spent in orders, work outside work/after hours, administrative tasks (billing and insurance related), fragmentation of workflow, measures of efficiency, EHR activity rate, and usability. The most common source of data for most measures was EHR usage logs. Direct tracking such as through time-motion analysis was fairly uncommon. Measures were developed and applied across various settings and populations, with physicians and nurses in the USA being the most frequently represented healthcare professionals. Evidence of validity of these measures was limited and incomplete. Data on the appropriateness of measures in terms of scalability, feasibility, or equity across various contexts were limited. The physician perspective was the most robustly captured and prominently focused on increased stress and burnout.

DISCUSSION

Numerous measures for documentation burden are available and have been tested in a variety of settings and contexts. However, most are one-dimensional, do not capture various domains of this construct, and lack robust validity evidence. This report serves as a call to action highlighting an urgent need for measure development that represents diverse clinical contexts and support future interventions.

Postdischarge needs identified by an automated text messaging program: A mixed-methods study

BACKGROUND

Text messaging has emerged as a popular strategy to engage patients after hospital discharge. Little is known about how patients use these programs and what types of needs are addressed through this approach.

OBJECTIVE

The goal of this study was to describe the types and timing of postdischarge needs identified during a 30-day automated texting program.

METHODS

The program ran from January to August 2021 at a primary care practice in Philadelphia. In this mixed-methods study, two reviewers conducted a directed content analysis of patient needs expressed during the program, categorizing them along a well-known transitional care framework. We describe the frequency of need categories and their timing relative to discharge.

RESULTS

A total of 405 individuals were enrolled; the mean (SD) age was 62.7 (16.2); 64.2% were female; 47.4% were Black; and 49.9% had Medicare insurance. Of this population, 178 (44.0%) expressed at least one need during the 30-day program. The most frequent needs addressed were related to symptoms (26.8%), coordinating follow-up care (20.4%), and medication issues (15.7%). The mean (SD) number of days from discharge to need was 10.8 (7.9); there were no significant differences in timing based on need category.

CONCLUSIONS

The needs identified via an automated texting program were concentrated in three areas relevant to primary care practice and within nursing scope of practice. This program can serve as a model for health systems looking to support transitions through an operationally efficient approach, and the findings of this analysis can inform future iterations of this type of program.

Accuracy, thoroughness, and quality of outpatient primary care documentation in the U.S. Department of Veterans Affairs

BACKGROUND

Electronic health records (EHRs) can accelerate documentation and may enhance details of notes, or complicate documentation and introduce errors. Comprehensive assessment of documentation quality requires comparing documentation to what transpires during the clinical encounter itself. We assessed outpatient primary care notes and corresponding recorded encounters to determine accuracy, thoroughness, and several additional key measures of documentation quality.

METHODS

Patients and primary care clinicians across five midwestern primary care clinics of the US Department of Veterans Affairs were recruited into a prospective observational study. Clinical encounters were video-recorded and transcribed verbatim. Using the Physician Documentation Quality Instrument (PDQI-9) added to other measures, reviewers scored quality of the documentation by comparing transcripts to corresponding encounter notes. PDQI-9 items were scored from 1 to 5, with higher scores indicating higher quality.

RESULTS

Encounters (N = 49) among 11 clinicians were analyzed. Most issues that patients initiated in discussion were omitted from notes, and nearly half of notes referred to information or observations that could not be verified. Four notes lacked concluding assessments and plans; nine lacked information about when patients should return. Except for thoroughness, PDQI-9 items that were assessed achieved quality scores exceeding 4 of 5 points.

CONCLUSIONS

Among outpatient primary care electronic records examined, most issues that patients initiated in discussion were absent from notes, and nearly half of notes referred to information or observations absent from transcripts. EHRs may contribute to certain kinds of errors. Approaches to improving documentation should consider the roles of the EHR, patient, and clinician together.

The incremental design of a machine learning framework for medical records processing

OBJECTIVES

This work presents the development and evaluation of coordn8, a web-based application that streamlines fax processing in outpatient clinics using a "human-in-the-loop" machine learning framework. We demonstrate the effectiveness of the platform at reducing fax processing time and producing accurate machine learning inferences across the tasks of patient identification, document classification, spam classification, and duplicate document detection.

METHODS

We deployed coordn8 in 11 outpatient clinics and conducted a time savings analysis by observing users and measuring fax processing event logs. We used statistical methods to evaluate the machine learning components across different datasets to show generalizability. We conducted a time series analysis to show variations in model performance as new clinics were onboarded and to demonstrate our approach to mitigating model drift.

RESULTS

Our observation analysis showed a mean reduction in individual fax processing time by 147.5 s, while our event log analysis of over 7000 faxes reinforced this finding. Document classification produced an accuracy of 81.6%, patient identification produced an accuracy of 83.7%, spam classification produced an accuracy of 98.4%, and duplicate document detection produced a precision of 81.0%. Retraining document classification increased accuracy by 10.2%.

DISCUSSION

coordn8 significantly decreased fax-processing time and produced accurate machine learning inferences. Our human-in-the-loop framework facilitated the collection of high-quality data necessary for model training. Expanding to new clinics correlated with performance decline, which was mitigated through model retraining.

CONCLUSION

Our framework for automating clinical tasks with machine learning offers a template for health systems looking to implement similar technologies.

Measuring cognitive effort using tabular transformer-based language models of electronic health record-based audit log action sequences

OBJECTIVES

To develop and validate a novel measure, action entropy, for assessing the cognitive effort associated with electronic health record (EHR)-based work activities.

MATERIALS AND METHODS

EHR-based audit logs of attending physicians and advanced practice providers (APPs) from four surgical intensive care units in 2019 were included. Neural language models (LMs) were trained and validated separately for attendings' and APPs' action sequences. Action entropy was calculated as the cross-entropy associated with the predicted probability of the next action, based on prior actions. To validate the measure, a matched pairs study was conducted to assess the difference in action entropy during known high cognitive effort scenarios, namely, attention switching between patients and to or from the EHR inbox.

RESULTS

Sixty-five clinicians performing 5 904 429 EHR-based audit log actions on 8956 unique patients were included. All attention switching scenarios were associated with a higher action entropy compared to non-switching scenarios (P < .001), except for the from-inbox switching scenario among APPs. The highest difference among attendings was for the from-inbox attention switching: Action entropy was 1.288 (95% CI, 1.256-1.320) standard deviations (SDs) higher for switching compared to non-switching scenarios. For APPs, the highest difference was for the to-inbox switching, where action entropy was 2.354 (95% CI, 2.311-2.397) SDs higher for switching compared to non-switching scenarios.

DISCUSSION

We developed a LM-based metric, action entropy, for assessing cognitive burden associated with EHR-based actions. The metric showed discriminant validity and statistical significance when evaluated against known situations of high cognitive effort (ie, attention switching). With additional validation, this metric can potentially be used as a screening tool for assessing behavioral action phenotypes that are associated with higher cognitive burden.

CONCLUSION

An LM-based action entropy metric-relying on sequences of EHR actions-offers opportunities for assessing cognitive effort in EHR-based workflows.

Common stressors, coping processes, and professional help-seeking of medical professionals in Hong Kong: A qualitative study

Despite the high prevalence of perceived stress and mental health problems among medical professionals (MPs), their professional help-seeking is extremely low. This qualitative study explored MPs' stressors, stress-coping, barriers and facilitators of professional help-seeking. 10 MPs (30% male, Mage = 34.8 years) were recruited by purposive-sampling for views from different roles/settings. Thematic analyses revealed five central stressors: emerging novel diseases, challenges from technology-advancement, patient-communication difficulties, lack of workplace mental health care culture, excessive workload/manpower shortage. Participants predominantly used peer support/supervision and de-stress activities for stress-coping. Five factors affecting professional help-seeking were time constraint versus flexibility, mental health stigma versus de-stigmatization, concern over confidentiality/anonymity versus sense of privacy, worry about damage on professional role versus least work disruption, doubts of service providers versus perceived efficacy. All participants indicated a preference for online mental health service delivery. Results reflected unmet needs and service gaps from MPs' perspectives for the development of future interventions.

Patient care in complex Sociotechnological ecosystems and learning health systems

The learning health system (LHS) model was proposed to provide real-time, bi-directional flow of learning using data captured in health information technology systems to deliver rapid learning in healthcare delivery. As highlighted by the landmark National Academy of Medicine report "Crossing the Quality Chasm," the U.S. healthcare delivery industry represents complex adaptive systems, and there is an urgent need to develop innovative methods to identify efficient team structures by harnessing real-world care delivery data found in the electronic health record (EHR). We offer a discussion surrounding the complexities of team communication and how solutions may be guided by theories such as the Multiteam System (MTS) framework and the Multitheoretical Multilevel Framework of Communication Networks. To advance healthcare delivery science and promote LHSs, our team has been building a new line of research using EHR data to study MTS in the complex real world of cancer care delivery. We are developing new network metrics to study MTSs and will be analyzing the impact of EHR communication network structures on patient outcomes. As this research leads to patient care delivery interventions/tools, healthcare leaders and healthcare professionals can effectively use health IT data to implement the most evidence-based collaboration approaches in order to achieve the optimal LHS and patient outcomes.

Leveraging large language models for generating responses to patient messages-a subjective analysis

OBJECTIVE

This study aimed to develop and assess the performance of fine-tuned large language models for generating responses to patient messages sent via an electronic health record patient portal.

MATERIALS AND METHODS

Utilizing a dataset of messages and responses extracted from the patient portal at a large academic medical center, we developed a model (CLAIR-Short) based on a pre-trained large language model (LLaMA-65B). In addition, we used the OpenAI API to update physician responses from an open-source dataset into a format with informative paragraphs that offered patient education while emphasizing empathy and professionalism. By combining with this dataset, we further fine-tuned our model (CLAIR-Long). To evaluate fine-tuned models, we used 10 representative patient portal questions in primary care to generate responses. We asked primary care physicians to review generated responses from our models and ChatGPT and rated them for empathy, responsiveness, accuracy, and usefulness.

RESULTS

The dataset consisted of 499 794 pairs of patient messages and corresponding responses from the patient portal, with 5000 patient messages and ChatGPT-updated responses from an online platform. Four primary care physicians participated in the survey. CLAIR-Short exhibited the ability to generate concise responses similar to provider's responses. CLAIR-Long responses provided increased patient educational content compared to CLAIR-Short and were rated similarly to ChatGPT's responses, receiving positive evaluations for responsiveness, empathy, and accuracy, while receiving a neutral rating for usefulness.

CONCLUSION

This subjective analysis suggests that leveraging large language models to generate responses to patient messages demonstrates significant potential in facilitating communication between patients and healthcare providers.

Physician Health Series Part One: Characteristics of Physicians as Patients

This is the first article of a 3-part series about physician health. In this installment, we outline the unique characteristics of physicians as patients, challenges and opportunities presented by physician-patients, and recommendations for treating physicians. Future articles will delve into role clarity, occupational considerations, mental health, and interactions with third parties such as the physician's employer or licensing board. Ultimately, this series will help treating clinicians provide the best care to their physician-patients and successfully navigate the unique challenges that may arise, especially when the diagnosis may have an impact on their ability to practice medicine.

Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions

Artificial intelligence (AI) in oncology is advancing beyond algorithm development to integration into clinical practice. This review describes the current state of the field, with a specific focus on clinical integration. AI applications are structured according to cancer type and clinical domain, focusing on the four most common cancers and tasks of detection, diagnosis, and treatment. These applications encompass various data modalities, including imaging, genomics, and medical records. We conclude with a summary of existing challenges, evolving solutions, and potential future directions for the field.

SIGNIFICANCE

AI is increasingly being applied to all aspects of oncology, where several applications are maturing beyond research and development to direct clinical integration. This review summarizes the current state of the field through the lens of clinical translation along the clinical care continuum. Emerging areas are also highlighted, along with common challenges, evolving solutions, and potential future directions for the field.

Primary Care Physician Preferences Regarding Communication from Orthopaedic Surgeons

BACKGROUND

Musculoskeletal consultations constitute a growing portion of primary care physician (PCP) referrals. Optimizing communication between PCPs and orthopaedists can potentially reduce time spent in the electronic medical record (EMR) as well as physician burnout. Little is known about the preferences of PCPs regarding communication from orthopaedic surgeons. Hence, the present study investigated, across a large health network, the preferences of PCPs regarding communication from orthopaedists.

METHODS

A total of 175 PCPs across 15 practices within our health network were surveyed. These providers universally utilized Epic as their EMR platform. Five-point, labeled Likert scales were utilized to assess the PCP-perceived importance of communication from orthopaedists in specific clinical scenarios. PCPs were further asked to report their preferred method of communication in each scenario and their overall interest in communication from orthopaedists. Logistic regression analyses were performed to determine whether any PCP characteristics were associated with the preferred method of communication and the overall PCP interest in communication from orthopaedists.

RESULTS

A total of 107 PCPs (61.1%) responded to the survey. PCPs most commonly rated communication from orthopaedists as highly important in the scenario of an orthopaedist needing information from the PCP. In this scenario, PCPs preferred to receive an Epic Staff Message. Scenarios involving a recommendation for surgery, hospitalization, or a major clinical change were also rated as highly important. In these scenarios, an Epic CC'd Chart rather than a Staff Message was preferred. Increased after-hours EMR use was associated with diminished odds of having a high interest in communication from orthopaedists (odds ratio, 0.65; 95% confidence interval, 0.48 to 0.88; p = 0.005). Ninety-three PCPs (86.9%) reported spending 1 to 1.5 hours or more per day in Epic after normal clinical hours, and 27 (25.2%) spent >3 hours per day. Forty-six PCPs (43.0%) reported experiencing ≥1 symptom of burnout.

CONCLUSIONS

There were distinct preferences among PCPs regarding clinical communication from orthopaedic surgeons. There was also evidence of substantial burnout and after-hours work effort by PCPs. These results may help to optimize communication between PCPs and orthopaedists while reducing the amount of time that PCPs spend in the EMR.

When rules turn into tools: An activity theory-based perspective on implementation processes and unintended consequences

The idea that actions of people, organizations or governments may lead to Unintended Consequences (UICs) is not new. In health, UICs have been reported as a result of various interventions including quality improvement initiatives, health information technology implementation, and knowledge translation, especially those involving translation of broad policies (evidence-based medicine and patient-centred care) or system level improvement into actionable items or tools. While some unintended consequences cannot be anticipated, others may be predictable. In this article, we present a model based on cultural historical activity theory, which may help policy-makers, health leaders, and researchers better anticipate UICs resulting from implementation of new programs or technologies and take action to address them or mitigate their risk of occurrence. We support this model using examples of UICs of implementing family centred care principles, electronic health records, and computerized templates for quality improvement in chronic disease management.

Virtual Scribes and Physician Time Spent on Electronic Health Records

Importance

Time on the electronic health record (EHR) is associated with burnout among physicians. Newer virtual scribe models, which enable support from either a real-time or asynchronous scribe, have the potential to reduce the burden of the EHR and EHR-related documentation.

Objective

To characterize the association of use of virtual scribes with changes in physicians' EHR time and note and order composition and to identify the physician, scribe, and scribe response factors associated with changes in EHR time upon virtual scribe use.

Design, Setting, and Participants

Retrospective, pre-post quality improvement study of 144 physicians across specialties who had used a scribe for at least 3 months from January 2020 to September 2022, were affiliated with Brigham and Women's Hospital and Massachusetts General Hospital, and cared for patients in the outpatient setting. Data were analyzed from November 2022 to January 2024.

Exposure

Use of either a real-time or asynchronous virtual scribe.

Main Outcomes

Total EHR time, time on notes, and pajama time (5:30 pm to 7:00 am on weekdays and nonscheduled weekends and holidays), all per appointment; proportion of the note written by the physician and team contribution to orders.

Results

The main study sample included 144 unique physicians who had used a virtual scribe for at least 3 months in 152 unique scribe participation episodes (134 [88.2%] had used an asynchronous scribe service). Nearly two-thirds of the physicians (91 physicians [63.2%]) were female and more than half (86 physicians [59.7%]) were in primary care specialties. Use of a virtual scribe was associated with significant decreases in total EHR time per appointment (mean [SD] of 5.6 [16.4] minutes; P < .001) in the 3 months after vs the 3 months prior to scribe use. Scribe use was also associated with significant decreases in note time per appointment and pajama time per appointment (mean [SD] of 1.3 [3.3] minutes; P < .001 and 1.1 [4.0] minutes; P = .004). In a multivariable linear regression model, the following factors were associated with significant decreases in total EHR time per appointment with a scribe use at 3 months: practicing in a medical specialty (-7.8; 95% CI, -13.4 to -2.2 minutes), greater baseline EHR time per appointment (-0.3; 95% CI, -0.4 to -0.2 minutes per additional minute of baseline EHR time), and decrease in the percentage of the note contributed by the physician (-9.1; 95% CI, -17.3 to -0.8 minutes for every percentage point decrease).

Conclusions and Relevance

In 2 academic medical centers, use of virtual scribes was associated with significant decreases in total EHR time, time spent on notes, and pajama time, all per appointment. Virtual scribes may be particularly effective among medical specialists and those physicians with greater baseline EHR time.

An Electronic Health Record-Integrated Application for Standardizing Care and Monitoring Patients With Autosomal Dominant Polycystic Kidney Disease Enrolled in a Tolvaptan Clinic: Design and Implementation Study

Background

Tolvaptan is the only US Food and Drug Administration-approved drug to slow the progression of autosomal dominant polycystic kidney disease (ADPKD), but it requires strict clinical monitoring due to potential serious adverse events.

Objective

We aimed to share our experience in developing and implementing an electronic health record (EHR)-based application to monitor patients with ADPKD who were initiated on tolvaptan.

Methods

The application was developed in collaboration with clinical informatics professionals based on our clinical protocol with frequent laboratory test monitoring to detect early drug-related toxicity. The application streamlined the clinical workflow and enabled our nursing team to take appropriate actions in real time to prevent drug-related serious adverse events. We retrospectively analyzed the characteristics of the enrolled patients.

Results

As of September 2022, a total of 214 patients were enrolled in the tolvaptan program across all Mayo Clinic sites. Of these, 126 were enrolled in the Tolvaptan Monitoring Registry application and 88 in the Past Tolvaptan Patients application. The mean age at enrollment was 43.1 (SD 9.9) years. A total of 20 (9.3%) patients developed liver toxicity, but only 5 (2.3%) had to discontinue the drug. The 2 EHR-based applications allowed consolidation of all necessary patient information and real-time data management at the individual or population level. This approach facilitated efficient staff workflow, monitoring of drug-related adverse events, and timely prescription renewal.

Conclusions

Our study highlights the feasibility of integrating digital applications into the EHR workflow to facilitate efficient and safe care delivery for patients enrolled in a tolvaptan program. This workflow needs further validation but could be extended to other health care systems managing chronic diseases requiring drug monitoring.

Exploring the Impact of In Basket Metrics on the Adoption of a New Electronic Health Record System Among Specialists in a Tertiary Hospital in Alberta: Descriptive Study

BACKGROUND

Health care organizations implement electronic health record (EHR) systems with the expectation of improved patient care and enhanced provider performance. However, while these technologies hold the potential to create improved care and system efficiencies, they can also lead to unintended negative consequences, such as patient safety issues, communication problems, and provider burnout.

OBJECTIVE

This study aims to document metrics related to the In Basket communication hub (time in In Basket per day, time in In Basket per appointment, In Basket messages received per day, and turnaround time) of the EHR system implemented by Alberta Health Services, the province-wide health delivery system called Connect Care (Epic Systems). The objective was to identify how a newly implemented EHR system was used, the timing of its use, and the duration of use specifically related to In Basket activities.

METHODS

A descriptive study was conducted. Due to the diversity of specialties, the providers were grouped into medical and surgical based on previous similar studies. The participants were further subgrouped based on their self-reported clinical full-time equivalent (FTE ) measure. This resulted in 3 subgroups for analysis: medical FTE <0.5, medical FTE >0.5, and surgical (all of whom reported FTE >0.5). The analysis was limited to outpatient clinical interactions and explicitly excluded inpatient activities.

RESULTS

A total of 72 participants from 19 different specialties enrolled in this study. The providers had, on average, 8.31 appointments per day during the reporting periods. The providers received, on average, 21.93 messages per day, and they spent 7.61 minutes on average in the time in In Basket per day metric and 1.84 minutes on average in the time in In Basket per appointment metric. The time for the providers to mark messages as done (turnaround time) was on average 11.45 days during the reporting period. Although the surgical group had, on average, approximately twice as many appointments per scheduled day, they spent considerably less connected time (based on almost all time metrics) than the medical group. However, the surgical group took much longer than the medical group to mark messages as done (turnaround time).

CONCLUSIONS

We observed a range of patterns with no consistent direction. There does not seem to be evidence of a "learning curve," which would have shown a consistent reduction in time spent on the system over time due to familiarity and experience. While this study does not show how the included metrics could be used as predictors of providers' satisfaction or feelings of burnout, the use trends could be used to start discussions about future Canadian studies needed in this area.

Assessing the Usability and Feasibility of Digital Assistant Tools for Direct Support Professionals: Participatory Design and Pilot-Testing

BACKGROUND

The United States is experiencing a direct support professional (DSP) crisis, with demand far exceeding supply. Although generating documentation is a critical responsibility, it is one of the most wearisome aspects of DSPs' jobs. Technology that enables DSPs to log informal time-stamped notes throughout their shift could help reduce the burden of end-of-shift documentation and increase job satisfaction, which in turn could improve the quality of life of the individuals with intellectual and developmental disabilities (IDDs) whom DSPs support. However, DSPs, with varied ages, levels of education, and comfort using technology, are not likely to adopt tools that detract from caregiving responsibilities or increase workload; therefore, technological tools for them must be relatively simple, extremely intuitive, and provide highly valued capabilities.

OBJECTIVE

This paper describes the development and pilot-testing of a digital assistant tool (DAT) that enables DSPs to create informal notes throughout their shifts and use these notes to facilitate end-of-shift documentation. The purpose of the pilot study was to assess the usability and feasibility of the DAT.

METHODS

The research team applied an established user-centered participatory design process to design, develop, and test the DAT prototypes between May 2020 and April 2023. Pilot-testing entailed having 14 DSPs who support adults with IDDs use the first full implementation of the DAT prototypes during 2 or 3 successive work shifts and fill out demographic and usability questionnaires.

RESULTS

Participants used the DAT prototypes to create notes and help generate end-of-shift reports. The System Usability Scale score of 81.79 indicates that they found the prototypes easy to use. Survey responses imply that using the DAT made it easier for participants to produce required documentation and suggest that they would adopt the DAT if this tool were available for daily use.

CONCLUSIONS

Simple technologies such as the DAT prototypes, which enable DSPs to use mobile devices to log time-stamped notes throughout their shift with minimal effort and use the notes to help write reports, have the potential to both reduce the burden associated with producing documentation and enhance the quality (level of detail and accuracy) of this documentation. This could help to increase job satisfaction and reduce turnover in DSPs, both of which would help improve the quality of life of the individuals with IDDs whom they support. The pilot test results indicate that DSPs found the DAT easy to use. Next steps include (1) producing more robust versions of the DAT with additional capabilities, such as storing data locally on mobile devices when Wi-Fi is not available; and (2) eliciting input from agency directors, families, and others who use data about adults with IDDs to help care for them to ensure that data produced by DSPs are relevant and useful.

The association between use of ambient voice technology documentation during primary care patient encounters, documentation burden, and provider burnout

BACKGROUND

The burden of documentation in the electronic medical record has been cited as a major factor in provider burnout. The aim of this study was to evaluate the association between ambient voice technology, coupled with natural language processing and artificial intelligence (DAX™), on primary care provider documentation burden and burnout.

METHODS

An observational study of 110 primary care providers within a community teaching health system. The primary objectives were to determine the association between DAX™ usage and provider burnout scores on the Oldenburg Burnout Inventory (OLBI) as well as the effect on documentation time per patient encounter (minutes).

RESULTS

The completion rate for the survey was 75% (83/110) and high DAX™ use (>60% of encounters) was seen in 28% of providers (23/83). High DAX™ use was associated with significantly less burnout on the OLBI disengagement sub-score (MD [Mean Difference] -2.1; 95% confidence interval [CI] -3.8 to -0.4) but not the OLBI disengagement sub-score (-1.0; 95% CI -2.9 to 1.0) or total score (MD -3.0; 95% CI -6.4 to 0.3). Nineteen providers with high implementation of DAX™ had pre and postimplementation data on documentation time per encounter. After DAX™ implementation average documentation time in notes per encounter was significantly reduced by 28.8% (1.8 min; 95% CI 1.4-2.2).

CONCLUSIONS

The use of ambient voice technology during patient encounters was associated with significantly reduced documentation burden and primary care provider disengagement but not total provider burnout scores.

Impact of Electronic Health Record Use on Cognitive Load and Burnout Among Clinicians: Narrative Review

The cognitive load theory suggests that completing a task relies on the interplay between sensory input, working memory, and long-term memory. Cognitive overload occurs when the working memory's limited capacity is exceeded due to excessive information processing. In health care, clinicians face increasing cognitive load as the complexity of patient care has risen, leading to potential burnout. Electronic health records (EHRs) have become a common feature in modern health care, offering improved access to data and the ability to provide better patient care. They have been added to the electronic ecosystem alongside emails and other resources, such as guidelines and literature searches. Concerns have arisen in recent years that despite many benefits, the use of EHRs may lead to cognitive overload, which can impact the performance and well-being of clinicians. We aimed to review the impact of EHR use on cognitive load and how it correlates with physician burnout. Additionally, we wanted to identify potential strategies recommended in the literature that could be implemented to decrease the cognitive burden associated with the use of EHRs, with the goal of reducing clinician burnout. Using a comprehensive literature review on the topic, we have explored the link between EHR use, cognitive load, and burnout among health care professionals. We have also noted key factors that can help reduce EHR-related cognitive load, which may help reduce clinician burnout. The research findings suggest that inadequate efforts to present large amounts of clinical data to users in a manner that allows the user to control the cognitive burden in the EHR and the complexity of the user interfaces, thus adding more "work" to tasks, can lead to cognitive overload and burnout; this calls for strategies to mitigate these effects. Several factors, such as the presentation of information in the EHR, the specialty, the health care setting, and the time spent completing documentation and navigating systems, can contribute to this excess cognitive load and result in burnout. Potential strategies to mitigate this might include improving user interfaces, streamlining information, and reducing documentation burden requirements for clinicians. New technologies may facilitate these strategies. The review highlights the importance of addressing cognitive overload as one of the unintended consequences of EHR adoption and potential strategies for mitigation, identifying gaps in the current literature that require further exploration.

Predicting Primary Care Physician Burnout From Electronic Health Record Use Measures

OBJECTIVE

To evaluate the ability of routinely collected electronic health record (EHR) use measures to predict clinical work units at increased risk of burnout and potentially most in need of targeted interventions.

METHODS

In this observational study of primary care physicians, we compiled clinical workload and EHR efficiency measures, then linked these measures to 2 years of well-being surveys (using the Stanford Professional Fulfillment Index) conducted from April 1, 2019, through October 16, 2020. Physicians were grouped into training and confirmation data sets to develop predictive models for burnout. We used gradient boosting classifier and other prediction modeling algorithms to quantify the predictive performance by the area under the receiver operating characteristics curve (AUC).

RESULTS

Of 278 invited physicians from across 60 clinics, 233 (84%) completed 396 surveys. Physicians were 67% women with a median age category of 45 to 49 years. Aggregate burnout score was in the high range (≥3.325/10) on 111 of 396 (28%) surveys. Gradient boosting classifier of EHR use measures to predict burnout achieved an AUC of 0.59 (95% CI, 0.48 to 0.77) and an area under the precision-recall curve of 0.29 (95% CI, 0.20 to 0.66). Other models' confirmation set AUCs ranged from 0.56 (random forest) to 0.66 (penalized linear regression followed by dichotomization). Among the most predictive features were physician age, team member contributions to notes, and orders placed with user-defined preferences. Clinic-level aggregate measures identified the top quartile of clinics with 56% sensitivity and 85% specificity.

CONCLUSION

In a sample of primary care physicians, routinely collected EHR use measures demonstrated limited ability to predict individual burnout and moderate ability to identify high-risk clinics.

Tracking persistent postoperative opioid use: a proof-of-concept study demonstrating a use case for natural language processing

BACKGROUND

Large language models have been gaining tremendous popularity since the introduction of ChatGPT in late 2022. Perioperative pain providers should leverage natural language processing (NLP) technology and explore pertinent use cases to improve patient care. One example is tracking persistent postoperative opioid use after surgery. Since much of the relevant data may be 'hidden' within unstructured clinical text, NLP models may prove to be advantageous. The primary objective of this proof-of-concept study was to demonstrate the ability of an NLP engine to review clinical notes and accurately identify patients who had persistent postoperative opioid use after major spine surgery.

METHODS

Clinical documents from all patients that underwent major spine surgery during July 2015-August 2021 were extracted from the electronic health record. The primary outcome was persistent postoperative opioid use, defined as continued use of opioids greater than or equal to 3 months after surgery. This outcome was ascertained via manual clinician review from outpatient spine surgery follow-up notes. An NLP engine was applied to these notes to ascertain the presence of persistent opioid use-this was then compared with results from clinician manual review.

RESULTS

The final study sample consisted of 965 patients, in which 705 (73.1%) were determined to have persistent opioid use following surgery. The NLP engine correctly determined the patients' opioid use status in 92.9% of cases, in which it correctly identified persistent opioid use in 95.6% of cases and no persistent opioid use in 86.1% of cases.

DISCUSSION

Access to unstructured data within the perioperative history can contextualize patients' opioid use and provide further insight into the opioid crisis, while at the same time improve care directly at the patient level. While these goals are in reach, future work is needed to evaluate how to best implement NLP within different healthcare systems for use in clinical decision support.

AI-Generated Draft Replies Integrated Into Health Records and Physicians' Electronic Communication

Importance

Timely tests are warranted to assess the association between generative artificial intelligence (GenAI) use and physicians' work efforts.

Objective

To investigate the association between GenAI-drafted replies for patient messages and physician time spent on answering messages and the length of replies.

Design, Setting, and Participants

Randomized waiting list quality improvement (QI) study from June to August 2023 in an academic health system. Primary care physicians were randomized to an immediate activation group and a delayed activation group. Data were analyzed from August to November 2023.

Exposure

Access to GenAI-drafted replies for patient messages.

Main Outcomes and Measures

Time spent (1) reading messages, (2) replying to messages, (3) length of replies, and (4) physician likelihood to recommend GenAI drafts. The a priori hypothesis was that GenAI drafts would be associated with less physician time spent reading and replying to messages. A mixed-effects model was used.

Results

Fifty-two physicians participated in this QI study, with 25 randomized to the immediate activation group and 27 randomized to the delayed activation group. A contemporary control group included 70 physicians. There were 18 female participants (72.0%) in the immediate group and 17 female participants (63.0%) in the delayed group; the median age range was 35-44 years in the immediate group and 45-54 years in the delayed group. The median (IQR) time spent reading messages in the immediate group was 26 (11-69) seconds at baseline, 31 (15-70) seconds 3 weeks after entry to the intervention, and 31 (14-70) seconds 6 weeks after entry. The delayed group's median (IQR) read time was 25 (10-67) seconds at baseline, 29 (11-77) seconds during the 3-week waiting period, and 32 (15-72) seconds 3 weeks after entry to the intervention. The contemporary control group's median (IQR) read times were 21 (9-54), 22 (9-63), and 23 (9-60) seconds in corresponding periods. The estimated association of GenAI was a 21.8% increase in read time (95% CI, 5.2% to 41.0%; P = .008), a -5.9% change in reply time (95% CI, -16.6% to 6.2%; P = .33), and a 17.9% increase in reply length (95% CI, 10.1% to 26.2%; P < .001). Participants recognized GenAI's value and suggested areas for improvement.

Conclusions and Relevance

In this QI study, GenAI-drafted replies were associated with significantly increased read time, no change in reply time, significantly increased reply length, and some perceived benefits. Rigorous empirical tests are necessary to further examine GenAI's performance. Future studies should examine patient experience and compare multiple GenAIs, including those with medical training.

Charting Tomorrow's Healthcare: A Traditional Literature Review for an Artificial Intelligence-Driven Future

Electronic health record (EHR) systems have developed over time in parallel with general advancements in mainstream technology. As artificially intelligent (AI) systems rapidly impact multiple societal sectors, it has become apparent that medicine is not immune from the influences of this powerful technology. Particularly appealing is how AI may aid in improving healthcare efficiency with note-writing automation. This literature review explores the current state of EHR technologies in healthcare, specifically focusing on possibilities for addressing EHR challenges through the automation of dictation and note-writing processes with AI integration. This review offers a broad understanding of existing capabilities and potential advancements, emphasizing innovations such as voice-to-text dictation, wearable devices, and AI-assisted procedure note dictation. The primary objective is to provide researchers with valuable insights, enabling them to generate new technologies and advancements within the healthcare landscape. By exploring the benefits, challenges, and future of AI integration, this review encourages the development of innovative solutions, with the goal of enhancing patient care and healthcare delivery efficiency.

Adapted large language models can outperform medical experts in clinical text summarization

Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor-patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.

Quantifying EHR and Policy Factors Associated with the Gender Productivity Gap in Ambulatory, General Internal Medicine

BACKGROUND

The gender gap in physician compensation has persisted for decades. Little is known about how differences in use of the electronic health record (EHR) may contribute.

OBJECTIVE

To characterize how time on clinical activities, time on the EHR, and clinical productivity vary by physician gender and to identify factors associated with physician productivity.

DESIGN, SETTING, AND PARTICIPANTS

This longitudinal study included general internal medicine physicians employed by a large ambulatory practice network in the Northeastern United States from August 2018 to June 2021.

MAIN MEASURES

Monthly data on physician work relative value units (wRVUs), physician and practice characteristics, metrics of EHR use and note content, and temporal trend variables.

KEY RESULTS

The analysis included 3227 physician-months of data for 108 physicians (44% women). Compared with men physicians, women physicians generated 23.8% fewer wRVUs per month, completed 22.1% fewer visits per month, spent 4.0 more minutes/visit and 8.72 more minutes on the EHR per hour worked (all p < 0.001), and typed or dictated 36.4% more note characters per note (p = 0.006). With multivariable adjustment for physician age, practice characteristics, EHR use, and temporal trends, physician gender was no longer associated with productivity (men 4.20 vs. women 3.88 wRVUs/hour, p = 0.31). Typing/dictating fewer characters per note, relying on greater teamwork to manage orders, and spending less time on documentation were associated with higher wRVUs/hour. The 2021 E/M code change was associated with higher wRVUs/hour for all physicians: 10% higher for men physicians and 18% higher for women physicians (p < 0.001 and p = 0.009, respectively).

CONCLUSIONS

Increased team support, briefer documentation, and the 2021 E/M code change were associated with higher physician productivity. The E/M code change may have preferentially benefited women physicians by incentivizing time-intensive activities such as medical decision-making, preventive care discussion, and patient counseling that women physicians have historically spent more time performing.

Getting More Out of Clinical Documentation: Can Clinical Dashboards Yield Clinically Useful Information?

This study investigated coded data retrieved from clinical dashboards, which are decision-support tools that include a graphical display of clinical progress and clinical activities. Data were extracted from clinical dashboards representing 256 youth (M age = 11.9) from 128 practitioners who were trained in the Managing and Adapting Practice (MAP) system (Chorpita & Daleiden in BF Chorpita EL Daleiden 2014 Structuring the collaboration of science and service in pursuit of a shared vision. 43(2):323 338. 2014, Chorpita & Daleiden in BF Chorpita EL Daleiden 2018 Coordinated strategic action: Aspiring to wisdom in mental health service systems. 25(4):e12264. 2018) in 55 agencies across 5 regional mental health systems. Practitioners labeled up to 35 fields (i.e., descriptions of clinical activities), with the options of drawing from a controlled vocabulary or writing in a client-specific activity. Practitioners then noted when certain activities occurred during the episode of care. Fields from the extracted data were coded and reliability was assessed for Field Type, Practice Element Type, Target Area, and Audience (e.g., Caregiver Psychoeducation: Anxiety would be coded as Field Type = Practice Element; Practice Element Type = Psychoeducation; Target Area = Anxiety; Audience = Caregiver). Coders demonstrated moderate to almost perfect interrater reliability. On average, practitioners recorded two activities per session, and clients had 10 unique activities across all their sessions. Results from multilevel models showed that clinical activity characteristics and sessions accounted for the most variance in the occurrence, recurrence, and co-occurrence of clinical activities, with relatively less variance accounted for by practitioners, clients, and regional systems. Findings are consistent with patterns of practice reported in other studies and suggest that clinical dashboards may be a useful source of clinical information. More generally, the use of a controlled vocabulary for clinical activities appears to increase the retrievability and actionability of healthcare information and thus sets the stage for advancing the utility of clinical documentation.

Building Capacity for Pragmatic Trials of Digital Technology in Primary Care

Frontline primary care teams face important challenges in seeking to transform the quality of care delivered to patients and to reduce clerical burden for clinicians. Digital technologies using artificial intelligence hold substantial promise to aid in this transformation. Both pragmatic clinical trials and implementation science are key tools to successfully introduce, evaluate, and sustain innovations in real-world primary care practices. Previous articles in this thematic series have provided an in-depth overview of pragmatic trials and implementation science. This paper demonstrates and provides a framework for how these concepts, together with digital transformation, can be used to solve many of the challenges facing primary care. This framework is conceived as the collaboration of frontline primary care teams with innovators in academic institutions and industry through pragmatic trials and implementation science.

Explaining Variability in Electronic Health Record Effort in Primary Care Ambulatory Encounters

BACKGROUND

 Electronic health record (EHR) user interface event logs are fast providing another perspective on the value and efficiency EHR technology brings to health care. Analysis of these detailed usage data has demonstrated their potential to identify EHR and clinical process design factors related to user efficiency, satisfaction, and burnout.

OBJECTIVE

 This study aimed to analyze the event log data across 26 different health systems to determine the variability of use of a single vendor's EHR based on four event log metrics, at the individual, practice group, and health system levels.

METHODS

 We obtained de-identified event log data recorded from June 1, 2018, to May 31, 2019, from 26 health systems' primary care physicians. We estimated the variability in total Active EHR Time, Documentation Time, Chart Review Time, and Ordering Time across health systems, practice groups, and individual physicians.

RESULTS

 In total, 5,444 physicians (Family Medicine: 3,042 and Internal Medicine: 2,422) provided care in a total of 2,285 different practices nested in 26 health systems. Health systems explain 1.29, 3.55, 3.45, and 3.30% of the total variability in Active Time, Documentation Time, Chart Review Time, and Ordering Time, respectively. Practice-level variability was estimated to be 7.96, 13.52, 8.39, and 5.57%, respectively, and individual physicians explained the largest proportion of the variability for those same outcomes 17.09, 27.49, 17.51, and 19.75%, respectively.

CONCLUSION

 The most variable physician EHR usage patterns occurs at the individual physician level and decreases as you move up to the practice and health system levels. This suggests that interventions to improve individual users' EHR usage efficiency may have the most potential impact compared with those directed at health system or practice levels.

Quantifying the impact of telemedicine and patient medical advice request messages on physicians' work-outside-work

The COVID-19 pandemic has boosted digital health utilization, raising concerns about increased physicians' after-hours clinical work ("work-outside-work"). The surge in patients' digital messages and additional time spent on work-outside-work by telemedicine providers underscores the need to evaluate the connection between digital health utilization and physicians' after-hours commitments. We examined the impact on physicians' workload from two types of digital demands - patients' messages requesting medical advice (PMARs) sent to physicians' inbox (inbasket), and telemedicine. Our study included 1716 ambulatory-care physicians in New York City regularly practicing between November 2022 and March 2023. Regression analyses assessed primary and interaction effects of (PMARs) and telemedicine on work-outside-work. The study revealed a significant effect of PMARs on physicians' work-outside-work and that this relationship is moderated by physicians' specialties. Non-primary care physicians or specialists experienced a more pronounced effect than their primary care peers. Analysis of their telemedicine load revealed that primary care physicians received fewer PMARs and spent less time in work-outside-work with more telemedicine. Specialists faced increased PMARs and did more work-outside-work as telemedicine visits increased which could be due to the difference in patient panels. Reducing PMAR volumes and efficient inbasket management strategies needed to reduce physicians' work-outside-work. Policymakers need to be cognizant of potential disruptions in physicians carefully balanced workload caused by the digital health services.

Physicians' attitudes towards secondary use of clinical data for biomedical research purposes in Germany. Results of a quantitative survey

BACKGROUND

For biomedical data-driven research purposes, secondary use of clinical data carries great but largely untapped potential. Physicians' attitudes and their needs towards secondary data use are essential to inform its practical and ethically sound implementation but are currently understudied.

OBJECTIVE

Therefore, the objectives of the study are to assess physicians' (i) general attitudes and concerns, (ii) willingness to adapt workflows and to make data available for secondary use, (iii) group-specific conditions toward implementation of secondary use and associated concerns of physician-scientists and purely clinical physicians.

METHODS

We developed an online survey based on a literature review and an expert interview study. Physicians in private practice and at two large German university hospitals were surveyed from May 2021 until January 2022.

RESULTS

In total, 446 physicians participated in the survey. 96% [380/397] of all physicians reported a positive attitude towards secondary use; 87% [31/397] are in-principle willing to support secondary use of clinical data along with a small proportion of physicians with fundamental reservations. Secondly, the most important conditions for adapting workflows were funding of additional time and effort for research-adequate documentation (71% [286/390]) and the most important condition for providing patients' clinical data was reliable protection of patients' privacy (67% [254/382]). Thirdly, physician-scientists were more likely than purely clinical physicians to request additional funding for research-adequate documentation as a precondition for support (83% vs 69%, P = .002) and the privilege to conduct research with their own patients' clinical data before other researchers are allowed to (43% vs 11%, P < .001); while purely clinical physicians more frequently require reliable protection of patient privacy (76% vs 62%, P = .007) and monetary compensation (45% vs 25%, P < .001).

CONCLUSION

Since this study presents high in-principle willingness of physicians to support secondary use along with little general concerns, it seems essential to address physicians' group-specific conditions toward secondary use in order to gain their support.

We Shape Our Tools and Then Our Tools Shape Us: OPAT and the EHR

A commentary on Canterino et al. (2024) and Munsiff et al. (2024), articles where clinicians from two large OPAT programs.

The role of organizations in shaping physician use of electronic health records

OBJECTIVE

The aim of the study was to (1) characterize organizational differences in primary care physicians' electronic health record (EHR) behavior; (2) assess within-organization consistency in EHR behaviors; and (3) identify whether organizational consistency is associated with physician-level efficiency.

DATA SOURCES

EHR metadata capturing averaged weekly measures of EHR time and documentation composition from 75,124 US primary care physicians across 299 organizations between September 2020 and May 2021 were taken. EHR time measures include active time in orders, chart review, notes, messaging, time spent outside of scheduled hours, and total EHR time. Documentation composition measures include note length and percentage use of templated text or copy/paste. Efficiency is measured as the percent of visits with same-day note completion.

STUDY DESIGN

All analyses are cross-sectional. Across-organization differences in EHR use and documentation composition are presented via 90th-to-10th percentile ratios of means and SDs. Multilevel modeling with post-estimation variance partitioning assesses the extent of an organizational signature-the proportion of variation in our measures attributable to organizations (versus specialty and individual behaviors). We measured organizational internal consistency for each measure via organization-level SD, which we grouped into quartiles for regression. Association between internally consistent (i.e., low SD) organizational EHR use and physician-level efficiency was assessed with multi-variable OLS models.

DATA COLLECTION

Extraction from Epic's Signal platform used for measuring provider EHR efficiency.

PRINCIPAL FINDINGS

EHR time per visit for physicians at a 90th percentile organization is 1.94 times the average EHR time at a 10th percentile organization. There is little evidence, on average, of an organizational signature. However, physicians in organizations with high internal consistency in EHR use demonstrate increased efficiency. Physicians in organizations with the highest internal consistency (top quartile) have a 3.77 percentage point higher same-day visit closure rates compared with peers in bottom quartile organizations (95% confidence interval: 0.0142-0.0612).

CONCLUSIONS

Results suggest unrealized opportunities for organizations and policymakers to support consistency in how physicians engage in EHR-supported work.

Attention-Based Deep Learning Model for Prediction of Major Adverse Cardiovascular Events in Peritoneal Dialysis Patients

Major adverse cardiovascular events (MACE) encompass pivotal cardiovascular outcomes such as myocardial infarction, unstable angina, and cardiovascular-related mortality. Patients undergoing peritoneal dialysis (PD) exhibit specific cardiovascular risk factors during the treatment, which can escalate the likelihood of cardiovascular events. Hence, the prediction and key factor analysis of MACE have assumed paramount significance for peritoneal dialysis patients. Current pathological methodologies for prognosis prediction are not only costly but also cumbersome in effectively processing electronic health records (EHRs) data with high dimensionality, heterogeneity, and time series. Therefore in this study, we propose the CVEformer, an attention-based neural network designed to predict MACE and analyze risk factors. CVEformer leverages the self-attention mechanism to capture temporal correlations among time series variables, allowing for weighted integration of variables and estimation of the probability of MACE. CVEformer first captures the correlations among heterogeneous variables through attention scores. Then, it analyzes the correlations within the time series data to identify key risk variables and predict the probability of MACE. When trained and evaluated on data from a large cohort of peritoneal dialysis patients across multiple centers, CVEformer outperforms existing models in terms of predictive performance.

Navigating the doctor-patient-AI relationship - a mixed-methods study of physician attitudes toward artificial intelligence in primary care

BACKGROUND

Artificial intelligence (AI) is a rapidly advancing field that is beginning to enter the practice of medicine. Primary care is a cornerstone of medicine and deals with challenges such as physician shortage and burnout which impact patient care. AI and its application via digital health is increasingly presented as a possible solution. However, there is a scarcity of research focusing on primary care physician (PCP) attitudes toward AI. This study examines PCP views on AI in primary care. We explore its potential impact on topics pertinent to primary care such as the doctor-patient relationship and clinical workflow. By doing so, we aim to inform primary care stakeholders to encourage successful, equitable uptake of future AI tools. Our study is the first to our knowledge to explore PCP attitudes using specific primary care AI use cases rather than discussing AI in medicine in general terms.

METHODS

From June to August 2023, we conducted a survey among 47 primary care physicians affiliated with a large academic health system in Southern California. The survey quantified attitudes toward AI in general as well as concerning two specific AI use cases. Additionally, we conducted interviews with 15 survey respondents.

RESULTS

Our findings suggest that PCPs have largely positive views of AI. However, attitudes often hinged on the context of adoption. While some concerns reported by PCPs regarding AI in primary care focused on technology (accuracy, safety, bias), many focused on people-and-process factors (workflow, equity, reimbursement, doctor-patient relationship).

CONCLUSION

Our study offers nuanced insights into PCP attitudes towards AI in primary care and highlights the need for primary care stakeholder alignment on key issues raised by PCPs. AI initiatives that fail to address both the technological and people-and-process concerns raised by PCPs may struggle to make an impact.

Creating Conversion Factors from EHR Event Log Data: A Comparison of Investigator-Derived and Vendor-Derived Metrics for Primary Care Physicians

Physicians spend a large amount of time with the electronic health record (EHR), which the majority believe contributes to their burnout. However, there are limitedstandardized measures of physician EHR time. Vendor-derived metrics are standardized but may underestimate real-world EHR experience. Investigator-derived metrics may be more reliable but not standardized, particularly with regard to timeout thresholds defining inactivity. This study aimed to enable standardized investigator-derived metrics using conversion factors between raw event log-derived metrics and Signal (Epic System's standardized metric) for primary care physicians. This was an observational, retrospective longitudinal study of EHR raw event logs and Signal data from a quaternary academic medical center and its community affiliates in California, over a 6-month period. The study evaluated 242 physicians over 1370 physician-months, comparing 53.7 million event logs to 6850 Signal metrics, in five different time based metrics. Results show that inactivity thresholds for event log metric derivation that most closely approximate Signal metrics ranged from 90 seconds (Visit Navigator) to 360 seconds ("Pajama time") depending on the metric. Based on this data, conversion factors for investigator-derived metrics across a wide range of inactivity thresholds, via comparison with Signal metrics, are provided which may allow researchers to consistently quantify EHR experience.

A Practical Approach to Optimize Computerized Provider Order Entry Systems and Reduce Clinician Burden: Pre-Post Evaluation of Vendor-Derived "Order Friction" Data

Computerized provider order entry (CPOE) systems have been cited as a significant contributor to clinician burden. Vendor-derived measures and data sets have been developed to help with optimization of CPOE systems. We describe how we analyzed vendor-derived Order Friction (OF) EHR log data at our health system and propose a practical approach for optimizing CPOE systems by reducing OF. We also conducted a pre-post intervention study using OF data to evaluate the impact of defaulting the frequency of urine, stool and nasal swab tests and found that all modified orders had significantly fewer changes required per order (p<0.01). Our proposed approach is a six-step process: 1) understand the ordering process, 2) understand OF data elements contextually, 3) explore ordering user-level factors, 4) evaluate order volume and friction from different order sources, 5) optimize order-level design, 6) identify high volume alerts to evaluate for appropriateness.

Physician Burnout: Evidence-Based Roadmaps to Prioritizing and Supporting Personal Wellbeing

Current literature validates the magnitude of physician burnout as a complex challenge affecting physicians, patients, and healthcare delivery that mandates science-informed intervention. Physician burnout affects both individual practitioners and patient care delivery. Interventions, defined as roadmaps, to prioritizing and supporting personal wellbeing encompass organizational, individual, and moral injury, with virtually no consensus on optimal approaches. The purpose of this conceptual review is to present evidence-based innovative insights on contributing factors, mitigation, and designing adaptive systems to combat and prevent burnout. Science-informed policy initiatives that support long-term organizational changes endorsed by both leadership and institutional stakeholders are keys to sustaining personal wellbeing and ending burnout.

Perspectives on the representation of frailty in the electronic frailty index

BACKGROUND

Frailty is a state of increased vulnerability from physical, social, and cognitive factors resulting in greater risk of negative health-related outcomes and increased healthcare expenditure. A 36-factor electronic frailty index (eFI) developed in the United Kingdom calculates frailty scores using electronic medical record data. There is currently no standardization of frailty screening in Canadian primary care. In order to implement the eFI in a Canadian context, adaptation of the tool is necessary because frailty is represented by different clinical terminologies in the UK and Canada. In considering the promise of implementing an eFI in British Columbia, Canada, we first looked at the content validation of the 36-factor eFI. Our research question was: Does the eFI represent frailty from the perspectives of primary care clinicians and older adults in British Columbia?

METHODS

A modified Delphi using three rounds of questionnaires with a panel of 23 experts (five family physicians, five nurse practitioners, five nurses, four allied health professionals, four older adults) reviewed and provided feedback on the 36-factor eFI. These professional groups were chosen because they closely work as interprofessional teams within primary care settings with older adults. Older adults provide real life context and experiences. Questionnaires involved rating the importance of each frailty factor on a 0-10 scale and providing rationale for ratings. Panelists were also given the opportunity to suggest additional factors that ought to be included in the screening tool. Suggested factors were similarly rated in two Delphi rounds.

RESULTS

Thirty-three of the 36 eFI factors achieved consensus (> 80% of panelists provided a rating of ≥ 8). Factors that did not achieve consensus were hypertension, thyroid disorder and peptic ulcer. These factors were perceived as easily treatable or manageable and/or not considered reflective of frailty on their own. Additional factors suggested by panelists that achieved consensus included: cancer, challenges to healthcare access, chronic pain, communication challenges, fecal incontinence, food insecurity, liver failure/cirrhosis, mental health challenges, medication noncompliance, poverty/financial difficulties, race/ethnic disparity, sedentary/low activity levels, and substance use/misuse. There was a 100% retention rate in each of the three Delphi rounds.

CONCLUSIONS AND NEXT STEPS

Three key findings emerged from this study: the conceptualization of frailty varied across participants, identification of frailty in community/primary care remains challenging, and social determinants of health affect clinicians' assessments and perceptions of frailty status. This study will inform the next phase of a broader mixed-method sequential study to build a frailty screening tool that could ultimately become a standard of practice for frailty screening in Canadian primary care. Early detection of frailty can help tailor decision making, frame discussions about goals of care, prevent advancement on the frailty trajectory, and ultimately decrease health expenditures, leading to improved patient and system level outcomes.

mTBI evaluation, management, and referral to allied healthcare: practices of first-line healthcare professionals

PRIMARY OBJECTIVE

To gain an understanding of current evaluation practices, post-injury recommendations, and referrals to allied healthcare professions (AHP) by first-line healthcare professionals (FHPs) providing care for people with mild traumatic brain injury (mTBI).

RESEARCH DESIGN

Survey study.

METHODS AND PROCEDURES

Physicians, physician assistants, nurse practitioners, nurses, and athletic trainers (n = 126) completed an online survey, including Likert scale and free response question relating to mTBI evaluation, management, and referral practices.

MAIN OUTCOMES AND RESULTS

FHPs surveyed reported being confident in their ability to evaluate patients with suspected mTBI, relying most heavily on patient-reported symptoms and physical signs as methods of evaluation. Most FHPs reported making recommendations to compensate for the symptoms experienced following mTBI diagnosis. In contrast, FHPs expressed challenges in the evaluation and management of symptoms associated with mTBI along with limited knowledge of and referrals to AHPs.

CONCLUSIONS

Overall, FHPs feel confident in the diagnosis of mTBI but experience assessment and management challenges. AHPs are underutilized on mTBI management teams calling for a need for multidisciplinary collaboration on research, education, and rehabilitation efforts to optimally care for people experiencing mTBI symptoms.

More Tethered to the EHR: EHR Workload Trends Among Academic Primary Care Physicians, 2019-2023

PURPOSE

The purpose of this study is to evaluate recent trends in primary care physician (PCP) electronic health record (EHR) workload.

METHODS

This longitudinal study observed the EHR use of 141 academic PCPs over 4 years (May 2019 to March 2023). Ambulatory full-time equivalency (aFTE), visit volume, and panel size were evaluated. Electronic health record time and inbox message volume were measured per 8 hours of scheduled clinic appointments.

RESULTS

From the pre-COVID-19 pandemic year (May 2019 to February 2020) to the most recent study year (April 2022 to March 2023), the average time PCPs spent in the EHR per 8 hours of scheduled clinic appointments increased (+28.4 minutes, 7.8%), as did time in orders (+23.1 minutes, 58.9%), inbox (+14.0 minutes, 24.4%), chart review (+7.2 minutes, 13.0%), notes (+2.9 minutes, 2.3%), outside scheduled hours on days with scheduled appointments (+6.4 minutes, 8.2%), and on unscheduled days (+13.6 minutes, 19.9%). Primary care physicians received more patient medical advice requests (+5.4 messages, 55.5%) and prescription messages (+2.3, 19.5%) per 8 hours of scheduled clinic appointments, but fewer patient calls (-2.8, -10.5%) and results messages (-0.3, -2.7%). While total time in the EHR continued to increase in the final study year (+7.7 minutes, 2.0%), inbox time decreased slightly from the year prior (-2.2 minutes, -3.0%). Primary care physicians' average aFTE decreased 5.2% from 0.66 to 0.63 over 4 years.

CONCLUSIONS

Primary care physicians' time in the EHR continues to grow. While PCPs' inbox time may be stabilizing, it is still substantially higher than pre-pandemic levels. It is imperative health systems develop strategies to change the EHR workload trajectory to minimize PCPs' occupational stress and mitigate unnecessary reductions in effective physician workforce resulting from the increased EHR burden.