Implementation Science Workshop: Implementation of an Electronic Referral System in a Large Academic Medical Center

Impact of an Opt-In eConsult Program on Primary Care Demand for Specialty Visits: Stepped-Wedge Cluster Randomized Implementation Study

BACKGROUND

eConsult programs have been instituted to increase access to specialty expertise. Opt-in choice eConsult programs maintain primary care physician (PCP) autonomy to decide whether to utilize eConsults versus traditional specialty referrals, but little is known about how this intervention may impact PCP eConsult adoption and traditional referral demand.

OBJECTIVE

We assessed the feasibility of implementing an opt-in choice eConsult program and examined whether this intervention reduces demand for in-person visits for primary care patients requiring specialty expertise.

DESIGN

Stepped-wedge, cluster randomized trial conducted from July 2018 to June 2019.

PARTICIPANTS

Sixteen primary care practices in a large, urban academic health care system.

INTERVENTION

Our intervention was an opt-in choice eConsult available in addition to traditional specialty referral; our implementation strategy included in-person training, audit and feedback, and incentive payments.

MAIN MEASURES

Our implementation outcome measure was the eConsult rate: weekly proportion of eConsults per PCP visit at each site. Our intervention outcome measure was traditional referral rate: weekly proportion of referrals per PCP visit at each site. We also assessed PCP experiences with questionnaires.

KEY RESULTS

Of 305,915 in-person PCP visits, there were 31,510 traditional referrals to specialties participating in the eConsult program, and 679 eConsults. All but one primary care site utilized the opt-in choice eConsult program, with a weekly rate of 0.05 eConsults per 100 PCP visits by the end of the study period. The weekly rate of traditional referrals was 11 per 100 PCP visits at the end of the study period; this represents a significant increase in traditional referral rate after implementation of eConsults. PCPs were generally satisfied with the eConsult program and valued prompt provider-to-provider communication.

CONCLUSIONS

Implementation of an opt-in choice eConsult program resulted in widespread PCP adoption; however, this did not decrease the demand for traditional referrals. Future studies should evaluate different strategies to incentivize and increase eConsult utilization while maintaining PCP choice.

A Sociotechnical Framework for Safety-Related Electronic Health Record Research Reporting: The SAFER Reporting Framework

Electronic health record (EHR)-based interventions to improve patient safety are complex and sensitive to who, what, where, why, when, and how they are delivered. Success or failure depends not only on the characteristics and behaviors of individuals who are targeted by an intervention, but also on the technical characteristics of the intervention and the culture and environment of the health system that implements it. Current reporting guidelines do not capture the complexity of sociotechnical factors (technical and nontechnical factors, such as workflow and organizational issues) that confound or influence these interventions. This article proposes a methodological reporting framework for EHR interventions targeting patient safety and builds on an 8-dimension sociotechnical model previously developed by the authors for design, development, implementation, use, and evaluation of health information technology. The Safety-related EHR Research (SAFER) Reporting Framework enables reporting of patient safety-focused EHR-based interventions while accounting for the multifaceted, dynamic sociotechnical context affecting intervention implementation, effectiveness, and generalizability. As an example, an EHR-based intervention to improve communication and timely follow-up of subcritical abnormal test results to operationalize the framework is presented. For each dimension, reporting should include what sociotechnical changes were made to implement an EHR-related intervention to improve patient safety, why the intervention did or did not lead to safety improvements, and how this intervention can be applied or exported to other health care organizations. A foundational list of research and reporting recommendations to address implementation, effectiveness, and generalizability of EHR-based interventions needed to effectively reduce preventable patient harm is provided. The SAFER Reporting Framework is not meant to replace previous research reporting guidelines, but rather provides a sociotechnical adjunct that complements their use.

Assessment of Health Information Technology-Related Outpatient Diagnostic Delays in the US Veterans Affairs Health Care System: A Qualitative Study of Aggregated Root Cause Analysis Data

IMPORTANCE

Diagnostic delay in the outpatient setting is an emerging safety priority that health information technology (HIT) should help address. However, diagnostic delays have persisted, and new safety concerns associated with the use of HIT have emerged.

OBJECTIVE

To analyze HIT-related outpatient diagnostic delays within a large, integrated health care system.

DESIGN, SETTING, AND PARTICIPANTS

This cohort study involved qualitative content analysis of safety concerns identified in aggregated root cause analysis (RCA) data related to HIT and outpatient diagnostic delays. The setting was the US Department of Veterans Affairs using all RCAs submitted to the Veterans Affairs (VA) National Center for Patient Safety from January 1, 2013, to July 31, 2018.

MAIN OUTCOMES AND MEASURES

Common themes associated with the role of HIT-related safety concerns were identified and categorized according to the Health IT Safety framework for measuring, monitoring, and improving HIT safety. This framework includes 3 related domains (ie, safe HIT, safe use of HIT, and using HIT to improve safety) situated within an 8-dimensional sociotechnical model accounting for interacting technical and nontechnical variables associated with safety. Hence, themes identified enhanced understanding of the sociotechnical context and domain of HIT safety involved.

RESULTS

Of 214 RCAs categorized by the terms delay and outpatient submitted during the study period, 88 were identified as involving diagnostic delays and HIT, from which 172 unique HIT-related safety concerns were extracted (mean [SD], 1.97 [1.53] per RCA). Most safety concerns (82.6% [142 of 172]) involved problems with safe use of HIT, predominantly sociotechnical factors associated with people, workflow and communication, and a poorly designed human-computer interface. Fewer safety concerns involved problems with safe HIT (14.5% [25 of 172]) or using HIT to improve safety (0.3% [5 of 172]). The following 5 key high-risk areas for diagnostic delays emerged: managing electronic health record inbox notifications and communication, clinicians gathering key diagnostic information, technical problems, data entry problems, and failure of a system to track test results.

CONCLUSIONS AND RELEVANCE

This qualitative study of a national RCA data set suggests that interventions to reduce outpatient diagnostic delays could aim to improve test result management, interoperability, data visualization, and order entry, as well as to decrease information overload.

Connected Subspecialty Care: Applying Telehealth Strategies to Specific Referral Barriers

Nearly a quarter of families of children with need of subspecialty care report difficulty accessing that care. Telehealth is a method to overcome barriers to subspecialty care. However, improving access to subspecialty care through telehealth requires granular identification of specific subspecialty barriers and recognition of the strengths and limitations of each telehealth strategy for addressing identified barriers. Focusing on each sequential step in subspecialty referrals and potential associated barriers, we summarize specific telehealth and technology-enabled strategies to improve access to subspecialty care, including electronic consultations, live interactive telemedicine, store-and-forward telemedicine, tele-mentoring, patient portals, and remote patient monitoring. Intentionally selecting telehealth strategies to target specific subspecialty referral barriers may avoid risks from misapplication of telehealth, may more clearly elevate equitable access as an essential goal within telehealth initiatives, and may also lead to synergistic use of strategies that overcome sequential barriers.

Primary Care Practitioners' Perceptions of Electronic Consult Systems: A Qualitative Analysis

IMPORTANCE

Safety-net health systems across the country are implementing electronic consult (eConsult) systems in which primary care practitioners (PCPs) submit all requests for specialty assistance electronically to be reviewed and discussed with specialists. Evidence suggests that eConsult systems can make significant improvements in specialty access, but the outcomes of these systems for frontline PCPs is poorly understood.

OBJECTIVE

To understand PCP perceptions of the results of eConsult initiation on PCP workflow, specialist access, and patient care.

DESIGN, SETTING, AND PARTICIPANTS

Qualitative interviews were conducted from December 1, 2016, to April 15, 2017, with 40 safety-net PCPs in Los Angeles County who use the Los Angeles County Department of Health Services (DHS) eConsult system. Interviewees were recruited to include diversity in PCP type, practice setting, and employer (DHS employed vs DHS affiliated). Participants were interviewed about their perceptions of clinical workflow, access to specialists, relationships with specialists, and referral decision making.

MAIN OUTCOMES AND MEASURES

Perceptions of the results of eConsult, including positive and negative themes and remaining perceived gaps in specialty care.

RESULTS

Of the 40 participants, 27 (68%) were women; 24 (60%) PCPs performed 5 or more eConsults per week. Primary care practitioners' perceptions of eConsult clustered around 4 main themes: access and timeliness of specialty care, shift of work to PCPs, relationships with specialists, and eConsult interface issues. Many PCPs praised the improved timeliness of specialist input with eConsult, as well as the added clinical and educational value of dialogue with specialists, particularly compared with the limitations of the prior referral process. However, PCPs also consistently perceived that eConsult shifted some of the work of specialty care to them. Many PCPs believed that this extra burden was worth the effort given the benefits of eConsult, such as improved timeliness of care and ability to manage specialty conditions. In contrast, others were frustrated by the increased administrative burden, broadened clinical responsibility, and restructuring of specialty care delivery.

CONCLUSIONS AND RELEVANCE

While associated with improved specialty care access, eConsult systems simultaneously created new challenges for PCPs, such as an increased burden of work in providing specialty care. Primary care practitioners varied in their enthusiasm for these workflow changes with diverging perceptions of the same processes. Our findings provide insights on challenges future primary care transformation efforts may face.

Impact of Implementation of Electronically Transmitted Referrals on Pediatric Subspecialty Visit Attendance

OBJECTIVE

One barrier to timely access to outpatient pediatric subspecialty care is the complexity of scheduling processes. We evaluated the impact of implementing electronically transmitted referrals on subspecialty visit attendance.

METHODS

Through collaboration with stakeholders, an electronically transmitted referral order system was designed, piloted, and implemented in 15 general pediatrics practices, with 24 additional practices serving as controls. We used statistical process control methods and difference-in-differences analysis to examine visits attended, appointments scheduled, appointment nonattendance, and referral volume. Electronically transmitted referrals then were expanded to all 39 practices. We surveyed referring pediatricians at all practices before and after implementation.

RESULTS

From April 2015 through September 2016 there were 33,485 referral orders across all practices (7770 before the pilot, 11,776 during the pilot, 13,939 after full implementation). At pilot practices, there was a significant and sustained improvement in subspecialty visits attended within 4 weeks of referral (10.9% to 20.0%; P < .001). Relative to control practices, pilot practices experienced an 8.6% improvement (P = .001). After implementation at control practices, rates of visits attended also improved but to a smaller degree: 11.8% to 14.7% (P < .001). In survey responses, referring pediatricians noted improved scheduling processes but had continued concerns with appointment availability and referral tracking.

CONCLUSIONS

While electronically transmitted referrals improved visit attendance after pediatric subspecialty referral, the sizable percentage of children without attended visits, the muted effect at control practices, and pediatrician survey responses indicate that additional work is needed to address barriers to pediatric subspecialty care.

Keeping care connected: e-Consultation program improves access to nephrology care

Introduction

Health systems are seeking innovative solutions to improve specialty care access. Electronic consultations (eConsults) allow specialists to provide formal clinical recommendations to primary care providers (PCPs) based on patient chart review, without a face-to-face visit.

Methods

We implemented a nephrology eConsult pilot program within a large, academic primary care practice to facilitate timely communication between nephrologists and PCPs. We used primary care referral data to compare wait times and completion rates between traditional referrals and eConsults. We surveyed PCPs to assess satisfaction with the program.

Results

For traditional nephrology referrals placed during the study period (July 2016–March 2017), there was a 51-day median appointment wait time and a 40.9% referral completion rate. For eConsults, there was a median nephrologist response time of one day and a 100% completion rate; 67.5% of eConsults did not require a subsequent face-to-face specialty appointment. For eConsults that were converted to an in-person visit, the median wait time and completion rate were 40 days and 73.1%, respectively. Compared to traditional referrals placed during the study period, eConsults converted to in-person visits were more likely to be completed ( p = 0.001). Survey responses revealed that PCPs were highly satisfied with the program and consider the quick turnaround time as the greatest benefit.

Discussion

Our eConsult pilot program reduced nephrology wait times and significantly increased referral completion rates. In large integrated health systems, eConsults have considerable potential to improve access to specialty care, reduce unnecessary appointments, and optimize the patient population being seen by specialists.