Innovative Implementation Studies Conducted in US Safety Net Health Care Settings: A Systematic Review

Application of the FRAME-IS to a multifaceted implementation strategy

BACKGROUND

Research demonstrates the importance of documenting adaptations to implementation strategies that support integration of evidence-based interventions into practice. While studies have utilized the FRAME-IS [Framework for Reporting Adaptations and Modifications for Implementation Strategies] to collect structured adaptation data, they are limited by a focus on discrete implementation strategies (e.g., training), which do not reflect the complexity of multifaceted strategies like practice facilitation. In this paper, we apply the FRAME-IS to our trial evaluating the effectiveness of PF on implementation fidelity of an evidence-based technology-facilitated team care model for improved hypertension control within a federally qualified health center (FQHC).

METHODS

Three data sources are used to document adaptations: (1) implementation committee meeting minutes, (2) narrative reports completed by practice facilitators, and (3) structured notes captured on root cause analysis and Plan-Do-Study-Act worksheets. Text was extracted from the data sources according to the FRAME-IS modules and inputted into a master matrix for content analysis by two authors; a third author conducted member checking and code validation.

RESULTS

We modified the FRAME-IS to include part 2 of module 2 (what is modified) to add greater detail of the modified strategy, and a numbering system to track adaptations across the modules. This resulted in identification of 27 adaptations, of which 88.9% focused on supporting practices in identifying eligible patients and referring them to the intervention. About half (52.9%) of the adaptations were made to modify the context of the PF strategy to include a group-based format, add community health workers to the strategy, and to shift the implementation target to nurses. The adaptations were often widespread (83.9%), affecting all practices within the FQHC. While most adaptations were reactive (84.6%), they resulted from a systematic process of reviewing data captured by multiple sources. All adaptations included the FQHC in the decision-making process.

CONCLUSION

With modifications, we demonstrate the ability to document our adaptation data across the FRAME-IS modules, attesting to its applicability and value for a range of implementation strategies. Based on our experiences, we recommend refinement of tracking systems to support more nimble and practical documentation of iterative, ongoing, and multifaceted adaptations.

TRIAL REGISTRATION

Clinicaltrials.gov NCT03713515, Registration date: October 19, 2018.

Application of the FRAME-IS to a Multifaceted Implementation Strategy

Background

Research demonstrates the importance of documenting adaptations to implementation strategies that support integration of evidence-based interventions into practice. While studies have utilized the FRAME-IS [Framework for Reporting Adaptations and Modifications for Implementation Strategies] to collect structured adaptation data, they are limited by a focus on discrete implementation strategies (e.g., training), which do not reflect the complexity of multifaceted strategies like practice facilitation (PF). In this paper, we apply the FRAME-IS to our trial evaluating the effectiveness of PF on implementation fidelity of an evidence-based technology-facilitated team care model for improved hypertension control within a federally qualified health center (FQHC).

Methods

Three data sources are used to document adaptations: (1) implementation committee meeting minutes, (2) narrative reports completed by practice facilitators, and (3) structured notes captured on root cause analysis and Plan-Do-Study-Act worksheets. Text was extracted from the data sources according to the FRAME-IS modules and inputted into a master matrix for content analysis by two authors; a third author conducted member checking and code validation.

Results

We modified the FRAME-IS to include part 2 of module 2 (what is modified) to add greater detail of the modified strategy, and a numbering system to track adaptations across the modules. This resulted in identification of 27 adaptations, of which 88.9% focused on supporting practices in identifying eligible patients and referring them to the intervention. About half (52.9%) of the adaptations were made to modify the context of the PF strategy to include a group-based format, add community health workers to the strategy, and to shift the implementation target to nurses. The adaptations were often widespread (83.9%), affecting all practices within the FQHC. While most adaptations were reactive (84.6%), they resulted from a systematic process of reviewing data captured by multiple sources. All adaptations included the FQHC in the decision-making process.

Conclusion

With modifications, we demonstrate the ability to document our adaptation data across the FRAME-IS modules, attesting to its applicability and value for a range of implementation strategies. Based on our experiences, we recommend refinement of tracking systems to support more nimble and practical documentation of iterative, ongoing, and multifaceted adaptations.

Trial Registration

clinicaltrials.gov NCT03713515, Registration date: October 19, 2018.

Telehealth and Health Equity in Older Adults With Heart Failure: A Scientific Statement From the American Heart Association

Enhancing access to care using telehealth is a priority for improving outcomes among older adults with heart failure, increasing quality of care, and decreasing costs. Telehealth has the potential to increase access to care for patients who live in underresourced geographic regions, have physical disabilities or poor access to transportation, and may not otherwise have access to cardiologists with expertise in heart failure. During the COVID-19 pandemic, access to telehealth expanded, and yet barriers to access, including broadband inequality, low digital literacy, and structural barriers, prevented many of the disadvantaged patients from getting equitable access. Using a health equity lens, this scientific statement reviews the literature on telehealth for older adults with heart failure; provides an overview of structural, organizational, and personal barriers to telehealth; and presents novel interventions that pair telemedicine with in-person services to mitigate existing barriers and structural inequities.

Reducing low-value ED coags across 11 hospitals in a safety net setting

BACKGROUND

Prothrombin/international normalized ratio and activated partial thromboplastin time (PT/INR and aPTT) are frequently ordered in emergency departments (EDs), but rarely affect management. They offer limited utility outside of select indications. Several quality improvement initiatives have shown reduction in ED use of PT/INR and aPTT using multifaceted interventions in well-resourced settings. Successful reduction of these low-value tests has not yet been shown using a single intervention across a large hospital system in a safety net setting. This study aims to determine if an intervention of two BPAs is associated with a reduction in PT/INR and aPTT usage across a large safety net system.

METHODS

This initiative was set at a large safety net system in the United States with 11 acute care hospitals. Two Best Practice Advisories (BPAs) discouraging inappropriate PT/INR and aPTT use were implemented from March 16, 2022-August 30, 2022. Order rate per 100 ED patients during the pre-intervention period was compared to the post-intervention period on both the system and individual hospital level. Complete blood count (CBC) testing served as a control, and packed red blood cell transfusions served as a balancing measure. An interrupted time series regression analysis was performed to capture immediate and temporal changes in ordering for all tests in the pre and post-intervention periods.

RESULTS

PT/INR tests exhibited an absolute decline of 4.11 tests per 100 ED encounters (95% confidence interval -5.17 to -3.05; relative reduction of 18.9%). aPTT tests exhibited absolute decline of 4.03 tests per 100 ED encounters (95% CI -5.10 to -2.97; relative reduction of 19.8%). The control measure, CBC, did not significantly change (-0.43, 95% CI -2.83 to 1.96). Individual hospitals showed variable response, with absolute reductions from 2.02 to 9.6 tests per 100 ED encounters for PT/INR (relative reduction 12.1%-30.5%) and 2.07 to 10.04 for aPTT (relative reduction 12.1%-31.4%). Regression analysis showed that the intervention caused an immediate 25.7% decline in PT/INR and 24.7% decline in aPTT tests compared to the control measure. The slope differences (rate of order increase pre vs post intervention) did not significantly decline compared to the control.

CONCLUSIONS

This BPA intervention reduced PT/INR and aPTT use across 11 EDs in a large, urban, safety net system. Further study is needed in implementation to other non-safety net settings.

Choosing Wisely and Promoting High-Value Care and Staff Safety During the COVID-19 Pandemic in a Large Safety Net System

BACKGROUND AND OBJECTIVES

As the COVID-19 pandemic brought surges of hospitalized patients, it was important to focus on reducing overuse of tests and procedures to not only reduce potential harm to patients but also reduce unnecessary exposure to staff. The objective of this study was to create a Choosing Wisely in COVID-19 list to guide clinicians in practicing high-value care at our health system.

METHODS

A Choosing Wisely in COVID-19 list was developed in October 2020 by an interdisciplinary High Value Care Council at New York City Health + Hospitals, the largest public health system in the United States. The first phase involved gathering areas of overuse from interdisciplinary staff across the system. The second phase used a modified Delphi scoring process asking participants to rate recommendations on a 5-point Likert scale based on criteria of degree of evidence, potential to prevent patient harm, and potential to prevent staff harm.

RESULTS

The top 5 recommendations included avoiding tracheal intubation without trial of noninvasive ventilation (4.4); not placing routine central venous catheters (4.33); avoiding routine daily laboratory tests and batching laboratory draws (4.19); not ordering daily chest radiographs (4.17); and not using bronchodilators in the absence of reactive airway disease (4.13).

CONCLUSION

We successfully developed Choosing Wisely in COVID-19 recommendations that focus on evidence and preventing patient and staff harm in a large safety net system to reduce overuse.

Diarrhea don'ts: Reducing inappropriate stool cultures and ova and parasite testing for nosocomial diarrhea

BACKGROUND

Diarrhea that develops in patients after 72 hours of hospitalization is likely to have a nosocomial or iatrogenic etiology. Testing with stool cultures and stool ova and parasites (O&P) is not recommended. Our goal was to reduce this inappropriate testing within a large, urban safety-net hospital system.

METHODS

This was a quality improvement project. We created a best practice advisory (BPA) within the electronic medical record that fires when a stool culture or O&P order is placed 72 hours after admission for any immunocompetent patient. It states that stool testing is low yield and offers the option to remove the order. We measured weekly counts of stool culture and stool O&P orders pre- and postintervention. We also measured the BPA acceptance rate, the 24-hour stool testing reorder rate, and Clostridioides difficile infection rates. Data were analyzed using Welch tests as well as a quasi-experimental pre- and postintervention interrupted time series regression analysis.

RESULTS

Stool culture orders decreased by 24.4% (P < .001). There was a significant level difference and slope difference with linear regression. Five of the 11 hospitals had a significant reduction in stool culture orders. Stool O&P orders decreased by 18.2% (P < .01). Three of the 11 hospitals had a significant reduction in stool O&P orders.

CONCLUSIONS

Our intervention successfully reduced inappropriate stool testing within a large safety-net hospital system.

Nudging to select single-lumen over multiple-lumen peripherally inserted central catheters (PICCs) in a large safety net system

BACKGROUND

Peripherally inserted central catheters (PICCs) are increasingly used for vascular access in inpatient settings. Compared to multilumen PICCs, single-lumen PICCs carry a lower rate of complications, including central-line-associated bloodstream infection and thrombosis. Despite this, multilumen PICCs are still overused.

METHODS

This quality improvement initiative was implemented across 11 hospitals at New York City Health + Hospitals safety net system. The electronic health record (EHR) interventional radiology or vascular access team consultation orders were modified to allow for lumen choice, with default selection to a single-lumen PICC.

RESULTS

Average single-lumen PICC utilization increased by 25.5%, from 44.4% to 69.9% (P < .001). CLABSI rates had a nonsignificant reduction by 26.7% from 2.44 to 1.79 infections per month (P = .255). Among provider types in the postintervention period, single-lumen PICC utilization ranged from 67.7% for advanced practice providers to 82.4%-94.6% for physicians. Among provider specialties, utilization ranged from 31.8% for neurology to 97.7% for orthopedics. Additionally, there was large variation in pre- and postintervention differences in utilization by hospital.

CONCLUSIONS

We successfully increased single-lumen PICC utilization across all 11 safety net hospitals. This expands on previous work on improving single-lumen PICC use and use of default nudges in large, resource-limited settings. Further study is needed to examine variation among provider types, specialties, and hospitals.

Reducing duplicate genetic testing in inpatient and outpatient settings across a large safety-net system

OBJECTIVES

To develop a system-wide electronic health record (EHR) intervention at a large safety-net hospital system to reduce wasteful duplicate genetic testing.

METHODS

This project was initiated at a large urban public health care system. An EHR alert was designed to be triggered when a clinician attempted to order any of 16 specified genetic tests for which a previous result existed within the EHR system. Measurements included the proportion of completed genetic tests that were duplicates and alerts per 1,000 tests. Data were stratified by clinician type, specialty, and inpatient vs ambulatory setting.

RESULTS

Across all settings, the rate of duplicate genetic testing decreased from 2.35% (1,050 of 44,592 tests) to 0.09% (21 of 22,323 tests) (96% relative reduction, P < .001). The alert rate per 1,000 tests was 277 for inpatient orders and 64 for ambulatory orders. Among clinician types, residents had the highest alert rate per 1,000 tests at 166 and midwives the lowest at 51 (P < .01). Among clinician specialties, internal medicine had the highest alert rate per 1,000 tests at 245 and obstetrics and gynecology the lowest at 56 (P < .01).

CONCLUSIONS

The EHR intervention successfully reduced duplicate genetic testing by 96% across a large safety-net setting.

Reducing daily dosing in opioid prescriptions in 11 safety net emergency departments

BACKGROUND

The United States continues to face a significant issue with opioid misuse, overprescribing, dependency, and overdose. Electronic health record (EHR) interventions have shown to be an effective tool to modify opioid prescribing behaviors. This quality improvement project describes an EHR intervention to reduce daily dosing in opioid prescriptions in 11 emergency departments (ED) across the largest safety net health system in the US.

MEASURES

The primary outcome measure was the rates of oxycodone-acetaminophen 5-325 mg prescriptions exceeding 50 morphine milligram equivalents per day (MMED) pre- vs. post-intervention; and stratified by individual hospitals and provider type.

INTERVENTION

The defaults for dose and frequency were uniformly changed to 'every 6 hours as needed' and '1 tablet', respectively, across 11 EDs.

OUTCOMES

The percentage of prescriptions greater than or equal to 50 MMED decreased from 46.0% (1624 of 3530 prescriptions) to 1.6% (52 of 3165 prescriptions) (96.4% relative reduction; p < 0.001). All 11 hospitals had a significant reduction in prescriptions exceeding 50 MMED. Nurse practitioners had the highest relative reduction of prescriptions exceeding 50 MMED at 100% (p < 0.001), and the attendings/fellows had the lowest relative reduction at 95.6% (p < 0.001).

CONCLUSIONS/LESSONS LEARNED

Default nudges are a simple yet powerful intervention that can strongly influence opioid prescribing patterns.

Evaluation of COVID-19 vaccine implementation in a large safety net health system

Objectives

To evaluate rapid COVID-19 vaccine clinic implementation from January-April 2021 in the Los Angeles County Department of Health Services (LACDHS), the second-largest US safety net health system. During initial vaccine clinic implementation, LACDHS vaccinated 59,898 outpatients, 69% of whom were Latinx (exceeding the LA County Latinx population of 46%). LACDHS is a unique safety net setting to evaluate rapid vaccine implementation due to system size, geographic breadth, language/racial/ethnic diversity, limited health staffing resources, and socioeconomic complexity of patients.

Methods

Implementation factors were assessed through semi-structured interviews of staff from all twelve LACDHS vaccine clinics from August-November 2021 using the Consolidated Framework for Implementation Research (CFIR) and themes analyzed using rapid qualitative analysis.

Results

Of 40 potential participants, 25 health professionals completed an interview (27% clinical providers/medical directors, 23% pharmacist, 15% nursing staff, and 35% other). Qualitative analysis of participant interviews yielded ten narrative themes. Implementation facilitators included bidirectional communication between system leadership and clinics, multidisciplinary leadership and operations teams, expanded use of standing orders, teamwork culture, use of active and passive communication structures, and development of patient-centered engagement strategies. Barriers to implementation included vaccine scarcity, underestimation of resources needed for patient outreach, and numerous process challenges encountered.

Conclusion

Previous studies focused on robust advance planning as a facilitator and understaffing and high staff turnover as barriers to implementation in safety net health systems. This study found facilitators that can mitigate lack of advance planning and staffing challenges present during public health emergencies such as the COVID-19 pandemic. The ten identified themes may inform future implementations in safety net health systems.

Initiative to reduce inappropriate venous thromboembolism prophylaxis in an 11-hospital safety net system: An electronic health records-based approach

BACKGROUND

While pharmacologic prophylaxis has benefits for venous thromboembolism (VTE) prevention in high-risk patients, unnecessary use carries potential harm, including bleeding, heparin-induced thrombocytopenia, and patient discomfort, and should be avoided in low-risk patients. While many quality improvement initiatives aim to reduce underuse, successful models on reducing overuse are sparse in the literature.

OBJECTIVE

We aimed to create a quality improvement initiative to reduce overuse of pharmacologic VTE prophylaxis.

DESIGNS, SETTINGS AND PARTICIPANTS

A quality improvement initiative was implemented across 11 safety net hospitals in New York City.

INTERVENTION

The first electronic health record (EHR) intervention consisted of a VTE order panel that facilitated risk assessment and recommended VTE prophylaxis for high-risk patients only. The second EHR intervention used a best practice advisory that alerted clinicians when prophylaxis was ordered for a patient previously deemed "low risk." Prescribing rates were compared through a three-segment interrupted time series linear regression design.

RESULTS

Compared to the preintervention period, the first intervention did not change the rate of total pharmacologic prophylaxis immediately after implementation (1.7% relative change, p = .38) or over time (slope difference of 0.20 orders per 1000 patient days, p = .08). Compared to the first intervention period, the second intervention led to an immediate 4.5% reduction in total pharmacologic prophylaxis (p = .04) but increased thereafter (slope difference of 0.24, p = .03) such that weekly rates at the end of the study were similar to rates prior to the second intervention.

Multilevel Determinants of Digital Health Equity: A Literature Synthesis to Advance the Field

Current digital health approaches have not engaged diverse end users or reduced health or health care inequities, despite their promise to deliver more tailored and personalized support to individuals at the right time and the right place. To achieve digital health equity, we must refocus our attention on the current state of digital health uptake and use across the policy, system, community, individual, and intervention levels. We focus here on (a) outlining a multilevel framework underlying digital health equity; (b) summarizingfive types of interventions/programs (with example studies) that hold promise for advancing digital health equity; and (c) recommending future steps for improving policy, practice, and research in this space.

Randomized trial protocol for remote monitoring for equity in advancing the control of hypertension in safety net systems (REACH-SNS) study

BACKGROUND

Self-measured blood pressure monitoring (SMBP) is essential to effective management of hypertension. This study aims to evaluate effectiveness and implementation of SMBP that leverages: cellular-enabled home BP monitors without a need for Wi-Fi or Bluetooth; simple communication modalities such as text messaging to support patient engagement; and integration into existing team-based workflows in safety-net clinics.

METHODS

This study will be conducted with patients in San Francisco who are treated within a network of safety-net clinics. English and Spanish-speaking patients with diagnosed hypertension will be eligible for the trial if they have recent BP readings ≥140/90 mmHg and do not have co-morbid conditions that make home BP monitoring more complex to manage. This study will implement a three-arm randomized controlled trial to compare varying levels of implementation support: 1) cellular-enabled BP monitors (with minimal implementation support), 2) cellular-enabled BP monitors with protocol-based implementation support (text reminders for patients; aggregated BP summaries sent to primary care providers), and 3) cellular-enabled BP monitors and pharmacist-led support (pharmacist coaching and independent medication adjustments).

RESULTS

For the main analysis, we will use mixed effects linear regression to compare the change in primary outcome of systolic BP. Secondary outcomes include BP control (<140/90 mmHg), medication intensification, patient-reported outcomes, and implementation processes (i.e., engagement with the intervention).

DISCUSSION

This study will design and test a digital health intervention for use in marginalized populations treated within safety net settings, evaluating both effectiveness and implementation to advance more equitable health outcomes.

Choosing wisely initiative for reducing urine cultures for asymptomatic bacteriuria and catheter-associated asymptomatic bacteriuria in an 11-hospital safety net system

BACKGROUND

Treatment of asymptomatic bacteriuria (ASB) is common. Overtreatment of ASB leads to harm, including adverse effects from antibiotics, antibiotic resistance, and increased length of stay.

METHODS

This quality improvement initiative targeted inappropriate urine cultures (UC) across 11 hospitals and 70 ambulatory clinics in a safety-net setting. A mandatory prompt for appropriate indications for UC orders and a best practice advisory (BPA) for UC on patients with urinary catheters were created. UC ordering was compared preintervention (June 2020-October 2021) to postintervention (December 2021-August 2022). Variation in hospitals and clinics were assessed, as well as BPA responses by clinician type and specialty.

RESULTS

Inpatient UCs decreased 20.9% (p < .001), and outpatient UCs was unchanged (2.6% increase, nonsignificant). Inpatient UCs on patients with urinary catheters decreased 21.6% (p < .001). Temporal trends were also assessed. High variation was seen among hospitals and clinics. Low BPA acceptance rates were seen in internal medicine and family medicine clinicians. Attendings showed high acceptance to the BPA compared to other clinicians.

CONCLUSIONS

This initiative successfully decreased UCs in a large, safety-net system. Further study is needed in assessing variation among hospitals and clinics, as well as among clinician types and specialties.

Telemedicine implementation and use in community health centers during COVID-19: Clinic personnel and patient perspectives

In March 2020, federal and state telehealth policy changes catalyzed telemedicine adoption and use in community health centers. There is a dearth of evidence on telemedicine implementation and use in these safety net settings and a lack of information reflecting the perspectives of patients with limited English proficiency. We conducted in-depth interviews with clinic personnel and patients during the pandemic in two federally qualified health centers that primarily serve Chinese and Latino immigrants. Twenty-four interviews (clinic personnel ​= ​15; patients who primarily speak a language other than English ​= ​9) were completed remotely between December 2020 and April 2021. Interview scripts included questions about their telemedicine experiences, technology, resources and needs, barriers, facilitators, language access, and continued use, with a brief socio-demographic survey. Data analyses involved a primarily deductive approach and thematic analysis of transcript content. Both FQHCs adopted telemedicine in a few weeks and transitioned primarily to video and audio-only visits within two months. Findings reveal third-party language interpretation services were challenging to integrate into telemedicine video visits. Bilingual personnel who provided language concordant care were seen as essential for efficient and high-quality patient telemedicine experiences. Audio-only visits were of particular benefit to reach patients of older age, with limited English proficiency, and with limited digital literacy. Continued use of telemedicine is contingent on reimbursement policy decisions and interventions to increase patient digital literacy and technological resources. Results highlight the importance of reimbursing audio-only visits post-pandemic and investing in efforts to improve the quality of language services in telemedicine encounters.

SEE the DIFFerence: Reducing unnecessary C. difficile orders through clinical decision support in a large, urban safety-net system

BACKGROUND

Clostridioides difficile (C. difficile) is a hospital-acquired infection. Overtesting for C. difficile leads to false positive results due to a high rate of asymptomatic colonization, resulting in unnecessary and harmful treatment for patients.

METHODS

This was a quality improvement initiative to decrease the rate of inappropriate C. difficile testing across 11 hospitals in an urban, safety-net setting. Three best practice advisories were created, alerting providers of recent laxative administration within 48 hours, a recent positive test within 14 days, and a recent negative test within 7 days. The outcome measures were the number of C. difficile tests per 1,000 patient days, as well as the rate of hospital onset C. difficile infection was compared pre- and post-intervention. The process measures included the rate of removal of the C. difficile test from the best practice advisory, as well as the subsequent 24-hour re-order rate.

RESULTS

The number of C. difficile tests decreased by 27.3% from 1.1 per 1,000 patient days preintervention (May 25, 2020-May 24, 2021) to 0.8 per 1,000 patient days postintervention, (May 25, 2021-March 25, 2022), P < .001. When stratified by hospital, changes in testing ranged from an increase of 12.5% to a decrease of 60%. Analysis among provider type showed higher behavior change among attendings than compared to trainees or advanced practice providers. There was a 12.1%, nonsignificant decrease in C. difficile rates from preintervention, 0.33 per 1,000 patient days compared to postintervention, 0.29 per 1,000 patient days, P=.32.

CONCLUSIONS

Using only an electronic health record intervention, we successfully decreased C. difficile orders after 72 hours of admission in a large, safety-net system. Variation existed among hospitals and by provider type.

Assessing Alignment of Patient and Clinician Perspectives on Community Health Resources for Chronic Disease Management

Addressing social determinants of health (SDoH) is associated with improved clinical outcomes for patients with chronic diseases in safety-net settings. This qualitative study supplemented by descriptive quantitative analysis investigates the degree of alignment between patient and clinicians’ perceptions of SDoH resources and referrals in clinics within the public healthcare delivery system in San Francisco. We conducted a qualitative analysis of in-depth interviews, patient-led neighborhood tours, and in-person clinic visit observations with 10 patients and 7 primary care clinicians. Using a convergent parallel mixed methodology, we also completed a descriptive quantitative analysis comparing the categories of neighborhood health resources mentioned by patients or community leaders to the resources integrated into the electronic health record. We found that patients held a wealth of knowledge about neighborhood resources relevant to SDoH that were highly localized and specific to their communities. In addition, multiple stakeholders were involved in conducting SDoH screenings and referrals, including clinicians, system navigators such as case workers, and community-based organizations. Yet, the information flow between these stakeholders and patients lacked systematization, and the prioritization of social needs by patients and clinicians was misaligned, as represented by qualitative themes as well as quantitative differences in resource category distribution analysis (p < 0.001). Our results shed light upon opportunities for strengthening social care delivery in safety-net healthcare settings by improving patient engagement, clinic workflow, EHR engagement, and resource dissemination.

Training the next generation of learning health system scientists

Introduction

The learning health system (LHS) aligns science, informatics, incentives, stakeholders, and culture for continuous improvement and innovation. The Agency for Healthcare Research and Quality and the Patient‐Centered Outcomes Research Institute designed a K12 initiative to grow the number of LHS scientists. We describe approaches developed by 11 funded centers of excellence (COEs) to promote partnerships between scholars and health system leaders and to provide mentored research training.

Methods

Since 2018, the COEs have enlisted faculty, secured institutional resources, partnered with health systems, developed and implemented curricula, recruited scholars, and provided mentored training. Program directors for each COE provided descriptive data on program context, scholar characteristics, stakeholder engagement, scholar experiences with health system partnerships, roles following program completion, and key training challenges.

Results

To date, the 11 COEs have partnered with health systems to train 110 scholars. Nine (82%) programs partner with a Veterans Affairs health system and 9 (82%) partner with safety net providers. Clinically trained scholars (n = 87; 79%) include 70 physicians and 17 scholars in other clinical disciplines. Non‐clinicians (n = 29; 26%) represent diverse fields, dominated by population health sciences. Stakeholder engagement helps scholars understand health system and patient/family needs and priorities, enabling opportunities to conduct embedded research, improve outcomes, and grow skills in translating research methods and findings into practice. Challenges include supporting scholars through roadblocks that threaten to derail projects during their limited program time, ranging from delays in access to data to COVID‐19‐related impediments and shifts in organizational priorities.

Conclusions

Four years into this novel training program, there is evidence of scholars' accomplishments, both in traditional academic terms and in terms of moving along career trajectories that hold the potential to lead and accelerate transformational health system change. Future LHS training efforts should focus on sustainability, including organizational support for scholar activities.

Brief Report: HIV Pre-exposure Prophylaxis Prescribing in an Urban Safety-Net Health System

INTRODUCTION

Safety-net health systems are key settings for HIV pre-exposure prophylaxis (PrEP) implementation, but little evidence is available about the frequency of PrEP prescribing in safety-net settings. We assessed PrEP prescribing among people with indications for PrEP at an urban safety-net health system that serves a county designated as an Ending the HIV Epidemic priority jurisdiction.

METHODS

We identified adults (aged 18 years or older) who engaged in primary care between January 2015 and December 2019 and had a documented indication for PrEP. PrEP indications included the presence of a behavioral or sexual risk factor of HIV acquisition or a positive bacterial sexually transmitted infection at the index visit. PrEP prescribing was defined as the proportion of patients with indications for PrEP who received a new prescription for PrEP. We estimated the cumulative incidence of PrEP prescription with corresponding 95% confidence limits (CL).

RESULTS

Our study population comprised 2957 individuals, of whom 58% was aged younger than 45 years, 56% was women, 67% was racial or ethnic minorities, and 60% was uninsured or provided care as part of a hospital-based managed care plan for individuals without insurance. We identified 41 individuals who were prescribed PrEP. The cumulative incidence of PrEP prescribing within 1 year of the first documented PrEP indication was 1.3% (95% CL: 0.91% to 1.7%).

CONCLUSIONS

Our results suggest extremely low frequency of PrEP prescribing among people with indications for PrEP in an urban safety-net health system. Strategies are needed to improve PrEP implementation in high-priority populations and safety-net settings.

Barriers and Facilitators to the Implementation of Virtual Reality as a Pain Management Modality in Academic, Community, and Safety-Net Settings: Qualitative Analysis

Background

Prior studies have shown that virtual reality (VR) is an efficacious treatment modality for opioid-sparing pain management. However, the majority of these studies were conducted among primarily White, relatively advantaged populations and in well-resourced settings.

Objective

We conducted a qualitative, theory-informed implementation science study to assess the readiness for VR in safety-net settings.

Methods

Using the theoretical lens of the Consolidated Framework for Implementation Research (CFIR) framework, we conducted semistructured interviews with current VR users and nonusers based in safety-net health systems (n=15). We investigated barriers and facilitators to a commercially available, previously validated VR technology platform AppliedVR (Los Angeles, CA, USA). We used deductive qualitative analysis using the overarching domains of the CFIR framework and performed open, inductive coding to identify specific themes within each domain.

Results

Interviewees deemed the VR intervention to be useful, scalable, and an appealing alternative to existing pain management approaches. Both users and nonusers identified a lack of reimbursement for VR as a significant challenge for adoption. Current users cited positive patient feedback, but safety-net stakeholders voiced concern that existing VR content may not be relevant or appealing to diverse patients. All respondents acknowledged the challenge of integrating and maintaining VR in current pain management workflows across a range of clinical settings, and this adoption challenge was particularly acute, given resource and staffing constraints in safety-net settings.

Conclusions

VR for pain management holds interest for frontline pain management clinicians and leadership in safety-net health settings but will require significant tailoring and adaption to address the needs of diverse populations. Integration into complex workflows for pain management is a significant barrier to adoption, and participants cited structural cost and reimbursement concerns as impediments to initial implementation and scaling of VR use.

Psychiatrist and Psychologist Experiences with Telehealth and Remote Collaborative Care in Primary Care: A Qualitative Study

PURPOSE

Availability of mental health services is limited in the rural United States. Two promising models to reach patients with limited access to care are telehealth referral and collaborative care. The objective of this study was to assess telepsychiatrist- and telepsychologist-level facilitators and barriers to satisfaction with and implementation of these 2 telehealth models in rural settings.

METHODS

Focus groups were held in 2019 using a semistructured interview guide. Participants were off-site telepsychiatrists (N = 10) and telepsychologists (N = 4) for primary care clinics across 3 states (Washington, Michigan, and Arkansas) involved in a recent pragmatic comparative effectiveness trial. Qualitative analysis occurred inductively by 2 independent coders.

FINDINGS

Participants were satisfied with the models partly owing to good patient rapport and expanding access to care. Teamwork was highlighted as a facilitator in collaborative care and was often related to work with care managers. However, participants described communication with primary care providers as a challenge, especially in the telehealth referral arm. Barriers centered on variability of logistical processes (eg, symptom monitoring, scheduling, electronic medical record processes, and credentialing) among sites. Staff turnover, variable clinic investment, and inadequacy of training were possible explanations for these barriers.

CONCLUSIONS

Participants described high motivation to provide team-based, remote care for patients, though they experienced operational challenges. Centralized credentialing, scheduling, and record keeping are possible solutions. These findings are important because consulting psychiatrists and psychologists may play a leadership role in the dissemination of these models.

Real-world insights from launching remote peer-to-peer mentoring in a safety net healthcare delivery setting

Peer mentors have been proven to improve diabetes outcomes, especially among diverse patients. Delivering peer mentoring via remote strategies (phone, text, mobile applications) is critical, especially in light of the recent pandemic. We conducted a real-world evaluation of a remote diabetes intervention in a safety-net delivery system in New York. We summarized the uptake, content, and pre-post clinical effectiveness for English- and Spanish-speaking participants. Of patients who could be reached, 71% (n = 690/974) were enrolled, and 90% of those (n = 618/690) participated in coaching. Patients and mentors had a mean of 32 check-ins, and each patient set an average of 10 goals. 29% of the participants accessed the program via the smartphone application. Among participants with complete hemoglobin A1c data (n = 179), there was an absolute 1.71% reduction (P < .01). There are multiple lessons for successful implementation of remote peer coaching into settings serving diverse patients, including meaningful patient-mentor matching and addressing social determinants.

Adaptive learning algorithms to optimize mobile applications for behavioral health: guidelines for design decisions

OBJECTIVE

Providing behavioral health interventions via smartphones allows these interventions to be adapted to the changing behavior, preferences, and needs of individuals. This can be achieved through reinforcement learning (RL), a sub-area of machine learning. However, many challenges could affect the effectiveness of these algorithms in the real world. We provide guidelines for decision-making.

MATERIALS AND METHODS

Using thematic analysis, we describe challenges, considerations, and solutions for algorithm design decisions in a collaboration between health services researchers, clinicians, and data scientists. We use the design process of an RL algorithm for a mobile health study "DIAMANTE" for increasing physical activity in underserved patients with diabetes and depression. Over the 1.5-year project, we kept track of the research process using collaborative cloud Google Documents, Whatsapp messenger, and video teleconferencing. We discussed, categorized, and coded critical challenges. We grouped challenges to create thematic topic process domains.

RESULTS

Nine challenges emerged, which we divided into 3 major themes: 1. Choosing the model for decision-making, including appropriate contextual and reward variables; 2. Data handling/collection, such as how to deal with missing or incorrect data in real-time; 3. Weighing the algorithm performance vs effectiveness/implementation in real-world settings.

CONCLUSION

The creation of effective behavioral health interventions does not depend only on final algorithm performance. Many decisions in the real world are necessary to formulate the design of problem parameters to which an algorithm is applied. Researchers must document and evaulate these considerations and decisions before and during the intervention period, to increase transparency, accountability, and reproducibility.

TRIAL REGISTRATION

clinicaltrials.gov, NCT03490253.

mHealth app using machine learning to increase physical activity in diabetes and depression: clinical trial protocol for the DIAMANTE Study

INTRODUCTION

Depression and diabetes are highly disabling diseases with a high prevalence and high rate of comorbidity, particularly in low-income ethnic minority patients. Though comorbidity increases the risk of adverse outcomes and mortality, most clinical interventions target these diseases separately. Increasing physical activity might be effective to simultaneously lower depressive symptoms and improve glycaemic control. Self-management apps are a cost-effective, scalable and easy access treatment to increase physical activity. However, cutting-edge technological applications often do not reach vulnerable populations and are not tailored to an individual's behaviour and characteristics. Tailoring of interventions using machine learning methods likely increases the effectiveness of the intervention.

METHODS AND ANALYSIS

In a three-arm randomised controlled trial, we will examine the effect of a text-messaging smartphone application to encourage physical activity in low-income ethnic minority patients with comorbid diabetes and depression. The adaptive intervention group receives messages chosen from different messaging banks by a reinforcement learning algorithm. The uniform random intervention group receives the same messages, but chosen from the messaging banks with equal probabilities. The control group receives a weekly mood message. We aim to recruit 276 adults from primary care clinics aged 18-75 years who have been diagnosed with current diabetes and show elevated depressive symptoms (Patient Health Questionnaire depression scale-8 (PHQ-8) >5). We will compare passively collected daily step counts, self-report PHQ-8 and most recent haemoglobin A1c from medical records at baseline and at intervention completion at 6-month follow-up.

ETHICS AND DISSEMINATION

The Institutional Review Board at the University of California San Francisco approved this study (IRB: 17-22608). We plan to submit manuscripts describing our user-designed methods and testing of the adaptive learning algorithm and will submit the results of the trial for publication in peer-reviewed journals and presentations at (inter)-national scientific meetings.

TRIAL REGISTRATION NUMBER

NCT03490253; pre-results.

National Outreach of Telepalliative Medicine Volunteers for a New York City Safety Net System COVID-19 Pandemic Response

The coronavirus disease 2019 surge in New York City created an increased demand for palliative care (PC) services. In staff-limited settings such as safety net systems, and amid growing reports of health care worker illness, leveraging help from less-affected areas around the country may provide an untapped source of support. A national social media outreach effort recruited 413 telepalliative medicine volunteers (TPMVs). After expedited credentialing and onboarding of 67 TPMVs, a two-week pilot was initiated in partnership with five public health hospitals without any previous existing telehealth structure. The volunteers completed 109 PC consults in the pilot period. Survey feedback from TPMVs and on-site PC providers was largely positive, with areas of improvement identified around electronic health record navigation and continuity of care. This was a successful, proof of concept, and quality improvement initiative leveraging TPMVs from across the nation for a PC pandemic response in a safety net system.

Research-grade data in the real world: challenges and opportunities in data quality from a pragmatic trial in community-based practices

Randomized controlled trials face cost, logistic, and generalizability limitations, including difficulty engaging racial/ethnic minorities. Real-world data (RWD) from pragmatic trials, including electronic health record (EHR) data, may produce intervention evaluation findings generalizable to diverse populations. This case study of Project IMPACT describes unique barriers and facilitators of optimizing RWD to improve health outcomes and advance health equity in small immigrant-serving community-based practices. Project IMPACT tested the effect of an EHR-based health information technology intervention on hypertension control among small urban practices serving South Asian patients. Challenges in acquiring accurate RWD included EHR field availability and registry capabilities, cross-sector communication, and financial, personnel, and space resources. Although using RWD from community-based practices can inform health equity initiatives, it requires multidisciplinary collaborations, clinic support, procedures for data input (including social determinants), and standardized field logic/rules across EHR platforms.

Redesigning primary care in the safety net: A qualitative analysis of team-based care implementation

BACKGROUND

Team-based care is an innovative approach to primary care in which groups of health care professionals work collaboratively to manage care for groups of patients. Safety-net organizations face specific barriers to implementing health care innovations. More research is needed that documents the dynamics that inform implementation and sustainment of innovative practices in the safety net.

METHODS

We conducted qualitative fieldwork, including purposeful observation and semi-structured and informal interviews, to explore the implementation of care teams in safety-net clinics in California. Field notes and transcripts were analyzed using an inductive approach to identify overarching themes across sites.

RESULTS

Salient themes across clinics suggest that 1) well-designed physical layouts that promote sustained patterns of structured and spontaneous communication and collaboration are critical to creating high-functioning teams; 2) implementation of team-based care relies on a relaxing of the traditional clinic hierarchy into an agile organizational structure that empowers all clinic staff-most notably medical assistants-by facilitating an ethos of collaborative decision-making, interdependence, and shared responsibility; and 3) resource constraints related to recruitment and retention of qualified staff are key barriers to implementation.

CONCLUSIONS

Team-based care has the potential to improve patient outcomes, strengthen relationships and well-being among clinic staff, and expand staff roles to facilitate collective accomplishment of work goals. To successfully launch and sustain care teams in safety-net clinics, greater commitments to creating well-designed spaces and a re-envisioning of the training and compensation for medical assistants that reflects the valuable role these individuals can play appear to be necessary.