Study protocol for "Study of Practices Enabling Implementation and Adaptation in the Safety Net (SPREAD-NET)": a pragmatic trial comparing implementation strategies

"I didn't know you could do that": A Pilot Assessment of EHR Optimization Training

BACKGROUND

Informatics tools within electronic health records (EHRs)-for example, data rosters and clinical reminders-can help disseminate care guidelines into clinical practice. Such tools' adoption varies widely, however, possibly because many primary care providers receive minimal training in even basic EHR functions.

OBJECTIVES

This mixed-methods evaluation of a pilot training program sought to identify factors to consider when providing EHR use optimization training in community health centers (CHCs) as a step toward supporting CHC providers' adoption of EHR tools.

METHODS

In spring 2018, we offered 10 CHCs a 2-day, 16-hour training in EHR use optimization, provided by clinician trainers, and customized to each CHC's needs. We surveyed trainees pre- and immediately post-training and again 3 months later. We conducted post-training interviews with selected clinic staff, and conducted a focus group with the trainers, to assess satisfaction with the training, and perceptions of how it impacted subsequent EHR use.

RESULTS

Six CHCs accepted and received the training; 122 clinic staff members registered to attend, and most who completed the post-training survey reported high satisfaction. Three months post-training, 80% of survey respondents said the training had changed their daily EHR use somewhat or significantly.

CONCLUSION

Factors to consider when planning EHR use optimization training in CHCs include: CHCs may face barriers to taking part in such training; it may be necessary to customize training to a given clinic's needs and to different trainees' clinic roles; identifying trainees' skill level a priori would help but is challenging; in-person training may be preferable; and inclusion of a practice coach may be helpful. Additional research is needed to identify how to provide such training most effectively.

Contemporary use of real-world data for clinical trial conduct in the United States: a scoping review

OBJECTIVE

Real-world data (RWD), defined as routinely collected healthcare data, can be a potential catalyst for addressing challenges faced in clinical trials. We performed a scoping review of database-specific RWD applications within clinical trial contexts, synthesizing prominent uses and themes.

MATERIALS AND METHODS

Querying 3 biomedical literature databases, research articles using electronic health records, administrative claims databases, or clinical registries either within a clinical trial or in tandem with methodology related to clinical trials were included. Articles were required to use at least 1 US RWD source. All abstract screening, full-text screening, and data extraction was performed by 1 reviewer. Two reviewers independently verified all decisions.

RESULTS

Of 2020 screened articles, 89 qualified: 59 articles used electronic health records, 29 used administrative claims, and 26 used registries. Our synthesis was driven by the general life cycle of a clinical trial, culminating into 3 major themes: trial process tasks (51 articles); dissemination strategies (6); and generalizability assessments (34). Despite a diverse set of diseases studied, <10% of trials using RWD for trial process tasks evaluated medications or procedures (5/51). All articles highlighted data-related challenges, such as missing values.

DISCUSSION

Database-specific RWD have been occasionally leveraged for various clinical trial tasks. We observed underuse of RWD within conducted medication or procedure trials, though it is subject to the confounder of implicit report of RWD use.

CONCLUSION

Enhanced incorporation of RWD should be further explored for medication or procedure trials, including better understanding of how to handle related data quality issues to facilitate RWD use.

Unintended consequences: a qualitative study exploring the impact of collecting implementation process data with phone interviews on implementation activities

Background

Qualitative data are crucial for capturing implementation processes, and thus necessary for understanding implementation trial outcomes. Typical methods for capturing such data include observations, focus groups, and interviews. Yet little consideration has been given to how such methods create interactions between researchers and study participants, which may affect participants’ engagement, and thus implementation activities and study outcomes. In the context of a clinical trial, we assessed whether and how ongoing telephone check-ins to collect data about implementation activities impacted the quality of collected data, and participants’ engagement in study activities.

Methods

Researchers conducted regular phone check-ins with clinic staff serving as implementers in an implementation study. Approximately 1 year into this trial, 19 of these study implementers were queried about the impact of these calls on study engagement and implementation activities. The two researchers who collected implementation process data through phone check-ins with the study implementers were also interviewed about their perceptions of the impact of the check-ins.

Results

Study implementers’ assessment of the check-ins’ impact fell into three categories: (1) the check-ins had no effect on implementation activities, (2) the check-ins served as a reminder about study participation (without relating a clear impact on implementation activities), and (3) the check-ins caused changes in implementation activities. The researchers similarly perceived that the phone check-ins served as reminders and encouraged some implementers’ engagement in implementation activities; their ongoing nature also created personal connections with study implementers that may have impacted implementation activities. Among some study implementers, anticipation of the check-in calls also improved their ability to recount implementation activities and positively affected quality of the data collected.

Conclusion

These results illustrate the potential impact of qualitative data collection on implementation activities during implementation science trials. Mitigating such effects may prove challenging, but acknowledging these consequences—or even embracing them, perhaps by designing data collection methods as implementation strategies—could enhance scientific rigor. This work is presented to stimulate debate about the complexities involved in capturing data on implementation processes using common qualitative data collection methods.

Trial registration

ClinicalTrials.gov, NCT02325531. Registered 15 December 2014.

A Practice-Based Research Network (PBRN) Roadmap for Evaluating COVID-19 in Community Health Centers: A Report From the OCHIN PBRN

BACKGROUND

Primary care practice-based research networks (PBRNs) are critical laboratories for generating evidence from real-world settings, including studying natural experiments. Primary care's response to the novel coronavirus-19 (COVID-19) pandemic is arguably the most impactful natural experiment in our lifetime. EVALUATING THE IMPACT OF COVID-19: We briefly describe the OCHIN PBRN of community health centers (CHCs), its partnership with implementation scientists, and how we are leveraging this infrastructure and expertise to create a rapid research response evaluating how CHCs across the country responded to the COVID-19 pandemic. COVID-19 RESEARCH ROADMAP: Our research agenda focuses on asking: How has care delivery in CHCs changed due to COVID-19? What impact has COVID-19 had on the delivery of preventive services in CHCs? Which PBRN services (e.g., data surveillance, training, evidence synthesis) are most impactful to real-world practices? What decision-making strategies were used in the PBRN and its practices to make real-time changes in response to the pandemic? What critical factors in successfully and sustainably transforming primary care are illuminated by pandemic-driven changes?

DISCUSSION AND CONCLUSIONS

PBRNs enable real-world evaluation of practice change and natural experiments, and thus are ideal laboratories for implementation science research. We present a real-time example of how a PBRN Implementation Laboratory activated a response to study a historic natural experiment, to help other PBRNs charting a course through this pandemic.

Does increased implementation support improve community clinics' guideline-concordant care? Results of a mixed methods, pragmatic comparative effectiveness trial

Background

Disseminating care guidelines into clinical practice remains challenging, partly due to inadequate evidence on how best to help clinics incorporate new guidelines into routine care. This is particularly true in safety net community health centers (CHCs).

Methods

This pragmatic comparative effectiveness trial used a parallel mixed methods design. Twenty-nine CHC clinics were randomized to receive increasingly intensive implementation support (implementation toolkit (arm 1); toolkit + in-person training + training webinars (arm 2); toolkit + training + webinars + offered practice facilitation (arm 3)) targeting uptake of electronic health record (EHR) tools focused on guideline-concordant cardioprotective prescribing for patients with diabetes. Outcomes were compared across study arms, to test whether increased support yielded additive improvements, and with 137 non-study CHCs that share the same EHR as the study clinics. Quantitative data from the CHCs’ EHR were used to compare the magnitude of change in guideline-concordant ACE/ARB and statin prescribing, using adjusted Poisson regressions. Qualitative data collected using diverse methods (e.g., interviews, observations) identified factors influencing the quantitative outcomes.

Results

Outcomes at CHCs receiving higher-intensity support did not improve in an additive pattern. ACE/ARB prescribing did not improve in any CHC group. Statin prescribing improved overall and was significantly greater only in the arm 1 and arm 2 CHCs compared with the non-study CHCs. Factors influencing the finding of no additive impact included: aspects of the EHR tools that reduced their utility, barriers to providing the intended implementation support, and study design elements, e.g., inability to adapt the provided support. Factors influencing overall improvements in statin outcomes likely included a secular trend in awareness of statin prescribing guidelines, selection bias where motivated clinics volunteered for the study, and study participation focusing clinic staff on the targeted outcomes.

Conclusions

Efforts to implement care guidelines should: ensure adaptability when providing implementation support and conduct formative evaluations to determine the optimal form of such support for a given clinic; consider how study data collection influences adoption; and consider barriers to clinics’ ability to use/accept implementation support as planned. More research is needed on supporting change implementation in under-resourced settings like CHCs.

Trial registration

ClinicalTrials.gov, NCT02325531. Registered 15 December 2014.

FLOW: Early results from a clinical demonstration project to improve the transition of patients with mental health disorders back to primary care

Access to mental health (MH) care is of paramount concern to U.S. health care delivery systems, including the Veterans Health Administration. To improve access, there is a need to better focus existing MH resources toward care for those most in need of specialty-level MH treatment. This article provides early results of Project FLOW's (not an acronym) approach to developing and evaluating electronic medical record (EMR)-based criteria to identify clinically stable patients and promote their effective transition from specialty MH back to primary care (PC). Implementation utilized a blended facilitation approach consistent with Integrated Promoting Action on Research Implementation in Health Services (iPARIHS). The Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework guided measurement of implementation outcomes. During FLOW, 424 unique MH patients transitioned from MH to PC; of those, only 9 (2.1%) returned to MH after that transition. Most of those patients (n = 335; 79.0%) were first identified on the MH FLOW report, but 89 (21.0%) were other MH patients. The total number of patients discharged due to recovery or stabilization was 411. The 335 patients represent 21.3% of all unique patients (n = 1,566) who met the EMR criteria during the project. The 411 recovered/stabilized patients are 16.4% of all unique MH patients (n = 2,504) treated at the site. These early results suggest that this EMR-based system, combined with sound clinical practices, can be used to identify MH patients who are candidates for transition and foster their effective transition to care management in PC. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Quality and utilization of the Finnish clinical practice guideline in schizophrenia: evaluation using AGREE II and the vignette approach

PURPOSE

This study aims to evaluate the quality of the Finnish clinical practice guideline related to schizophrenia care and how it can be utilized in psychiatric services.

PARTICIPANTS AND METHODS

The data were collected from one psychiatric ward and seven psychiatric outpatient units situated in two cities in Southern Finland. A total of 49 professionals working in these sites participated in this study. A descriptive study design was adopted. The Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument with six domains including 23 items was used to assess the quality of the guideline, whereas a vignette was implemented to describe how staff would use the guideline in hypothetical cases in daily practice. The analysis of AGREE II was based on rating each of the 23 items on a seven-point scale and calculating a quality score for the six domains. To describe the utilization of the guideline, eight key recommendations of the guideline were deductively rated out of the respondents' answers.

RESULTS

The results showed that the "scope and purpose" of the guideline were well described, but "applicability" was insufficient. The overall quality of the guideline was high (73%). Almost one fifth of the respondents were in agreement with key recommendations.

CONCLUSION

The overall quality of Guideline for Schizophrenia was good, but its "applicability" and utilization should be improved.

Innovative Implementation Studies Conducted in US Safety Net Health Care Settings: A Systematic Review

Little is known about dissemination and implementation in safety net settings. The authors conducted a literature review of innovation/implementation studies in US safety net health care settings between 2008 and 2017. Each article was coded for (1) intervention characteristics, (2) implementation stage, (3) internal versus external ownership, and (4) prespecified implementation outcomes (eg, acceptability and fidelity). Twenty studies were identified; the majority were implemented within community clinics or integrated safety net systems (15 articles), most involved care process improvements (13 articles), and most were internally developed (13 articles). The internally developed innovations reported fewer barriers to acceptability among staff/providers, higher leadership involvement and organizational alignment, greater amounts of customization to the local setting, and better sustainment. Future work should harness the high levels of alignment and acceptability in implementation research within safety net settings, with an eye toward maintaining fidelity to facilitate dissemination across sites.

A pragmatic randomized control trial and realist evaluation on the implementation and effectiveness of an internet application to support self-management among individuals seeking specialized mental health care: a study protocol

BACKGROUND

Mental illness is a substantial and rising contributor to the global burden of disease. Access to and utilization of mental health care, however, is limited by structural barriers such as specialist availability, time, out-of-pocket costs, and attitudinal barriers including stigma. Innovative solutions like virtual care are rapidly entering the health care domain. The advancement and adoption of virtual care for mental health, however, often occurs in the absence of rigorous evaluation and adequate planning for sustainability and spread.

METHODS

A pragmatic randomized controlled trial with a nested comparative effectiveness arm, and concurrent realist process evaluation to examine acceptability, effectiveness, and cost-effectiveness of the Big White Wall (BWW) online platform for mental health self-management and peer support among individuals aged 16 and older who are accessing mental health services in Ontario, Canada. Participants will be randomized to 3 months of BWW or treatment as usual. At the end of the 3 months, participants in the intervention group will have the opportunity to opt-in to an intervention extension arm. Those who opt-in will be randomized to receive an additional 3 months of BWW or no additional intervention. The primary outcome is recovery at 3 months as measured by the Recovery Assessment Scale-revised (RAS-r). Secondary outcomes include symptoms of depression and anxiety measured with the Personal Health Questionnaire-9 item (PHQ-9) and the Generalized Anxiety Disorder Questionnaire-7 item (GAD-7) respectively, quality of life measured with the EQ-5D-5L, and community integration assessed with the Community Integration Questionnaire. Cost-effectiveness evaluations will account for the cost of the intervention and direct health care costs. Qualitative interviews with participants and stakeholders will be conducted throughout.

DISCUSSION

Understanding the impact of virtual strategies, such as BWW, on patient outcomes and experience, and health system costs is essential for informing whether and how health system decision-makers can support these strategies system-wide. This requires clear evidence of effectiveness and an understanding of how the intervention works, for whom, and under what circumstances. This study will produce such effectiveness data for BWW, while simultaneously exploring the characteristics and experiences of users for whom this and similar online interventions could be helpful.

TRIAL REGISTRATION

Clinicaltrials.gov NCT02896894 . Registered on 31 August 2016 (retrospectively registered).