Implementation fidelity, attitudes, and influence: a novel approach to classifying implementer behavior

Process evaluation of the implementation of the assessment of burden of chronic conditions tool in Dutch primary care - lessons from a qualitative implementation study

BACKGROUND

The Assessment of Burden of Chronic Conditions (ABCC-)tool is developed to facilitate a personalized approach to care in the patient-healthcare provider (HCP) conversation based on shared decision-making and individualized care plans. An effectiveness study highlighted its effect on the perceived quality of care and patient activation. Successful implementation of novel interventions necessitates an understanding of the user's actual application, user experiences and an evaluation of implementation outcomes. This study aims to evaluate the implementation of the ABCC-tool by HCPs in Dutch primary care.

METHODS

This study is the process evaluation of a larger type 1 effectiveness-implementation hybrid trial. Semi-structured interviews with HCPs, who were interventionists in the hybrid trial, were held at three and twelve months after they started using the ABCC-tool. The Reach-Effectiveness-Adoption-Implementation-Maintenance (RE-AIM) framework was used to evaluate implementation outcomes. The Implementation domain was further strengthened with an evaluation of implementation fidelity using Carroll's framework. Inductive coding and thematic analysis were applied to identify relevant participant experiences and implementation outcomes within the RE-AIM framework.

RESULTS

Seventeen HCPs (1 general practitioner, 16 practice nurses) participated in the study, representing 39% of potentially eligible participants. Most HCPs applied the tool after finishing their own routines instead of how it is intended to be used, namely from the beginning of the consultation. HCPs reached 2-6 patients. The ABCC-tool was initially adopted, but twelve HCPs stopped using the tool due to COVID-19 related cancellation of consultations. High fidelity was found for applying the questionnaire and visualization. Low fidelity was present for applying shared decision-making, formulating care goals and monitoring progress. HCPs indicated that maintaning the ABCC-tool depended on accompanying training and implementation support.

CONCLUSIONS

HCPs applied the ABCC-tool critically different from intended, potentially diminishing its benefits and ease of use. This evaluation stresses the need for a tailored implementation plan that includes more detailed training and guidance on how and when to use the ABCC-tool.

How the experiences of implementation support recipients contribute to implementation outcomes

Introduction

There is a growing body of literature on the activities and competencies of implementation support practitioners (ISPs) and the outcomes of engaging ISPs to support implementation efforts. However, there remains limited knowledge about the experiences of implementation support recipients who engage with ISPs and how these experiences shape the trajectory of implementation and contribute to implementation outcomes. This study aimed to extend the research on ISPs by describing the experiences of professionals who received implementation support and inform our understanding of the mechanisms by which ISPs produce behavior change and contribute to implementation outcomes.

Methods

Thirteen individuals with roles in supporting implementation efforts at a private foundation participated in semi-structured interviews. Data were analyzed using qualitative narrative analysis and episode profile analysis approaches. Iterative diagramming was used to visualize the pathway of experiences of implementation support recipients evidenced by the interview data.

Results

The majority of recipients described how positive experiences and trusting relationships with ISPs increased acceptance of implementation science throughout the foundation and increased the perception of implementation science as both an appropriate and feasible approach for strengthening the impact of foundation strategies. As perceptions of appropriateness and feasibility increased, recipients of implementation support described increasing knowledge and application of implementation science in their funding engagements and internal foundation strategies. Finally, recipients reported that the application of implementation science across the foundation led to sustained implementation capacity and better outcomes.

Discussion

The experiences of implementation support recipients described in this paper provide a source for further understanding the mechanisms of change for delivering effective implementation support leading to better implementation quality. Insights from these experiences can enhance our understanding for building implementation capacity and the rationales for evolving approaches that emphasize the dynamic, emotional, and highly relational nature of supporting others to use evidence in practice.

Assessing the fidelity of a behavioural intervention involving academic detailing in general practice: a sub-study of the 'Implementing work-related Mental health guidelines in general PRacticE' (IMPRovE) trial

BACKGROUND

Assessing the fidelity of intervention components enables researchers to make informed judgements about the influence of those components on the observed outcome. The 'Implementing work-related Mental health guidelines in general PRacticE' (IMPRovE) trial is a hybrid III trial aiming to increase adherence to the 'Clinical Guidelines for the diagnosis and management of work-related mental health conditions in general practice'. IMPRovE is a multifaceted intervention, with one of the central components being academic detailing (AD). This study describes the fidelity to the protocol for the AD component of the IMPRovE intervention.

METHOD

All AD sessions for the trial were audio-recorded and a sample of 22% were randomly selected for fidelity assessment. Fidelity was assessed using a tailored proforma based on the Modified Conceptual Framework for fidelity assessment, measuring duration, coverage, frequency and content. A descriptive analysis was used to quantify fidelity to the protocol and a content analysis was used to elucidate qualitative aspects of fidelity.

RESULTS

A total of eight AD sessions were included in the fidelity assessment. The average fidelity score was 89.2%, ranging from 80 to 100% across the eight sessions. The sessions were on average 47 min long and addressed all of the ten chapters in the guideline. Of the guideline chapters, 9 were frequently discussed. The least frequently discussed chapter related to management of comorbid conditions. Most general practitioner (GP) participants used the AD sessions to discuss challenges with managing secondary mental conditions. In line with the protocol, opinion leaders who delivered the AD sessions largely offered evidence-based strategies aligning with the clinical guideline recommendations.

CONCLUSIONS/IMPLICATIONS

The IMPRovE AD intervention component was delivered to high fidelity. The sessions adhered to the intended duration, coverage, frequency, and content allowing participating GPs to comprehend the implementation of the guideline in their own practice. This study also demonstrates that the Modified Conceptual Fidelity Framework with a mixed methods approach can support the assessment of implementation fidelity of a behavioural intervention in general practice. The findings enhance the trustworthiness of reported outcomes from IMPRovE and show that assessing fidelity is amenable for AD and should be incorporated in other studies using AD.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN 12620001163998, November 2020.

Results of a multi-site pragmatic hybrid type 3 cluster randomized trial comparing level of facilitation while implementing an intervention in community-dwelling disabled and older adults in a Medicaid waiver

Background

Evidence-based interventions that optimize physical function for disabled and older adults living in the community who have difficulty with daily living tasks are available. However, uptake has been limited, particularly in resource-constrained (Medicaid) settings. Facilitation may be an effective implementation strategy. This study’s aim was to compare internal facilitation (IF) versus IF and external facilitation (EF) on adoption and sustainability of an intervention in a Medicaid home and community-based waiver.

Methods

In a hybrid type 3 trial, waiver sites (N = 18) were randomly assigned to implement the intervention using a bundle of strategies with either IF or IF and EF. Adoption and sustainability were assessed via Stages of Implementation Completion (SIC) for each site. Clinician attitudes toward evidence-based practice and self-efficacy were evaluated among 539 registered nurses, social workers, and occupational therapists. Medicaid beneficiary outcomes of activities of daily living, depression, pain, falls, emergency department visits, and hospitalizations were evaluated in a sample of N = 7030 as reflected by electronic health records data of the Medicaid waiver program. Linear mixed-effects models were used to compare outcomes between trial arms while accounting for cluster-randomized design.

Results

The mean SIC scores were 72.22 (standard deviation [SD] = 16.98) in the IF arm (9 sites) and 61.33 (SD = 19.29) in the IF + EF arm (9 sites). The difference was not statistically significant but corresponded to the medium clinically important effect size Cohen’s d = 0.60. Clinician implementation outcomes of attitudes and self-efficacy did not differ by trial arm. Beneficiary depression was reduced significantly in the IF + EF arm compared to the IF arm (p = .04, 95% confidence interval for the difference [0.01, 0.24]). No differences between trial arms were found for other beneficiary outcomes.

Conclusions

Level of facilitation did not enhance capacity for adoption and sustainability of an evidence-based intervention in a Medicaid setting that cares for disabled and older adults. Improved beneficiary depression favored use of IF and EF compared to IF alone, and no differences were found for other outcomes. These findings also suggest level of facilitation may not have impacted beneficiary outcomes.

Trial registration

ClinicalTrials.gov, NCT03634033; date registered August 16, 2018.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01232-5.