Professionals' management of the fidelity-adaptation dilemma in the use of evidence-based interventions-an intervention study

An assessment of adaptation and fidelity in the implementation of an audit and feedback-based intervention to improve transition to adult type 1 diabetes care in Ontario, Canada

BACKGROUND

The fit between an intervention and its local context may affect its implementation and effectiveness. Researchers have stated that both fidelity (the degree to which an intervention is delivered, enacted, and received as intended) and adaptation to the local context are necessary for high-quality implementation. This study describes the implementation of an audit and feedback (AF)-based intervention to improve transition to type 1 diabetes adult care, at five sites, in terms of adaptation and fidelity.

METHODS

An audit and feedback (AF)-based intervention for healthcare teams to improve transition to adult care for patients with type 1 diabetes was studied at five pediatric sites. The Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS) was used to document the adaptations made during the study. Fidelity was determined on three different levels: delivery, enactment, and receipt.

RESULTS

Fidelity of delivery, receipt, and enactment were preserved during the implementation of the intervention. Of the five sites, three changed their chosen quality improvement initiative, however, within the parameters of the study protocol; therefore, fidelity was preserved while still enabling participants to adapt accordingly.

CONCLUSIONS

We describe implementing a multi-center AF-based intervention across five sites in Ontario to improve the transition from pediatric to adult diabetes care for youth with type 1 diabetes. This intervention adopted a balanced approach considering both adaptation and fidelity to foster a community of practice to facilitate implementing quality improvement initiatives for improving transition to adult diabetes care. This approach may be adapted for improving transition care for youth with other chronic conditions and to other complex AF-based interventions.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03781973. Registered 13 December 2018. Date of enrolment of the first participant to the trial: June 1, 2019.

Training-of-trainers program for community health workers involved in an innovative and community-based intervention against malaria among goldminers in the Guiana shield: a quality and effectiveness evaluation

Introduction

An innovative and community-based intervention is implemented in the Guiana Shield to eliminate malaria among people involved in artisanal and small-scale gold mining. The intervention consists of the distribution of malaria self-management kits to goldminers and the presumptive treatment for individuals at risk of carrying Plasmodium vivax hypnozoites. The intervention is possible owing to community health workers (CHWs) who are previously trained to master all intervention procedures, including health education activities and goldmining training. This study aimed to evaluate the training program provided to CHWs in terms of quality and effectiveness.

Methods

A training-of-trainers program for CHWs has been developed based on the CDC framework. A mixed-method case study was implemented in two steps between February and March 2023. The evaluation was based on a knowledge survey, satisfaction test, observations, and semi-structured interviews. Quantitative and qualitative data were analyzed and triangulated.

Results

A total of 20 CHWs participated in the training and the first-step evaluation. For the second step, four semi-structured interviews were conducted. The Qualitative data showed that group dynamics and adaptations were central elements of a high-quality training program. Quantitative analysis found that CHWs' satisfaction was elevated (> 4/5 overall), especially regarding format and learning results. Improvements in knowledge level demonstrated good effectiveness (pre-training vs. post-training, p < 0.05). Nevertheless, some difficulties persisted regarding tasks of the intervention procedure, such as informed consent and smartphone application procedures (with an inaccuracy rate of 29.2% and 16.7%, respectively). Further on-the-job training permitted to address these issues. The project team's previous experience and the Guiana Shield countries' commitment to the WHO-E-2025-initiative were identified as levers for the quality of the training, while the complexity of the project context was a challenge.

Discussion

High-quality, effective, and appropriate training programs are required for effective and sustainable interventions involving CHW profiles. Training design is a crucial point to address to accomplish quality and effectiveness. The training-of-trainers model has been shown to allow a high level of satisfaction, good learning results, and satisfactory implementation in the field. Initial and continuing training is an indispensable continuum to sustain good practices in the field and CHWs' motivation. Training evaluation permits standardizing methods and facilitates transferability.

Family-based Interventions of Preventing Substance Use Among Immigrant Youth: A Scoping Review

Introduction

Immigrant youth face heightened risks of substance use due to the stress associated with immigration and acculturation. While parental intervention can have a preventative impact on substance use, parents need to be well-informed about substance use and effective interventions that can prevent substance use among immigrant youth. Such interventions ought to be culturally sensitive, family-based, and targeted at the specific substances that are prevalent in a given context. Identifying and curating interventions that can empower parents in addressing substance use can help mitigate the risks that immigrant youth may face.

Methods

This scoping review aimed to identify the types, characteristics, and effectiveness of family-based substance use intervention programs. Based on Arksay and O'Malley's guidelines, interventions included in the review must have met the following criteria: (a) was a family-based intervention aiming to prevent substance use; (b) targeted immigrant teens aged 12 to 17 years old; (c) was published in English; (d) originated from Australia, Canada, New Zealand, or the United States. The pinch table was used to synthesize included articles, after which studies were compared and categorized, and cross-cutting categories were identified.

Results

After screening 4551 searched literature, 13 studies that utilized family-based interventions were included in the review. All interventions were face-to-face programs, and most interventions involved parents and youth as participants. Eco-developmental theory and active learning strategies were used by multiple interventions. Given immigrant families were target stakeholders, both deep structure and surface structure cultural adaptations were utilized. Interventions increased parents' knowledge and skills regarding substance use prevention and delayed substance use initiation among youth.

Conclusion

From the review, it was evident that parents are an essential element in any program aiming to prevent or reduce children's substance use. Besides information about substance use prevention, the curriculum also involves parenting and communication skills for parents to understand the protective effects of family. Effective family-based interventions for immigrant youth require attention to parenting and immigration stress, while also considering cultural adaptation. Future directions and limitations are also discussed.

The adaptation and fidelity tool to support social service practitioners in balancing fidelity and adaptations: Longitudinal, mixed-method evaluation study

Background

Evidence-based interventions (EBIs) seldom fit seamlessly into a setting and are often adapted. The literature identifies practitioners' management of fidelity and adaptations as problematic but offers little guidance. This study aimed to investigate practitioners' perceptions of the feasibility and usability of an intervention aimed to support them in fidelity and adaptation management when working with EBIs.

Methods

The intervention, the adaptation and fidelity tool (A-FiT), was developed based on the literature, along with input from social service practitioners and social services' Research and Development units' personnel. The intervention consisted of two workshops where the participants were guided through a five-step process to manage fidelity and adaptations. It was tested in a longitudinal mixed-method intervention study with 103 practitioners from 19 social service units in Stockholm, Sweden. A multimethod data collection was employed, which included interviews at follow-up, questionnaires at baseline and follow-up (readiness for change and self-rated knowledge), workshop evaluation questionnaires (usability and feasibility) after each workshop, and documentation (participants' notes on worksheets). To analyze the data, qualitative content analysis, Kruskal-Wallis tests, and Wilcoxon rank-sum tests were performed.

Results

Overall, the practitioners had a positive perception of the intervention and perceived it as relevant for fidelity and adaptation management (mean ratings over 7.0 on usability and feasibility). The workshops also provided new knowledge and skills to manage fidelity and adaptations. Furthermore, the intervention provided insights into the practitioners' understanding about adaptation and fidelity through a more reflective approach.

Conclusion

Practical tools are needed to guide professionals not only to adhere to intervention core elements but also to help them to manage fidelity and adaptation. The proposed A-FiT intervention for practitioners' management of both fidelity and adaptation is a novel contribution to the implementation literature. Potentially, the next step is an evaluation of the intervention's impact in an experimental design.

Engaging stakeholders to inform national implementation of critical time intervention in a program serving homeless-experienced Veterans

The Veterans Affairs (VA) Grant and Per Diem Case Management "Aftercare" program provides 6 months of case management for homeless-experienced Veterans (HEVs) transitioning to permanent housing, with the aim of decreasing returns to homelessness. Implementing Critical Time Intervention (CTI)-an evidence-based case management practice-would standardize care across the 128 community-based agencies that provide Aftercare services. To prepare for national CTI implementation in Aftercare, guided by Replicating Effective Programs (REP), we conducted a four-site pilot in which we adapted a CTI implementation package (training, technical assistance, and external facilitation); characterized stakeholder perspectives regarding the acceptability and appropriateness of this package; and identified contextual factors that affected CTI implementation. We engaged a stakeholder workgroup to tailor existing CTI training and technical assistance materials for Aftercare. To provide tailored support for providers and leaders to adopt and incorporate evidence-based practices (EBPs) into routine care, we also developed external facilitation materials and processes. Over 9 months, we implemented this package at four sites. We conducted semi-structured interviews at pre-implementation, mid-implementation, and 6 months post-implementation, with HEVs (n = 37), case managers (n = 16), supervisors (n = 10), and VA leaders (n = 4); these data were integrated with templated reflection notes from the project facilitator. We used rapid qualitative analysis and targeted coding to assess the acceptability and appropriateness of CTI and our implementation package and identify factors influencing CTI implementation. Stakeholders generally found CTI acceptable and appropriate; there was consensus that components of CTI were useful and compatible for this setting. To adapt our implementation package for scale-up, this pilot highlighted the value of robust and tangible CTI training and technical assistance-grounded in real-world cases-that highlights the congruence of CTI with relevant performance metrics. Variations in agency-level contextual factors may necessitate more intense and tailored supports to implement and sustain complex EBPs like CTI. Processes used in this pilot are relevant for implementing other EBPs in organizations that serve vulnerable populations. EBP scale-up and sustainment can be enhanced by engaging stakeholders to tailor EBPs for specific contexts; pilot testing and refining implementation packages for scale-up; and using qualitative methods to characterize contextual factors that affect EBP implementation.

A Slippery Slope When Using an Evidence-Based Intervention Out of Context. How Professionals Perceive and Navigate the Fidelity-Adaptation Dilemma-A Qualitative Study

Introduction

Adaptations are often necessary to effectively translate evidence-based interventions (EBI) between contexts, but compliance with the EBIs' core components is still important, which is referred to as the fidelity-adaptation dilemma. In the sustainment phase of implementation, it is the professionals delivering the EBIs who are tasked with the decision-making regarding adaptations, but the currently used models and frameworks mostly focus on the initial phases of implementation. To better understand and guide professionals in using EBIs, there is a need to explore professionals' perceptions of the fidelity-adaptation dilemma. The aim of this study is consequently to explore how professionals perceive and navigate the fidelity-adaptation dilemma when using an EBI out of context.

Materials and Methods

Semi-structured interviews were held with 19 psychologists working in primary care. The interviews concerned EBIs in general and Cool Kids, an evidence-based parenting education program designed for children with anxiety that is now used for children with lower levels of anxiety in another setting. The data were analyzed using an inductive content analysis method.

Results

The analysis resulted in two themes: My standpoint regarding fidelity and adaptation is clear and Managing fidelity and adaptations is complicated. The first theme summarizes the professionals' perceptions of confidence for either favoring fidelity or adaptations, as well as reasons for why they made adaptations. For the second theme, the professionals expressed concern about sometimes meeting difficulties with the dilemma when following their original inclination and having second thoughts about the impact the adaptations have in practice.

Conclusion

The professionals generally had strong preferences regarding fidelity and adaptations, but neither preference prevented them from facing difficulties with the dilemma. The results point to a need for better information about possible adaptations from developers but also better support and guidance for professionals when implementing EBIs to ensure quality implementation and facilitate implementation. The results of this study can inform the design of support for professionals in managing the dilemma.

Misunderstanding and Sensemaking Among Juvenile Probation Officers Working with Evidence-Based Practices

Evidence-based practices and programs (EBPs) have been adopted in juvenile probation agencies nationwide to maximize the number of successful probation cases. However, various pragmatic studies have found that JPOs are not yielding the expected benefits when compared to efficacy studies (Lipsey et al., 2010; Taxman & Belenko, 2011). Using focus group and survey data, the current study sought to increase our understanding of the gap between pragmatic and efficacy studies in juvenile probation settings by examining JPOs' perceptions and utilization of EBPs. The findings suggest that JPOs are misunderstanding how to use EBPs in daily practice, leaving them with negative perceptions of EBPs. Implications for improving JPO understanding and use of EBPs are discussed.