Methods for capturing and analyzing adaptations: implications for implementation research

Training-of-trainers program for community health workers involved in an innovative and community-based intervention against malaria among goldminers in the Guiana shield: a quality and effectiveness evaluation

Introduction

An innovative and community-based intervention is implemented in the Guiana Shield to eliminate malaria among people involved in artisanal and small-scale gold mining. The intervention consists of the distribution of malaria self-management kits to goldminers and the presumptive treatment for individuals at risk of carrying Plasmodium vivax hypnozoites. The intervention is possible owing to community health workers (CHWs) who are previously trained to master all intervention procedures, including health education activities and goldmining training. This study aimed to evaluate the training program provided to CHWs in terms of quality and effectiveness.

Methods

A training-of-trainers program for CHWs has been developed based on the CDC framework. A mixed-method case study was implemented in two steps between February and March 2023. The evaluation was based on a knowledge survey, satisfaction test, observations, and semi-structured interviews. Quantitative and qualitative data were analyzed and triangulated.

Results

A total of 20 CHWs participated in the training and the first-step evaluation. For the second step, four semi-structured interviews were conducted. The Qualitative data showed that group dynamics and adaptations were central elements of a high-quality training program. Quantitative analysis found that CHWs' satisfaction was elevated (> 4/5 overall), especially regarding format and learning results. Improvements in knowledge level demonstrated good effectiveness (pre-training vs. post-training, p < 0.05). Nevertheless, some difficulties persisted regarding tasks of the intervention procedure, such as informed consent and smartphone application procedures (with an inaccuracy rate of 29.2% and 16.7%, respectively). Further on-the-job training permitted to address these issues. The project team's previous experience and the Guiana Shield countries' commitment to the WHO-E-2025-initiative were identified as levers for the quality of the training, while the complexity of the project context was a challenge.

Discussion

High-quality, effective, and appropriate training programs are required for effective and sustainable interventions involving CHW profiles. Training design is a crucial point to address to accomplish quality and effectiveness. The training-of-trainers model has been shown to allow a high level of satisfaction, good learning results, and satisfactory implementation in the field. Initial and continuing training is an indispensable continuum to sustain good practices in the field and CHWs' motivation. Training evaluation permits standardizing methods and facilitates transferability.

Adapting and scaling a proven diabetes prevention program across 11 worksites in India: the INDIA-WORKS trial

BACKGROUND

Structured lifestyle change education reduces the burden of cardiometabolic diseases such as diabetes. Delivery of these programs at worksites could overcome barriers to program adoption and improve program sustainability and reach; however, tailoring to the worksite setting is essential.

METHODS

The Integrating Diabetes Prevention in Workplaces (INDIA-WORKS) study tested the implementation and effectiveness of a multi-level program for reducing cardiometabolic disease risk factors at 11 large and diverse worksites across India. Herein, we describe and classify program adaptations reported during in-depth interviews and focus group discussions with worksite managers, program staff, and peer educators involved in program delivery, and program participants and drop-outs. We used thematic analysis to identify key themes in the data and classified reported program adaptations using the FRAME classification system.

RESULTS

Adaptations were led by worksite managers, peer educators, and program staff members. They occurred both pre- and during program implementation and were both planned (proactive) and unplanned (proactive and reactive). The most frequently reported adaptations to the individual-level intervention were curriculum changes to tailor lessons to the local context, make the program more appealing to the workers at the site, or add a wider variety of exercise options. Other content adaptations included improvements to the screening protocol, intervention scheduling, and outreach plans to tailor participant recruitment and retention to the sites. Environment-level content adaptations included expanding or leveraging healthy food and exercise options at the worksites. Challenges to adaptation included scheduling and worksite-level challenges. Participants discussed the need to continue adapting the program in the future to continue making it relevant for worksite settings and engaging for employees.

CONCLUSION

This study describes and classifies site-specific modifications to a structured lifestyle change education program with worksite-wide health improvements in India. This adds to the literature on implementation adaptation in general and worksite wellness in India, a country with a large and growing workforce with, or at risk of, serious cardiometabolic diseases. This information is key for program scale-up, dissemination, and implementation in other settings.

TRIAL REGISTRATION

Clinicaltrials.gov NCT02813668. Registered June 27, 2016.

Adapting and scaling a proven diabetes prevention program across 11 worksites in India: the INDIA-WORKS trial

Background

Delivery of proven structured lifestyle change education for reducing the burden of cardiometabolic diseases such as diabetes at worksites could overcome barriers to program adoption and improve sustainability and reach of these programs; however, tailoring to the worksite setting is essential.

Methods

The Integrating Diabetes Prevention in Workplaces (INDIA-WORKS) study tested the implementation and effectiveness of a multi-level program for reducing cardiometabolic disease risk factors at eleven large and diverse worksites across India. Herein, we describe and classify program adaptations reported during in-depth interviews and focus group discussions with worksite managers, program staff, peer educators involved in program delivery, and program participants and drop-outs. We used thematic analysis to identify key themes in the data and classified reported program adaptations using the FRAME classification system.

Results

Adaptations were led by worksite managers, peer educators, and program staff members. They occurred both pre- and during program implementation and were both planned (proactive) and unplanned (proactive and reactive). The most frequently reported adaptations to the individual-level intervention were curriculum changes to tailor lessons to the local context, make the program more appealing to the workers at the site, or add exercise options. Other content adaptations included improvements to the screening protocol, intervention scheduling, and outreach plans to tailor participant recruitment and retention to the sites. Environment-level content adaptations included expanding or leveraging healthy food and exercise options at the worksites. Challenges to adaptation included scheduling and worksite-level challenges. Participants discussed the need to continue adapting the program in the future to continue making it relevant for worksite settings and engaging for employees.

Conclusion

This study describes and classifies site-specific modifications to a structured lifestyle change education program with worksite-wide health improvements in India. This adds to the literature on implementation adaptation in general and worksite wellness in India, a country with a large and growing workforce with, or at risk of, serious cardiometabolic diseases. This information is key for program scale-up, dissemination, and implementation in other settings.

Trial Registration

Clinicaltrial.gov NCT02813668, registered June 27, 2016.

Context matters-the phased development of an adaptable food literacy intervention: Up for Cooking

Adapting interventions to the context increases the impact and sustainability of interventions. Literature acknowledges the need to adapt existing interventions and that these adaptations should be clearly reported. However, little is known about how to incorporate adaptation from the beginning. This paper argues that interventions should be developed and adaptations should be made using an ongoing non-linear approach. An action-oriented research approach with feedback loops is proposed. We illustrate this with the development of a food literacy intervention 'Up for Cooking' (Dutch: Zin in Koken) and present lessons learned in developing, implementing and studying such adaptable interventions. Interventions should clearly define and differentiate the intervention function and form. Implementers, in turn, should be encouraged to tailor interventions within a form that fits with a specific context. Sufficient time, continuous adaptation based on co-creation, feedback loops and interdisciplinary collaboration are important prerequisites for the development of adaptable interventions.

Implementation of PM-JAY in India: a qualitative study exploring the role of competency, organizational and leadership drivers shaping early roll-out of publicly funded health insurance in three Indian states

BACKGROUND

The Pradhan Mantri Jan Arogya Yojana (PM-JAY), a publicly funded health insurance scheme, was launched in India in September 2018 to provide financial access to health services for poor Indians. PM-JAY design enables state-level program adaptations to facilitate implementation in a decentralized health implementation space. This study examines the competency, organizational, and leadership approaches affecting PM-JAY implementation in three contextually different Indian states.

METHODS

We used a framework on implementation drivers (competency, organizational, and leadership) to understand factors facilitating or hampering implementation experiences in three PM-JAY models: third-party administrator in Uttar Pradesh, insurance in Chhattisgarh, and hybrid in Tamil Nadu. We adopted a qualitative exploratory approach and conducted 92 interviews with national, state, district, and hospital stakeholders involved in program design and implementation in Delhi, three state capitals, and two anonymized districts in each state, between February and April 2019. We used a deductive approach to content analysis and interpreted coded material to identify linkages between organizational features, drivers, and contextual elements affecting implementation.

RESULTS AND CONCLUSION

PM-JAY guideline flexibilities enabled implementation in very different states through state-adapted implementation models. These models utilized contextually relevant adaptations for staff and facility competencies and organizational and facilitative administration, which had considerable scope for improvement in terms of recruitment, competency development, programmatic implementation support, and rationalizing the joint needs of the program and implementers. Adaptations also created structural barriers in staff interactions and challenged implicit power asymmetries and organizational culture, indicating a need for aligning staff hierarchies and incentive structures. At the same time, specific adaptations such as decentralizing staff selection and task shifting (all models); sharing of claims processing between the insurer and state agency (insurance and hybrid model); and using stringent empanelment, accreditation, monitoring, and benchmarking criteria for performance assessment, and reserving secondary care benefit packages for public hospitals (both in the hybrid model) contributed to successful implementation. Contextual elements such as institutional memory of previous schemes and underlying state capacities influenced all aspects of implementation, including leadership styles and autonomy. These variations make comparisons across models difficult, yet highlight constraints and opportunities for cross-learning and optimizing implementation to achieve universal health coverage in decentralized contexts.

Anticipating adaptation: tracking the impact of planned and unplanned adaptations during the implementation of a complex population-based genomic screening program

In 2021, the Medical University of South Carolina (MUSC) launched In Our DNA SC. This large-scale initiative will screen 100,000 individuals in South Carolina for three preventable hereditary conditions that impact approximately two million people in the USA but often go undetected. In anticipation of inevitable changes to the delivery of this complex initiative, we developed an approach to track and assess the impact of evaluate adaptations made during the pilot phase of program implementation. We used a modified version of the Framework for Reporting Adaptations and Modification-Enhanced (FRAME) and Adaptations to code adaptations made during the 3-month pilot phase of In Our DNA SC. Adaptations were documented in real-time using a REDCap database. We used segmented linear regression models to independently test three hypotheses about the impact of adaptations on program reach (rate of enrollment in the program, rate of messages viewed) and implementation (rate of samples collected) 7 days pre- and post-adaptation. Effectiveness was assessed using qualitative observations. Ten adaptations occurred during the pilot phase of program implementation. Most adaptations (60%) were designed to increase the number and type of patient contacted (reach). Adaptations were primarily made based on knowledge and experience (40%) or from quality improvement data (30%). Of the three adaptations designed to increase reach, shortening the recruitment message potential patients received significantly increased the average rate of invitations viewed by 7.3% (p = 0.0106). There was no effect of adaptations on implementation (number of DNA samples collected). Qualitative findings support improvement in effectiveness of the intervention after shortening the consent form and short-term positive impact on uptake of the intervention as measured by team member's participation. Our approach to tracking adaptations of In Our DNA SC allowed our team to quantify the utility of modifications, make decisions about pursuing the adaptation, and understand consequences of the change. Streamlining tools for tracking and responding to adaptations can help monitor the incremental impact of interventions to support continued learning and problem solving for complex interventions being delivered in health systems based on real-time data.

Adaptations to scale-up an early childhood education and care physical activity intervention for real-world availability - Play Active

BACKGROUND

Adaptations for scale-up are ubiquitous but are rarely described in detail. Adaptations may be a key reason for the "scale-up penalty" which is when there is a reduction in intervention effect size following scale-up. The Play Active intervention consists of a physical activity policy for early childhood education and care (ECEC) services, with accompanying implementation support strategies. It was first implemented with 81 ECEC services in Perth, Western Australia, in 2021 - with significant positive changes in physical activity practice uptake. The aim of this paper is to describe the extent, type, fidelity consistency, goals, size, scope, and proposed impact of proposed adaptations to the implementation support strategies for scaling-up Play Active.

METHODS

Proposed adaptations were defined as planned changes, made prior to making the intervention available. The authors created a list of adaptations from a comparison of the Play Active implementation support strategies, before and after adaptation for proposed statewide availability across Western Australia, Queensland and South Australia, Australia. We used the Framework for Reporting Adaptations and Modifications-Enhanced Implementation Strategies (FRAME-IS) to code adaptations to implementation support strategies. Three authors coded each adaptation and rated their size, scope and proposed impact.

RESULTS

Fifty-three adaptations to Play Active were identified. Most (68%) were proposed for the 'content' of implementation strategies, including aspects of their delivery. In practice, this involved changing the delivery mode of implementation support strategies from phone call and email support, to website-based delivery. More than half (56%) of adaptations involved 'adding elements' for scale-up. Most adaptations were 'fidelity consistent' (95%). The main goals for adaptations were related to 'increasing the acceptability, appropriateness, or feasibility' (45%), 'decreasing the costs' (19%) and 'increasing adoption of the evidence-based practice' (19%). Adaptations were small to medium in size, with most proposed to have a positive (87%) or neutral (8%) effect on the effectiveness of the intervention, rather than negative (4%).

CONCLUSIONS

A large number of small, fidelity-consistent, adaptations were proposed for Play Active scale-up. Overall, the process of reporting adaptations was found to be feasible. To understand the impact of these adaptations, it will be important to re-evaluate implementation, effectiveness and process outcomes, at-scale.

Delivering diabetes shared medical appointments in primary care: early and mid-program adaptations and implications for successful implementation

BACKGROUND

Self-management is essential for good outcomes in type 2 diabetes and patients often benefit from self-management education. Shared medical appointments (SMAs) can increase self-efficacy for self management but are difficult for some primary care practices to implement. Understanding how practices adapt processes and delivery of SMAs for patients with type 2 diabetes may provide helpful strategies for other practices interested in implementing SMAs.

METHODS

The Invested in Diabetes study was a pragmatic cluster-randomized, comparative effectiveness trial designed to compare two different models of diabetes SMAs delivered in primary care. We used a multi-method approach guided by the FRAME to assess practices' experience with implementation, including any planned and unplanned adaptations. Data sources included interviews, practice observations and field notes from practice facilitator check-ins.

RESULTS

Several findings were identified from the data: 1) Modifications and adaptations are common in implementation of SMAs, 2) while most adaptations were fidelity-consistent supporting the core components of the intervention conditions as designed, some were not, 3) Adaptations were perceived to be necessary to help SMAs meet patient and practice needs and overcome implementation challenges, and 4) Content changes in the sessions were often planned and enacted to better address the contextual circumstances such as patient needs and culture.

DISCUSSION

Implementing SMAs in primary care can be challenging and adaptations of both implementation processes and content and delivery of SMAS for patients with type 2 diabetes were common in the Invested in Diabetes study. Recognizing the need for adaptations based on practice context prior to implementation may help improve fit and success with SMAs, but care needs to be given to ensure that adaptations do not weaken the impact of the intervention. Practices may be able to assess what might need to adapted for them to be successful prior to implementation but likely will continue to adapt after implementation.

CONCLUSION

Adaptations were common in the Invested in Diabetes study. Practices may benefit from understanding common challenges in implementing SMAs and adapting processes and delivery based on their own context.

TRIAL REGISTRATION

This trial is registered on clinicaltrials.gov under Trial number NCT03590041, posted 18/07/2018.

Implementing clinical decision support for reducing women Veterans' cardiovascular risk in VA: A mixed-method, longitudinal study of context, adaptation, and uptake

Evaluations of clinical decision support (CDS) implementation often struggle to measure and explain heterogeneity in uptake over time and across settings, and to account for the impact of context and adaptation on implementation success. In 2017-2020, the EMPOWER QUERI implemented a cardiovascular toolkit using a computerized template aimed at reducing women Veterans' cardiovascular risk across five Veterans Healthcare Administration (VA) sites, using an enhanced Replicating Effective Programs (REP) implementation approach. In this study, we used longitudinal joint displays of qualitative and quantitative findings to explore (1) how contextual factors emerged across sites, (2) how the template and implementation strategies were adapted in response to contextual factors, and (3) how contextual factors and adaptations coincided with template uptake across sites and over time. We identified site structure, staffing changes, relational authority of champions, and external leadership as important contextual factors. These factors gave rise to adaptations such as splitting the template into multiple parts, pairing the template with a computerized reminder, conducting academic detailing, creating cheat sheets, and using small-scale pilot testing. All five sites exhibited variability in utilization over the months of implementation, though later sites exhibited higher template utilization immediately post-launch, possibly reflecting a "preloading" of adaptations from previous sites. These findings underscore the importance of adaptive approaches to implementation, with intentional shifts in intervention and strategy to meet the needs of individual sites, as well as the value of integrating mixed-method data sources in conducting longitudinal evaluation of implementation efforts.