Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care

Knowledge translation strategies used to promote evidence-based interventions for children with cerebral palsy: a scoping review

BACKGROUND

Cerebral palsy (CP) is the most common childhood physical disability, imposing substantial costs on individuals and society. Early interventions that promote brain optimization and reorganization are vital for children with CP. Integrating early evidence-based practice (EBP) remains challenging but enhances functional outcomes.

METHODS

Following a scoping review methodology, databases were searched to identify studies examining the impact of knowledge translation (KT) strategies for pediatric CP interventions. Extraction included study characteristics, methodology, KT strategies, barriers, and facilitators. Numerical and inductive content analysis identified themes among KT strategies. A final stakeholder consultation to discuss the results was conducted.

RESULTS

This review included seventeen articles. Common outcomes included participant change in EBP knowledge and behaviour. Common barriers included a need for more resources, protected time, and funding. Most studies followed a multifaceted KT approach. Various KT strategies were used, primarily mentoring, workshops, case studies, and online tools.

INTERPRETATION

Results underscored the need for tailored KT strategies for implementing EBP for children with CP. Additionally, user-friendly KT tools and involving mentors to facilitate the intervention can haste EBP uptake. Successful adoption depends on challenges in healthcare settings. This study provides insights into current KT strategies for advancing best practices for children with CP.

Fidelity of implementation of national guidelines on malaria diagnosis for children under-five years in Rivers State, Nigeria

BACKGROUND

Malaria is still a disease of global public health importance and children under-five years of age are the most vulnerable to the disease. Nigeria adopted the "test and treat" strategy in the national malaria guidelines as one of the ways to control malaria transmission. The level of adherence to the guidelines is an important indicator for the success or failure of the country's roadmap to malaria elimination by 2030. This study aimed to assess the fidelity of implementation of the national guidelines on malaria diagnosis for children under-five years and examine its associated moderating factors in health care facilities in Rivers State, Nigeria.

METHODS

This was a descriptive, cross-sectional study conducted in Port Harcourt metropolis. Data were collected from 147 public, formal private and informal private health care facilities. The study used a questionnaire developed based on Carroll's Conceptual Framework for Implementation Fidelity. Frequency, mean and median scores for implementation fidelity and its associated factors were calculated. Associations between fidelity and the measured predictors were examined using Mann Whitney U test, Kruskal Wallis test, and multiple linear regression modelling using robust estimation of errors. Regression results are presented in adjusted coefficient (β) and 95% confidence intervals.

RESULTS

The median (IQR) score fidelity score for all participants was 65% (43.3, 85). Informal private facilities (proprietary patent medicine vendors) had the lowest fidelity scores (47%) compared to formal private (69%) and public health facilities (79%). Intervention complexity had a statistically significant inverse relationship to implementation fidelity (β = - 1.89 [- 3.42, - 0.34]). Increase in participant responsiveness (β = 8.57 [4.83, 12.32]) and the type of malaria test offered at the facility (e.g., RDT vs. no test, β = 16.90 [6.78, 27.03]; microscopy vs. no test, β = 21.88 [13.60, 30.16]) were positively associated with fidelity score.

CONCLUSIONS

This study showed that core elements of the "test and treat" strategy, such as testing all suspected cases with approved diagnostic methods before treatment, are still not fully implemented by health facilities. There is a need for strategies to increase fidelity, especially in the informal private health sector, for malaria elimination programme outcomes to be achieved.

Assessing the implementation fidelity, feasibility, and sustainability of community-based house improvement for malaria control in southern Malawi: a mixed-methods study

BACKGROUND

Despite significant success in the fight against malaria over the past two decades, malaria control programmes rely on only two insecticidal methods: indoor residual spraying and insecticidal-treated nets. House improvement (HI) can complement these interventions by reducing human-mosquito contact, thereby reinforcing the gains in disease reduction. This study assessed the implementation fidelity, which is the assessment of how closely an intervention aligns with its intended design, feasibility, and sustainability of community-led HI in southern Malawi.

METHODS

The study, conducted in 22 villages (2730 households), employed a mixed-methods approach. Implementation fidelity was assessed using a modified framework, with longitudinal surveys collecting data on HI coverage indicators. Quantitative analysis, employing descriptive statistics, evaluated the adherence to HI implementation. Qualitative data came from in-depth interviews, key informant interviews, and focus groups involving project beneficiaries and implementers. Qualitative data were analysed using content analysis guided by the implementation fidelity model to explore facilitators, challenges, and factors affecting intervention feasibility.

RESULTS

The results show that HI was implemented as planned. There was good adherence to the intended community-led HI design; however, the adherence could have been higher but gradually declined over time. In terms of intervention implementation, 74% of houses had attempted to have eaves closed in 2016-17 and 2017-18, compared to 70% in 2018-19. In 2016-17, 42% of houses had all four sides of the eaves closed, compared to 33% in 2018-19. Approximately 72% of houses were screened with gauze wire in 2016-17, compared to 57% in 2018-19. High costs, supply shortages, labour demands, volunteers' poor living conditions and adverse weather were reported to hinder the ideal HI implementation. Overall, the community described community-led HI as feasible and could be sustained by addressing these socioeconomic and contextual challenges.

CONCLUSION

Our study found that although HI was initially implemented as planned, its fidelity declined over time. Using trained volunteers facilitated the fidelity and feasibility of implementing the intervention. A combination of rigorous community education, consistent training, information, education and communication, and intervention modifications may be necessary to address the challenges and enhance the intervention's fidelity, feasibility, and sustainability.

Using CFIR framework for understanding barriers and facilitators to implementation of community tuberculosis program in Burkina Faso

Introduction

In 2020, there were nearly 9.9 million new Tuberculosis cases and 1.3 million deaths, with about 95% occurring in developing nations. Burkina Faso implemented a community Tuberculosis program, involving Civil Society Organizations, to increase screening and improve treatment outcomes. Therefore, this study aims to identify the factors influencing the implementation of community interventions involving these organizations in the fight against TB in Burkina Faso.

Method

This qualitative study conducted semi-structured key informant interviews with a purposive sample of health providers from the ministry of health and community health workers. We used framework (the consolidated framework for implementation research was used method to identify barriers and facilitators to implementation of community tuberculosis program in Burkina Faso.

Results

A total of 23 interviews were conducted. The results of this research shed light on several key factors that either contributed to or hindered the program's success. Among the facilitating factors, we identified close collaboration between national and international stakeholders, as well as remarkable program flexibility to adapt to local conditions. Furthermore, continuous training and support for community health workers proved crucial for the program's implementation. However, significant challenges were also unveiled. These challenges encompassed insufficient financial resources, difficulties related to the recruitment and management of civil society associations, and issues regarding program ownership at the peripheral level. Additionally, irregular payments to community health workers had a detrimental impact on their motivation and commitment.

Conclusions

Our study conducted a comprehensive examination of the obstacles and facilitators encountered in the implementation of a community-based tuberculosis control program in Burkina Faso. The results of this research shed light on several key factors that either contributed to or hindered the success implementation of program. Measures should be taken to mobilize national resources, strengthen the capacities of associations, and promote local ownership of the program. Special attention should also be given to improving financial management and resolving issues related to the recruitment and compensation of community health workers. For such community-based tuberculosis programs to succeed in Burkina Faso and in similar context it is essential to address these obstacles and facilitators.

Implementing Dementia Caregiver Programs in Real-World Settings: Fidelity Considerations

Testing interventions in real-world settings requires fidelity monitoring to ensure implementation integrity. However, strategies to enhance, monitor, and measure fidelity deployed in efficacy trials may not be feasible in pragmatic trials or sustainable in practice. This paper reviews published translational or pragmatic studies of dementia caregiver support interventions to understand how fidelity was previously treated in order to derive recommendations for future pragmatic-like trials. A search using SCOPUS, EMBASE, and Google Scholar identified 31 translational caregiver intervention studies of which 20 (64.5%) referenced fidelity. Of these 20, 11 (55.0%) reported fidelity measurement, whereas 9 (45.0%) only recognized its importance. Of the 11 studies, fidelity was assessed using investigator-developed scoring forms, audio/video recordings, evaluations from caregivers and interventionists, and by comparing outcomes with the original efficacy trial. Additionally, 7 (63.6%) of 11 studies reported fidelity results, representing only 22.5% of 31 studies reporting outcomes demonstrating the inconsistency in the field concerning the reporting outcomes of fidelity. We conclude that fidelity methods used in translational studies to date are not practical nor sustainable for ongoing monitoring of evidence-based programs in real-world settings and that only 2 aspects of fidelity, intervention and adherence, are considered. New approaches are needed to ensure fidelity integrity in pragmatic trials and which can be sustained thereafter.

Identifying the Active Ingredients of a Computerized Speech and Language Therapy Intervention for Poststroke Aphasia: Multiple Methods Investigation Alongside a Randomized Controlled Trial

BACKGROUND

Aphasia is a communication disorder affecting more than one-third of stroke survivors. Computerized Speech and Language Therapy (CSLT) is a complex intervention requiring computer software, speech and language therapists, volunteers, or therapy assistants, as well as self-managed practice from the person with aphasia. CSLT was found to improve word finding, a common symptom of aphasia, in a multicenter randomized controlled trial (Clinical and Cost Effectiveness of Computer Treatment for Aphasia Post Stroke [Big CACTUS]).

OBJECTIVE

This study provides a detailed description of the CSLT intervention delivered in the Big CACTUS trial and identified the active ingredients of the intervention directly associated with improved word finding for people with aphasia.

METHODS

We conducted a multiple methods study within the context of a randomized controlled trial. In study 1, qualitative interviews explored key informants' understanding of the CSLT intervention, how the components interacted, and how they could be measured. Qualitative data were transcribed verbatim and analyzed thematically. Qualitative findings informed the process measures collected as part of a process evaluation of the CSLT intervention delivered in the Big CACTUS trial. In study 2, quantitative analyses explored the relationship between intervention process measures (length of computer therapy access; therapists' knowledge of CSLT; degree of rationale for CSLT tailoring; and time spent using the software to practice cued confrontation naming, noncued naming, and using words in functional sentences) and change in word-finding ability over a 6-month intervention period.

RESULTS

Qualitative interviews were conducted with 7 CSLT approach experts. Thematic analysis identified four overarching components of the CSLT approach: (1) the StepByStep software (version 5; Steps Consulting Ltd), (2) therapy setup: tailoring and personalizing, (3) regular independent practice, and (4) support and monitoring. Quantitative analyses included process and outcome data from 83 participants randomized to the intervention arm of the Big CACTUS trial. The process measures found to be directly associated with improved word-finding ability were therapists providing a thorough rationale for tailoring the computerized therapy exercises and the amount of time the person with aphasia spent using the computer software to practice using words in functional sentences.

CONCLUSIONS

The qualitative exploration of the CSLT approach provided a detailed description of the components, theories, and mechanisms underpinning the intervention and facilitated the identification of process measures to be collected in the Big CACTUS trial. Quantitative analysis furthered our understanding of which components of the intervention are associated with clinical improvement. To optimize the benefits of using the CSLT approach for word finding, therapists are advised to pay particular attention to the active ingredients of the intervention: tailoring the therapy exercises based on the individual's specific language difficulties and encouraging people with aphasia to practice the exercises focused on saying words in functional sentences.

TRIAL REGISTRATION

ISRCTN Registry ISRCTN68798818; https://www.isrctn.com/ISRCTN68798818.

Evaluating an intervention to improve the safety and experience of transitions from hospital to home for older people (Your Care Needs You): a protocol for a cluster randomised controlled trial and process evaluation

BACKGROUND

Older patients often experience safety issues when transitioning from hospital to home. The 'Your Care Needs You' (YCNY) intervention aims to support older people to 'know more' and 'do more' whilst in hospital so that they are better prepared for managing at home.

METHODS

A multi-centre cluster randomised controlled trial (cRCT) will evaluate the effectiveness and cost-effectiveness of the YCNY intervention. Forty acute hospital wards (clusters) in England from varying medical specialities will be randomised to deliver YCNY or care-as-usual on a 1:1 basis. The primary outcome will be unplanned hospital readmission rates within 30 days of discharge. This will be extracted from routinely collected data of at least 5440 patients (aged 75 years and older) discharged to their own homes during the 4- to 5-month YCNY intervention period. A nested cohort of up to 1000 patients will be recruited to the study to collect secondary outcomes via follow-up questionnaires at 5-, 30- and 90-day post-discharge. These will include measures of patient experience of transitions, patient-reported safety events, quality of life and healthcare resource use. Unplanned hospital readmission rates at 60 and 90 days of discharge will be collected from routine data. A process evaluation (primarily interviews and observations with patients, carers and staff) will be conducted to understand the implementation of the intervention and the contextual factors that shape this, as well as the intervention's underlying mechanisms of action. Fidelity of intervention delivery will also be assessed across all intervention wards.

DISCUSSION

This study will establish the effectiveness and cost-effectiveness of the YCNY intervention which aims to improve patient safety and experience for older people during transitions of care. The process evaluation will generate insights about how the YCNY intervention was implemented, what elements of the intervention work and for whom, and how to optimise its implementation so that it can be delivered with high fidelity in routine service contexts.

TRIAL REGISTRATION

UK Clinical Research Network Portfolio: 44559; ISTCRN: ISRCTN17062524. Registered on 11/02/2020.

Implementation of a multicomponent family support intervention in adult intensive care units: study protocol for an embedded mixed-methods multiple case study (FICUS implementation study)

BACKGROUND

The implementation of complex interventions is considered challenging, particularly in multi-site clinical trials and dynamic clinical settings. This study protocol is part of the family intensive care units (FICUS) hybrid effectiveness-implementation study. It aims to understand the integration of a multicomponent family support intervention in the real-world context of adult intensive care units (ICUs). Specifically, the study will assess implementation processes and outcomes of the study intervention, including fidelity, and will enable explanation of the clinical effectiveness outcomes of the trial.

METHODS AND ANALYSIS

This mixed-methods multiple case study is guided by two implementation theories, the Normalisation Process Theory and the Consolidated Framework for Implementation Research. Participants are key clinical partners and healthcare professionals of eight ICUs allocated to the intervention group of the FICUS trial in the German-speaking part of Switzerland. Data will be collected at four timepoints over the 18-month active implementation and delivery phase using qualitative (small group interviews, observation, focus group interviews) and quantitative data collection methods (surveys, logs). Descriptive statistics and parametric and non-parametric tests will be used according to data distribution to analyse within and between cluster differences, similarities and factors associated with fidelity and the level of integration over time. Qualitative data will be analysed using a pragmatic rapid analysis approach and content analysis.

ETHICS AND DISSEMINATION

Ethics approval was obtained from the Cantonal Ethics Committee of Zurich BASEC ID 2021-02300 (8 February 2022). Study findings will provide insights into implementation and its contribution to intervention outcomes, enabling understanding of the usefulness of applied implementation strategies and highlighting main barriers that need to be addressed for scaling the intervention to other healthcare contexts. Findings will be disseminated in peer-reviewed journals and conferences.

PROTOCOL REGISTRATION NUMBER

Open science framework (OSF) https://osf.io/8t2ud Registered on 21 December 2022.

Utilizing a mixed-methods approach to assess implementation fidelity of a group antenatal care trial in Rwanda

BACKGROUND

The Preterm Birth Initiative (PTBi)-Rwanda conducted a cluster randomized controlled trial to assess the impact of group antenatal care (group ANC) on preterm birth, using a group ANC approach adapted for the Rwanda setting, and implemented in 18 health centers. Previous research showed high overall fidelity of implementation, but lacked correlation with provider self-assessment and left unanswered questions. This study utilizes a mixed-methods approach to study the fidelity with which the health centers' implementation followed the model specified for group ANC.

METHODS

Implementation fidelity was measured using two tools, repeated Model Fidelity Assessments (MFAs) and Activity Reports (ARs) completed by Master Trainers, who visited each health center between 7 and 13 times (9 on average) to provide monitoring and training over 18 months between 2017 and 2019. Each center's MFA item and overall scores were regressed (linear regression) on the time elapsed since the center's start of implementation. The Activity Report (AR) is an open-ended template to record comments on implementation. For the qualitative analysis, the ARs from the times of each center's highest and lowest MFA score were analyzed using thematic analysis. Coding was conducted via Dedoose, with two coders independently reviewing and coding transcripts, followed by joint consensus coding.

RESULTS

A total of 160 MFA reports were included in the analysis. There was a significant positive association between elapsed time since a health center started implementation and greater implementation fidelity (as measured by MFA scores). In the qualitative AR analysis, Master Trainers identified key areas to improve fidelity of implementation, including: group ANC scheduling, preparing the room for group ANC sessions, provider capacity to co-facilitate group ANC, and facilitator knowledge and skills regarding group ANC content and process. These results reveal that monitoring visits are an important part of acquisition and fidelity of the "soft skills" required to effectively implement group ANC and provide an understanding of the elements that may have impacted fidelity as described by Master Trainers.

CONCLUSIONS

For interventions like Group ANC, where "soft-skills" like group facilitation are important, we recommend continuous monitoring and mentoring throughout program implementation to strengthen these new skills, provide corrective feedback and guard against skills decay. We suggest the use of quantitative tools to provide direct measures of implementation fidelity over time and qualitative tools to gain a more complete understanding of what factors influence implementation fidelity. Identifying areas of implementation requiring additional support and mentoring may ensure effective translation of evidence-based interventions into real-world settings.

Fidelity to the ACT SMART Toolkit: an instrumental case study of implementation strategy fidelity

BACKGROUND

Evidence-based practices (EBPs) are shown to improve a variety of outcomes for autistic children. However, EBPs often are mis-implemented or not implemented in community-based settings where many autistic children receive usual care services. A blended implementation process and capacity-building implementation strategy, developed to facilitate the adoption and implementation of EBPs for autism spectrum disorder (ASD) in community-based settings, is the Autism Community Toolkit: Systems to Measure and Adopt Research-based Treatments (ACT SMART Toolkit). Based on an adapted Exploration, Adoption decision, Preparation, Implementation, Sustainment (EPIS) Framework, the multi-phased ACT SMART Toolkit is comprised of (a) implementation facilitation, (b) agency-based implementation teams, and (c) a web-based interface. In this instrumental case study, we developed and utilized a method to evaluate fidelity to the ACT SMART Toolkit. This study responds to the need for implementation strategy fidelity evaluation methods and may provide evidence supporting the use of the ACT SMART Toolkit.

METHODS

We used an instrumental case study approach to assess fidelity to the ACT SMART Toolkit during its pilot study with six ASD community agencies located in southern California. We assessed adherence, dose, and implementation team responsiveness for each phase and activity of the toolkit at both an aggregate and individual agency level.

RESULTS

Overall, we found that adherence, dose, and implementation team responsiveness to the ACT SMART Toolkit were high, with some variability by EPIS phase and specific activity as well as by ASD community agency. At the aggregate level, adherence and dose were rated notably lowest during the preparation phase of the toolkit, which is a more activity-intensive phase of the toolkit.

CONCLUSIONS

This evaluation of fidelity to the ACT SMART Toolkit, utilizing an instrumental case study design, demonstrated the potential for the strategy to be used with fidelity in ASD community-based agencies. Findings related to the variability of implementation strategy fidelity in the present study may also inform future adaptations to the toolkit and point to broader trends of how implementation strategy fidelity may vary by content and context.

Implementation fidelity to a behavioral diabetes prevention intervention in two New York City safety net primary care practices

BACKGROUND

It is critical to assess implementation fidelity of evidence-based interventions and factors moderating fidelity, to understand the reasons for their success or failure. However, fidelity and fidelity moderators are seldom systematically reported. The study objective was to conduct a concurrent implementation fidelity evaluation and examine fidelity moderators of CHORD (Community Health Outreach to Reduce Diabetes), a pragmatic, cluster-randomized, controlled trial to test the impact of a Community Health Workers (CHW)-led health coaching intervention to prevent incident type 2 Diabetes Mellitus in New York (NY).

METHODS

We applied the Conceptual Framework for Implementation Fidelity to assess implementation fidelity and factors moderating it across the four core intervention components: patient goal setting, education topic coaching, primary care (PC) visits, and referrals to address social determinants of health (SDH), using descriptive statistics and regression models. PC patients with prediabetes receiving care from safety-net patient-centered medical homes (PCMHs) at either, VA NY Harbor or at Bellevue Hospital (BH) were eligible to be randomized into the CHW-led CHORD intervention or usual care. Among 559 patients randomized and enrolled in the intervention group, 79.4% completed the intake survey and were included in the analytic sample for fidelity assessment. Fidelity was measured as coverage, content adherence and frequency of each core component, and the moderators assessed were implementation site and patient activation measure.

RESULTS

Content adherence was high for three components with nearly 80.0% of patients setting ≥ 1 goal, having ≥ 1 PC visit and receiving ≥ 1 education session. Only 45.0% patients received ≥ 1 SDH referral. After adjusting for patient gender, language, race, ethnicity, and age, the implementation site moderated adherence to goal setting (77.4% BH vs. 87.7% VA), educational coaching (78.9% BH vs. 88.3% VA), number of successful CHW-patient encounters (6 BH vs 4 VA) and percent of patients receiving all four components (41.1% BH vs. 25.7% VA).

CONCLUSIONS

The fidelity to the four CHORD intervention components differed between the two implementation sites, demonstrating the challenges in implementing complex evidence-based interventions in different settings. Our findings underscore the importance of measuring implementation fidelity in contextualizing the outcomes of randomized trials of complex multi-site behavioral interventions.

TRIAL REGISTRATION

The trial was registered with ClinicalTrials.gov on 30/12/2016 and the registration number is NCT03006666 .

Evaluating the implementation fidelity to a successful nurse-led model (INTERCARE) which reduced nursing home unplanned hospitalisations

BACKGROUND

Implementation fidelity assesses the degree to which an intervention is delivered as it should be. Fidelity helps to determine if the outcome(s) of an intervention are attributed to the intervention itself or to a failure of its implementation. Little is known about how fidelity impacts the intended outcome(s) and what elements or moderators can affect the fidelity trajectory over time. We exemplify the meaning of implementation fidelity with INTERCARE, a nurse-led care model that was implemented in eleven Swiss nursing homes (NHs) and showed effectiveness in reducing unplanned hospital transfers. INTERCARE comprises six core elements, including advance care planning and tools to support inter- and interprofessional communication, which were introduced with carefully developed implementation strategies.

METHODS

A mixed-methods convergent/triangulation design was used to investigate the influence of implementation fidelity on unplanned transfers. A fidelity questionnaire measuring the degree of fidelity to INTERCARE's core components was fielded at four time points in the participating NHs. Two-monthly meetings were conducted with NHs (September 2018-January 2020) and structured notes were used to determine moderators affecting fidelity (e.g., participant responsiveness). We used the fidelity scores and generalized linear mixed models to analyze the quantitative data. The Framework method was used for the qualitative analysis. The quantitative and qualitative findings were integrated using triangulation.

RESULTS

A higher overall fidelity score showed a decreasing rate of unplanned hospital transfers post-intervention (OR: 0.65 (CI = 0.43-0.99), p = 0.047). A higher fidelity score to advance care planning was associated with lower unplanned transfers (OR = 0.24 (CI 0.13-0.44), p = < 0.001) and a lower fidelity score for communication tools (e.g., ISBAR) to higher rates in unplanned transfers (OR = 1.69 (CI 1.30-2.19), p = < 0.003). In-house physicians with a collaborative approach and staff's perceived need for nurses working in extended roles, were important moderators to achieve and sustain high fidelity.

CONCLUSION

Implementation fidelity is challenging to measure and report, especially in complex interventions, yet is crucial to better understand how such interventions may be tailored for scale-up. This study provides both a detailed description of how fidelity can be measured and which ingredients highly contributed to reducing unplanned NH transfers.

TRIAL REGISTRATION

The INTERCARE study was registered at clinicaltrials.gov Protocol Record NCT03590470.

Assessing the implementation fidelity of HPV self-collection offered by community health workers during home visits (the EMA strategy): a case study in a low-middle-resource setting in Argentina

BACKGROUND

In Argentina, HPV  self-collection offered by community health workers was demonstrated to be effective to improve cervical cancer screening uptake. Based on these findings, the EMA strategy was scaled up in nine Argentinian provinces. However, there is no evidence about the degree of fidelity-in relation to the core components proposed by the National Program on Cervical Cancer Prevention-with which this strategy was implemented in the new jurisdictions. We carried out a fidelity evaluation of the EMA strategy scaling-up aimed at evaluating the level of adherence to the core components of the EMA strategy, and how different moderating factors affected the implementation fidelity.

METHODS

This descriptive study used a multi-method approach involving quantitative and qualitative evaluations of the implementation fidelity using the Conceptual Framework for Implementation Fidelity. Evaluation of the degree of adherence to the core components of the EMA strategy was carried out through the analysis of a self-administered survey of health promoters, observations, and secondary data from the National Screening Information System. The analysis of moderating factors was carried out through analysis of field notes, and semi-structured interviews with key stakeholders.

RESULTS

Our results showed that the core components with highest fidelity were training, sample handling, and transportation. Regarding the offer of HPV self-collection, we found some adaptations such as locations in which health promoters offered HPV self-collection, and fewer pieces of information provided to women during the offer. In the follow-up and treatment core component, we found a reduced adherence to triage and colposcopy. Some contextual factors had a negative impact on implementation fidelity, such as urban insecurity and the reduction in the number of health promoters that offered HPV self-collection. Moderating factors that contributed to achieve high level of fidelity included a well-defined strategy with clear steps to follow, permanent feedback and high level of engagement among implementers.

CONCLUSIONS

Our study shows how the analysis of fidelity and adaptations of HPV self-collection in real-world contexts are key to measure and maximize its effectiveness in low-middle-income settings.

The mechanics of implementation strategies and measures: advancing the study of implementation mechanisms

BACKGROUND

There is a fundamental gap in understanding the causal mechanisms by which strategies for implementing evidence-based practices address local barriers to effective, appropriate service delivery. Until this gap is addressed, scientific knowledge and practical guidance about which implementation strategies to use in which contexts will remain elusive. This research project aims to identify plausible strategy-mechanism linkages, develop causal models for mechanism evaluation, produce measures needed to evaluate such linkages, and make these models, methods, and measures available in a user-friendly website. The specific aims are as follows: (1) build a database of strategy-mechanism linkages and associated causal pathway diagrams, (2) develop psychometrically strong, pragmatic measures of mechanisms, and (3) develop and disseminate a website of implementation mechanisms knowledge for use by diverse stakeholders.

METHODS

For the first aim, a combination of qualitative inquiry, expert panel methods, and causal pathway diagramming will be used to identify and confirm plausible strategy-mechanism linkages and articulate moderators, preconditions, and proximal and distal outcomes associated with those linkages. For the second aim, rapid-cycle measure development and testing methods will be employed to create reliable, valid, pragmatic measures of six mechanisms of common strategies for which no high-quality measures exist. For the third aim, we will develop a user-friendly website and searchable database that incorporates user-centered design, disseminating the final product using social marketing principles.

DISCUSSION

Once strategy-mechanism linkages are identified using this multi-method approach, implementation scientists can use the searchable database to develop tailored implementation strategies and generate more robust evidence about which strategies work best in which contexts. Moreover, practitioners will be better able to select implementation strategies to address their specific implementation problems. New horizons in implementation strategy development, optimization, evaluation, and deployment are expected to be more attainable as a result of this research, which will lead to enhanced implementation of evidence-based interventions for cancer control, and ultimately improvements in patient outcomes.

Developing a method to assess fidelity to a complex vocational rehabilitation intervention in the FRESH trial: a feasibility study

Background

Determining whether complex rehabilitation interventions are delivered with fidelity is important. Implementation fidelity can differ between sites, therapists delivering interventions and, over time, threatening trial outcomes and increasing the risk of type II and III errors. This study aimed to develop a method of assessing occupational therapists’ fidelity to deliver a complex, individually tailored vocational rehabilitation (VR) intervention to people with traumatic brain injury (TBI) and assess the feasibility of its use in a randomised controlled trial.

Methods

Using mixed methods and drawing on the intervention logic model, we developed data collection tools to measure fidelity to early specialist TBI VR (ESTVR). Fidelity was measured quantitatively using intervention case report forms (CRF), fidelity checklists and clinical records. Qualitative data from mentoring records, interviews with intervention therapists, participants with TBI, employers and NHS staff at trial sites explored moderators of implementation fidelity. The conceptual framework of implementation fidelity (CFIF) guided measurement and analysis of and factors affecting fidelity. Data were triangulated and benchmarked against an earlier cohort study.

Results

Fidelity to a complex individually tailored VR intervention could be measured. Overall, OTs delivered ESTVR with fidelity. Different fidelity measures answered different questions, offering unique insights into fidelity. Fidelity was best assessed using a fidelity checklist, intervention CRFs and clinical notes. The OT clinical notes and mentoring records were best at identifying fidelity moderating factors. Interviews added little insight into fidelity moderating factors over and above mentoring or clinical records. Data triangulation offered a comprehensive assessment of fidelity, highlighting limitations of measurement methods and learning for future trials but was resource intensive. Interviews, fidelity visits and analysing clinical notes were also resource intense. Comparing fidelity data to a benchmark and using CFIF as a framework for organising the fidelity assessment helped.

Conclusions

OTs delivered the VR intervention with fidelity. A fidelity checklist and benchmark plus mentoring may offer a practical and effective way of measuring fidelity and identifying fidelity moderating factors in trials of complex individually-tailored rehabilitation interventions. Mentoring provided real-time indicators of and reasons for fidelity deviations. These methods require further evaluation.

Trial registration

ISRCTN Registry, ISRCTN38581822 (Registered: 02/01/2014).

Supplementary Information

The online version contains supplementary material available at 10.1186/s40814-022-01111-2.

Evolutionary game model of health care and social care collaborative services for the elderly population in China

INTRODUCTION

The synergy of health care and elderly social care organizations has become the focus of the research on integrated health care and social care. This study aims to propose a collaborative strategy among health care and elderly social care service providers.

METHODS

An evolutionary game model is applied for performance analysis and optimization of the cooperation between health care and elderly social care organizations. The behavioural strategies and the impact of key parameters on promoting the cooperation of the players are presented in detail.

RESULTS

Simulation experiments and sensitivity analysis results indicate that (1) the behavioural evolution of health care organizations and elderly social care organizations forms three types of integrated health care and social care services, namely, the bilateral cooperation type, health care organization-led type and elderly social care organization-led type. (2) Increasing the additional benefits for cooperation and reducing the additional costs for cooperation can promote the willingness to synergize to provide integrated health care and elderly social care services. At the early stage of evolution, increasing the costs that elderly social care organizations pay to purchase health care services or pay for negotiation in the bilateral cooperation type can provide incentives for health care organizations to cooperate while reducing the cooperation preferences of elderly social care organizations. However, the long-term impact of the costs on the behavioural strategies for cooperation of the two players cannot be determined.

CONCLUSION

The behavioural decisions on cooperation between health care and elderly social care organizations influence each other; commitment to integration and effective collaboration can be achieved by increasing the additional benefits and reducing the marginal costs. The findings suggest that the political-economic context and government policies have a greater influence on promoting cooperation, thus yielding positive or negative results for integrated care practice.

The unappreciated relevance of auxiliary assumptions for evaluating theory-based interventions in health psychology

The use of theory in health behavior change interventions has been recently questioned with mixed results found for theory-based intervention effectiveness. But theory testing in intervention depends on not only theoretical assumptions, but on auxiliary assumptions too. Specifically, auxiliary assumptions are required to traverse the distance from nonobservational terms in theories and observational terms at the level of the empirical hypotheses in interventions. We believe intervention failures are often due to flaws in auxiliary assumptions rather than assumptions at the theoretical level. We use the theory of planned behavior to illustrate how the consideration of these auxiliary assumptions is important to appraise the effectiveness of interventions informed by theory. We hope that bringing attention to the importance of auxiliary assumptions provides a more nuanced and accurate appraisal of theory utility.

Implementation and modification of an organizational-level intervention: a prospective analysis

Background

Modifications to interventions can jeopardize intervention outcomes. Pre-existing perceived barriers and facilitators to the intervention arising in the implementation preparation phase may help explain why modifications to the intervention may occur during the implementation phase. This two-site comparative case study describes modifications made to a complex organizational-level intervention and examines how known implementation science factors may have enabled such changes to occur.

Methods

Northwestern Medicine’s^TM Hispanic Kidney Transplant Program (HKTP) is a culturally competent transplant center-based intervention designed to reduce disparities in living donor kidney transplantation among Hispanics. In-depth qualitative interviews and discussions were longitudinally conducted with transplant stakeholders (i.e., physicians, administrators, clinicians) at two kidney transplant programs with large Hispanic populations during implementation preparation and implementation phases. The Consolidated Framework for Implementation Research (CFIR) guided interview design and qualitative analysis, and Stirman’s Framework for Reporting Adaptations and Modifications-Expanded (FRAME) guided modification classification.

Results

Across sites, 57 stakeholders participated in an interview, group discussion, and/or learning collaborative discussion. Site-B made more modifications than Site-A (n = 29 versus n = 18). Sites differed in the proportions of delaying/skipping (Site-A 50% versus Site-B 28%) and adding (Site-A 11% versus Site-B 28%) but had comparable substituting (Site-A 17% versus Site-B 17%) and tweaking (Site-A 17% versus Site-B 14%) modification types. Across sites, the transplant team consistently initiated the most modifications (Site-A 66%; Site-B 62%). While individuals initiated slightly more modifications at Site-B (21% versus Site-A 17%), institutions instigated proportionately slightly more modifications at Site-A (17% versus Site-B 10%). CFIR inner setting factors (i.e., structural characteristics, culture, available resources, implementation climate) that prominently emerged during the implementation preparation phase explained similarities and differences in sites’ modification numbers, types, and agents in the implementation phase.

Conclusion

Organizations implementing a culturally competent care intervention made modifications. CFIR inner setting factors emerging in the implementation preparation phase largely explained similarities and differences in study sites’ modifications. Identifying factors contributing to modifications may help institutions become better prepared to implement an intervention by addressing known factors in advance, which may foster greater fidelity leading to desired outcomes.

Trial registration

ClinicalTrials.govNCT03276390. We registered the study retrospectively on 9-7-17.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00296-0.

Feasibility and implementation fidelity of a co-designed intervention to promote in-hospital mobility among older medical patients-the WALK-Copenhagen project (WALK-Cph)

BACKGROUND

Mobility interventions can prevent functional decline among older patients, but implementation of such interventions may be complicated by barriers in the clinical setting. The WALK-Copenhagen project (WALK-Cph) is aimed at promoting a 24-h mobility among older medical patients during hospitalization. The WALK-Cph intervention was co-designed by researchers and stakeholders to tailor the intervention to the clinical context. The aim of this study was to investigate the feasibility and implementation fidelity of the WALK-Cph intervention before evaluating clinical effectiveness in a randomized controlled trial (ClinicalTrials.gov NCT03825497).

METHODS

The WALK-Cph intervention consisted of six components: a welcome folder explaining the importance of in-hospital activity, a WALK-plan prescribing up to three daily walking sessions during and after hospitalization, a WALK-path in the hallway that patients were motivated to use daily, exercise posters in the hallways and bedrooms, self-service on beverages and clothes, and discharge with a WALK-plan. The present study reports on phase 2 of WALK-Cph and consists of a feasibility and a fidelity component. The study was conducted at the two WALK-Cph intervention departments after the initiation of the WALK-Cph intervention. A cohort of older medical patients (+65) was recruited for the feasibility study to assess recruitment and data collection procedures and the method for assessment of activity. Simultaneously, implementation fidelity was assessed by observing clinical practice and intervention delivery at the intervention departments.

RESULTS

A feasibility cohort of 48 patients was included. Inclusion was considered feasible with recruitment rates between 62% and 70% of all eligible patients. Also, data collection was conducted without obstacles, and all patients accepted to wear activity monitors. The fidelity observations showed that three of the six intervention components were partially implemented as planned whereas three components were not implemented as planned.

CONCLUSION

The WALK-Cph intervention was found feasible, and although the intervention was not implemented with fidelity, the level of fidelity was considered sufficient to continue with further testing of the WALK-Cph intervention in a large-scale trial.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03825497 (retrospectively registered). Protocol PubMed ID (PMID): 29523569.

Translation of the geriatric emergency department intervention into other emergency departments: a post implementation evaluation of outcomes for older adults

Background

Increasing numbers of older adults with complex health deficits presenting to emergency departments has prompted the development of innovative models of care. One such model designed to reduce poor outcomes associated with acute healthcare, is the Geriatric Emergency Department Intervention. This intervention is a nurse-led, physician-championed, Emergency Department intervention that improves the health outcomes for frail older adults in the emergency department.

Methods

This quantitative cohort study aimed to evaluate the healthcare outcomes and costs associated with the implementation of the Geriatric Emergency Department Intervention (GEDI) for adults aged 70 years and over at two hospital sites that implemented the model using the integrated-Promoting Action on Research Implementation in Health Services (i-PARHIS) framework. Hospital A was large teaching hospital located in the tropical north of Australia. Hospital B was a medium sized teaching hospital near Brisbane, Queensland Australia.

The effect of the intervention was examined in two ways. Outcomes were compared between: 1) all patients in the pre- and post- implementation periods, and 2) patients seen or not seen by the Geriatric Emergency Department Intervention team in the post-implementation period. The outcomes measured were disposition (discharged home, admitted); emergency department length of stay; hospital length of stay; all cause in-hospital mortality within 28 days; time to re-presentation up to 28 days post-discharge; emergency department and in-hospital costs.

Survival analyses were used for the primary and secondary outcome variables and a Cox survival model was used to estimate the associations between variables and outcomes. Multiple regression models were used to examine other secondary outcomes whilst controlling for a range of confounders.

Results

The Geriatric Emergency Department Intervention was successfully translated into two different emergency departments. Both demonstrated an increased likelihood of discharge, decreased emergency department length of stay, decreased hospital costs for those who were admitted, with an associated reduction in risk of mortality, for adults aged 70 years and over.

Conclusions

The Geriatric Emergency Department Intervention was successfully translated into new sites that adapted the model design. Improvement in healthcare outcomes for older adults presenting to the emergency department was demonstrated, although this was more subtle than in the original model setting.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-022-02999-4.

A cluster randomised controlled trial of a ward-based intervention to improve access to psychologically-informed care and psychological therapy for mental health in-patients

BACKGROUND

There is good evidence that psychological interventions improve patient well-being and independent living, but patients on acute mental health wards often do not have access to evidence-based psychological therapies which are strongly advised by NICE guidance for severe mental health problems. The overall aim of this programme of work is to increase patient access to psychological therapies on acute mental health inpatient wards. Stage one of the programme (which is complete) aimed to identify barriers and facilitators to delivering therapy in these settings through a large qualitative study. The key output of stage one was an intervention protocol that is designed to be delivered on acute wards to increase patient access to psychologically-informed care and therapy. Stage two of the programme aims to test the effects of the intervention on patient wellbeing and serious incidents on the ward (primary outcomes), patient social functioning and symptoms, staff burnout, ward atmosphere from staff and patient perspectives and cost effectiveness of the intervention (secondary outcomes).

METHODS

The study is a single blind, pragmatic, cluster randomised controlled trial and will recruit thirty-four wards across England that will be randomised to receive the new intervention plus treatment as usual, or treatment as usual only. Primary and secondary outcomes will be assessed at baseline and 6-month and 9-month follow-ups, with serious incidents on the ward collected at an additional 3-month follow-up.

DISCUSSION

The key output will be a potentially effective and cost-effective ward-based psychological intervention that increases patient access to psychological therapy in inpatient settings, is feasible to deliver in inpatient settings and is acceptable to patients.

TRIAL REGISTRATION NUMBER

ClinicalTrials.gov Identifier: NCT03950388. Registered 15^th May 2019. https://clinicaltrials.gov/ct2/show/NCT03950388.

Measuring implementation fidelity in a cluster-randomized pragmatic trial: development and use of a quantitative multi-component approach

BACKGROUND

In pragmatic trials, on-site partners, rather than researchers, lead intervention delivery, which may result in implementation variation. There is a need to quantitatively measure this variation. Applying the Framework for Implementation Fidelity (FIF), we develop an approach for measuring variability in site-level implementation fidelity. This approach is then applied to measure site-level fidelity in a cluster-randomized pragmatic trial of Music & MemorySM (M&M), a personalized music intervention targeting agitated behaviors in residents living with dementia, in US nursing homes (NHs).

METHODS

Intervention NHs (N = 27) implemented M&M using a standardized manual, utilizing provided staff trainings and iPods for participating residents. Quantitative implementation data, including iPod metadata (i.e., song title, duration, number of plays), were collected during baseline, 4-month, and 8-month site visits. Three researchers developed four FIF adherence dimension scores. For Details of Content, we independently reviewed the implementation manual and reached consensus on six core M&M components. Coverage was the total number of residents exposed to the music at each NH. Frequency was the percent of participating residents in each NH exposed to M&M at least weekly. Duration was the median minutes of music received per resident day exposed. Data elements were scaled and summed to generate dimension-level NH scores, which were then summed to create a Composite adherence score. NHs were grouped by tercile (low-, medium-, high-fidelity).

RESULTS

The 27 NHs differed in size, resident composition, and publicly reported quality rating. The Composite score demonstrated significant variation across NHs, ranging from 4.0 to 12.0 [8.0, standard deviation (SD) 2.1]. Scaled dimension scores were significantly correlated with the Composite score. However, dimension scores were not highly correlated with each other; for example, the correlation of the Details of Content score with Coverage was τb = 0.11 (p = 0.59) and with Duration was τb = - 0.05 (p = 0.78). The Composite score correlated with CMS quality star rating and presence of an Alzheimer's unit, suggesting face validity.

CONCLUSIONS

Guided by the FIF, we developed and used an approach to quantitatively measure overall site-level fidelity in a multi-site pragmatic trial. Future pragmatic trials, particularly in the long-term care environment, may benefit from this approach.

TRIAL REGISTRATION

Clinicaltrials.gov NCT03821844. Registered on 30 January 2019, https://clinicaltrials.gov/ct2/show/NCT03821844 .

Outcomes and Critical Factors for Successful Implementation of Organizational Health Literacy Interventions: A Scoping Review

Organizational health literacy (OHL)-interventions can reduce inequality and demands in health care encountered by patients. However, an overview of their impact and critical factors for organization-wide implementation is lacking. The aim of this scoping review is to summarize the evidence on: (1) the outcomes of OHL-interventions at patient, professional and organizational levels; and (2) the factors and strategies that affect implementation and outcomes of OHL-interventions. We reviewed empirical studies following the five-stage framework of Arksey and O'Malley. The databases Scopus, PubMed, PsychInfo and CINAHL were searched from 1 January 2010 to 31 December 2019, focusing on OHL-interventions using terms related to "health literacy", "health care organization" and "intervention characteristics". After a full-text review, we selected 24 descriptive stu-dies. Of these, 23 studies reported health literacy problems in relation to OHL-assessment tools. Nine out of thirteen studies reported that the use of interventions resulted in positive changes on OHL-domains regarding comprehensible communication, professionals' competencies and practices, and strategic organizational changes. Organization-wide OHL-interventions resulted in some improvement of patient outcomes but evidence was scarce. Critical factors for organization-wide implementation of OHL-interventions were leadership support, top-down and bottom-up approaches, a change champion, and staff commitment. Organization-wide interventions lead to more positive change on OHL-domains, but evidence regarding OHL-outcomes needs strengthening.

Fidelity of implementation of TB screening guidelines by health providers at selected HIV clinics in Ghana

Introduction

Tuberculosis screening of people living with human immunodeficiency virus is an intervention recommended by the WHO to control the dual epidemic of TB and HIV. The extent to which the intervention is adhered to by the HIV healthcare providers (fidelity) determines the intervention’s effectiveness as measured by patient outcomes, but literature on fidelity is scarce. This study assessed provider implementation fidelity to national guidelines on TB screening at HIV clinics in Ghana.

Methods

It was a cross-sectional study that used structured questionnaires to gather data, involving 226 of 243 HIV healthcare providers in 27 HIV clinics across Ghana. The overall fidelity score comprised sixteen items with a maximum score of 48 grouped into three components of the screening intervention (TB diagnosis, TB awareness and TB symptoms questionnaire). Simple summation of item scores was done to determine fidelity score per provider. In this paper, we define the level of fidelity as low if the scores were below the median score and were otherwise categorized as high. Background factors potentially associated with implementation fidelity level were assessed using cluster-based logistic regression. Odds ratio with 95% confidence interval (CI) was used as the measure of association.

Results

Of the 226 healthcare providers interviewed, 60% (135) were females with a mean age of 34.5 years (SD = 8.3). Most of them were clinicians [63% (142)] and had post-secondary non-tertiary education [62% (141)]. Overall, 53% (119) of the healthcare providers were categorized to have implemented the intervention with high fidelity. Also, 56% (126), 53% (120), and 59% (134) of the providers implemented the TB diagnosis, TB awareness and TB symptoms questionnaire components respectively with high fidelity. After adjusting for cluster effect, female providers (AOR = 2.36, 95%CI: 1.09–5.10, p = <0.029), those with tertiary education (AOR = 4.31, 95%CI: 2.12–9.10, p = 0.040), and clinicians (AOR = 1.78, 95%CI: 1.07–3.50, p = 0.045) were more likely to adhere to the guidelines compared to their counterparts.

Conclusion

The number of providers with fidelity scores above the median was marginally greater (6%) than the number with fidelity score below the median. Similarly, for each of the components, the number of providers with fidelity scores higher than the median was marginally higher. This could explain the existing fluctuations in the intervention outcomes in Ghana. We found gender, profession and education were associated with provider implementation fidelity. To improve fidelity level among HIV healthcare providers, and realize the aims of the TB screening intervention among PLHIV in Ghana, further training on implementing all components of the intervention is critical.

Integrating pharmacists into aged care facilities to improve the quality use of medicine (PiRACF Study): protocol for a cluster randomised controlled trial

BACKGROUND

Medication management in residential aged care facilities is an ongoing concern. Numerous studies have reported high rates of inappropriate prescribing and medication use in aged care facilities, which contribute to residents' adverse health outcomes. There is a need for new models of care that enhance inter-disciplinary collaboration between residential aged care facility staff and healthcare professionals, to improve medication management. Pilot research has demonstrated the feasibility and benefits of integrating a pharmacist into the aged care facility team to improve the quality use of medicines. This protocol describes the design and methods for a cluster randomised controlled trial to evaluate the outcomes and conduct economic evaluation of a service model where on-site pharmacists are integrated into residential aged care facility healthcare teams to improve medication management.

METHODS

Intervention aged care facilities will employ on-site pharmacists to work as part of their healthcare teams 2 to 2.5 days per week for 12 months. On-site pharmacists, in collaboration with facility nurses, prescribers, community pharmacists, residents and families will conduct medication management activities to improve the quality use of medicines. Aged care facilities in the control group will continue usual care. The target sample size is 1188 residents from a minimum of 13 aged care facilities. The primary outcome is the appropriateness of prescribing, measured by the proportion of residents who are prescribed at least one potentially inappropriate medicine according to the 2019 Beers Criteria. Secondary outcomes include hospital and emergency department presentations, fall rates, prevalence and dose of antipsychotics and benzodiazepines, Anticholinergic Cognitive Burden Score, staff influenza vaccination rate, time spent on medication rounds, appropriateness of dose form modification and completeness of resident's allergy and adverse drug reaction documentation. A cost-consequence and cost-effectiveness analysis will be embedded in the trial.

DISCUSSION

The results of this study will provide information on clinical and economic outcomes of a model that integrates on-site pharmacists into Australian residential aged care facilities. The results will provide policymakers with recommendations relevant to further implementation of this model.

TRIAL REGISTRATION

ACTRN12620000430932 . Registered on 1 April 2020 with ANZCTR.

Implementation fidelity of the Systems for Person-Centered Elder Care (SPEC): a process evaluation study

BACKGROUND

The Systems for Person-Centered Elder Care (SPEC), a complex intervention, was conducted to examine its effectiveness as a technology-enhanced, multidisciplinary, and integrated care model for frail older persons among ten nursing homes (NHs) in South Korea where formal long-term care has recently been introduced. The purpose of this study was to evaluate the implementation fidelity of the SPEC intervention and to identify moderating factors that influence the implementation fidelity.

METHODS

This study was a process evaluation based on an evidence-based framework for implementation fidelity using a mixed-methods design. Quantitative data from consultant logbooks, NH documentations, an information and communications technology (ICT) system, and a standardized questionnaire were collected from April 2015 to December 2016 and analyzed by calculating the descriptive statistics. Semi-structured focus group interviews were held with multidisciplinary teams from the participating NHs. Qualitative data from a semi-structured questionnaire and the focus group interviews were analyzed using content analysis.

RESULTS

The SPEC program demonstrated good implementation fidelity, and adherence to the SPEC program was strong in all aspects, such as content, coverage, frequency, and duration. Of the participating on-site coordinators, 60% reported that the SPEC model positively impacted needs assessment and the reporting system for resident care. The important facilitating factors were tailored facilitating strategies, assurance of the quality of delivery, and recruitment strategies.

CONCLUSION

The effectiveness of the SPEC program was driven by good implementation fidelity. The key factors of good implementation fidelity were tailored delivery of evidence-based interventions over process evaluation work, facilitating strategies, and ICT support. Larger implementation studies with a more user-friendly ICT system are recommended.

TRIAL REGISTRATION

ISRCTN registry, ISRCTN11972147 . Registered on 16 March 2015.

What matters when exploring fidelity when using health IT to reduce disparities?

BACKGROUND

Implementation of evidence-based interventions often involves strategies to engage diverse populations while also attempting to maintain external validity. When using health IT tools to deliver patient-centered health messages, systems-level requirements are often at odds with 'on-the ground' tailoring approaches for patient-centered care or ensuring equity among linguistically diverse populations.

METHODS

We conducted a fidelity and acceptability-focused evaluation of the STAR MAMA Program, a 5-month bilingual (English and Spanish) intervention for reducing diabetes risk factors among 181 post-partum women with recent gestational diabetes. The study's purpose was to explore fidelity to pre-determined 'core' (e.g. systems integration) and 'modifiable' equity components (e.g. health coaching responsiveness, and variation by language) using an adapted implementation fidelity framework. Participant-level surveys, systems-level databases of message delivery, call completion, and coaching notes were included.

RESULTS

96.6% of participants are Latina and 80.9% were born outside the US. Among those receiving the STAR MAMA intervention; 55 received the calls in Spanish (61%) and 35 English (39%). 90% (n = 81) completed ≥ one week. Initially, systems errors were common, and increased triggers for health coach call-backs. Although Spanish speakers had more triggers over the intervention period, the difference was not statistically significant. Of the calls triggering a health coach follow-up, attempts were made for 85.4% (n = 152) of the English call triggers and for 80.0% (n = 279) of the Spanish call triggers (NS). Of attempted calls, health coaching calls were complete for 55.6% (n = 85) of English-language call triggers and for 56.6% of Spanish-language call triggers (NS). Some differences in acceptability were noted by language, with Spanish-speakers reporting higher satisfaction with prevention content (p = < 0.01) and English-speakers reporting health coaches were less considerate of their time (p = 0.03).

CONCLUSIONS

By exploring fidelity by language-specific factors, we identified important differences in some but not all equity indicators, with early systems errors quicky remedied and high overall engagement and acceptability. Practice implications include: (1) establishing criteria for languge-equity in interventions, (2) planning for systems level errors so as to reduce their impact between language groups and over time; and (3) examining the impact of engagement with language-concordant interventions on outcomes, including acceptability. Trial Registration National Clinical Trials registration number: CT02240420 Registered September 15, 2014. ClinicalTrials.gov.

Evaluation of intervention fidelity of a complex psychosocial intervention Lifestyle Matters: a randomised controlled trial

Objectives

Robust research of complex interventions designed to promote mental well-being in later life is required to inform service development. An essential component is ensuring that such interventions are delivered as intended. We present a detailed description of the design and implementation of a fidelity assessment within a trial of one such intervention (Lifestyle Matters). The findings help to explain the trial results and also inform the design of embedded fidelity assessments within future evaluations of complex interventions.

Design

We conducted a mixed-method fidelity assessment embedded as part of a multicentre pragmatic randomised controlled trial. A conceptual fidelity framework was developed from the Behaviour Change Consortium framework. From this the fidelity assessment was designed. The resulting instrument assessed the following parameters: intervention design, training, supervision; and delivery, receipt and enactment of the intervention.

Intervention

The Lifestyle Matters intervention was designed to assist older people to improve and sustain mental well-being through participation in meaningful activity. The aim is to enable participants to engage in both new and neglected activities through a mix of facilitated group meetings and individual sessions.

Results

The fidelity assessment demonstrated that the intervention was delivered as per protocol for the group component and was tailored to meet individual needs. There was substantial inter-rater agreement for training; and group member performance 0.72; and moderate agreement for facilitator performance 0.55. It was not possible to determine whether small declines seen in facilitator performance were due to facilitator drift or moderating factors such as group dynamics or participant characteristics.

Conclusions

The assessment methods adequately measured criteria identified as being significant indicators of fidelity. Adherence during training, delivery and supervision was good. The subjective nature of identification and rating observed behaviours was the main challenge. Future research should explore alternative methods of assessing fidelity in trials of complex interventions.

Trial registration number

ISRCTN67209155.

Preconditions to implementation of an integrated care process programme

Purpose

The purpose of this study was to investigate the preconditions of a full-scale implementation of an integrated care process programme for frail older people from the staff's understanding, commitment and ability to change their work procedures with comparisons over time and between organisations.

Design/methodology/approach

A repeated cross-sectional study was conducted in a hospital, municipal health and social care setting.

Findings

Staff commitment decreased to the importance of a permanent municipal contact from baseline compared to the 12-months follow-up (p = 0.02) and the six- and 12-months follow-up (p = 0.05), to the information transfer from emergency department from the six- to the 12-months follow-up (p = 0.04), to discharge planning at the hospital at six- and 12-months follow-up (p = 0.04) and towards discharge planning at home from baseline to the six-month follow-up (p = 0.04). Significant differences occurred between the organisations about information transfer from the emergency department (p = 0.01) and discharge planning at home (p = 0.03). The hospital staff were the most committed.

Practical implications

The results can guide the implementation of complex interventions in organisations with high-employee turnover and heavy workload.

Originality/value

The study design, allowing the comparison of implementation results over time and between organisations in a later phase, gives this study a unique perspective.

Therapist competence and clinical outcome in the rehabilitation of memory following traumatic brain injury trial

Objectives: Therapist competencies when delivering an intervention can impact clinical outcomes. However, the relationship between competence and outcome in the context of memory rehabilitation has not previously been investigated. We aimed to evaluate whether therapist competencies in delivering a group-based memory rehabilitation intervention, as part of a randomized-controlled trial, was associated with changes in clinical outcome in the participants with traumatic brain injury.Method: Outcome data on the Everyday Memory Questionnaire (EMQ) and Goal Attainment Scaling (GAS) were obtained for 24 participants of the memory rehabilitation trial. Video recordings of intervention sessions delivered by six therapists were coded using a checklist designed to evaluate therapist competence in group facilitation.Results: Therapists demonstrated consistently high levels of competency in facilitating the memory group. Correlational analyses showed that there was a non-significant weak association between EMQ and therapist competence (r = .28, p = .18), and between GAS and therapist competence for short- and long-term goals (r = -.33, p = .12 and r = -.24, p = .25, respectively).Conclusions: We found no significant association between competence and outcome in this memory rehabilitation trial. This may reflect insufficient variability in therapist competence and/or the nature of the clinical outcome measures used. Further research is needed to understand competence-outcome relationships in cognitive rehabilitation.

Developing and testing a principle-based fidelity index for peer support in mental health services

PURPOSE

Evidence suggests that the distinctive relational qualities of peer support-compared to clinical-patient relationships-can be eroded in regulated healthcare environments. Measurement of fidelity in trials of peer support is lacking. This paper reports the development and testing of a fidelity index for one-to-one peer support in mental health services, designed to assess fidelity to principles that characterise the distinctiveness of peer support.

METHODS

A draft index was developed using expert panels of service user researchers and people doing peer support, informed by an evidence-based, peer support principles framework. Two rounds of testing took place in 24 mental health services providing peer support in a range of settings. Fidelity was assessed through interviews with peer workers, their supervisors and people receiving peer support. Responses were tested for spread and internal consistency, independently double rated for inter-rater reliability, with feedback from interviewees and service user researchers used to refine the index.

RESULTS

A fidelity index for one-to-one peer support in mental health services was produced with good psychometric properties. Fidelity is assessed in four principle-based domains; building trusting relationships based on shared lived experience; reciprocity and mutuality; leadership, choice and control; building strengths and making connections to community.

CONCLUSIONS

The index offers potential to improve the evidence base for peer support in mental health services, enabling future trials to assess fidelity of interventions to peer support principles, and service providers a means of ensuring that peer support retains its distinctive qualities as it is introduced into mental health services.

Implementation fidelity and acceptability of an intervention to improve vaccination uptake and child health in rural India: a mixed methods evaluation of a pilot cluster randomized controlled trial

Background

The Tika Vaani intervention, an initiative to improve basic health knowledge and empower beneficiaries to improve vaccination uptake and child health for underserved rural populations in India, was assessed in a pilot cluster randomized trial. The intervention was delivered through two strategies: mHealth (using mobile phones to send vaccination reminders and audio-based messages) and community mobilization (face-to-face meetings) in rural Indian villages from January to September 2018. We assessed acceptability and implementation fidelity to determine whether the intervention delivered in the pilot trial can be implemented at a larger scale.

Methods

We adapted the Conceptual Framework for implementation fidelity to assess acceptability and fidelity of the pilot interventions using a mixed methods design. Quantitative data sources include a structured checklist, household surveys, and mobile phone call patterns. Qualitative data came from field observations, intervention records, semi-structured interviews and focus groups with project recipients and implementers. Quantitative analyses assessed whether activities were implemented as planned, using descriptive statistics to describe participant characteristics and the percentage distribution of activities. Qualitative data were analyzed using content analysis and in the light of the implementation fidelity model to explore moderating factors and to determine how well the intervention was received.

Results

Findings demonstrated high (86.7%) implementation fidelity. A total of 94% of the target population benefited from the intervention by participating in a face-to-face group meeting or via mobile phone. The participants felt that the strategies were useful means for obtaining information. The clarity of the intervention theory, the motivation, and commitment of the implementers as well as the periodic meetings of the supervisors largely explain the high level of fidelity obtained. Geographic distance, access to a mobile phone, level of education, and gender norms are contextual factors that contributed to heterogeneity in participation.

Conclusions

Although the intervention was evaluated in the context of a randomized trial that could explain the high level of fidelity obtained, this evaluation provides confirmatory evidence that the results of the study reflect the underlying theory. The mobile platform coupled with community mobilization was well-received by the participants and could be a useful way to improve health knowledge and change behavior.

Trial registration

ISRCTN 44840759 (22 April 2018)

Measuring Implementation Strategy Fidelity in HealthyHearts NYC: A Complex Intervention Using Practice Facilitation in Primary Care

Few studies have assessed the fidelity of practice facilitation (PF) as an implementation strategy, and none have used an a priori definition or conceptual framework of fidelity to guide fidelity assessment. The authors adapted the Conceptual Framework for Implementation Fidelity to guide fidelity assessment in HealthyHearts NYC, an intervention that used PF to improve adoption of cardiovascular disease evidence-based guidelines in primary care practices. Data from a web-based tracking system of 257 practices measured fidelity using 4 categories: frequency, duration, content, and coverage. Almost all (94.2%) practices received at least the required 13 PF visits. Facilitators spent on average 26.3 hours at each site. Most practices (95.7%) completed all Task List items, and 71.2% were educated on all Chronic Care Model strategies. The majority (65.8%) received full coverage. This study provides a model that practice managers and implementers can use to evaluate fidelity of PF, and potentially other implementation strategies.

Implementation of a complex intervention to improve participation in older people with joint contractures living in nursing homes: a process evaluation of a cluster-randomised pilot trial

Background

Joint contractures in frail older people are associated with serious restrictions in participation. We developed the Participation Enabling CAre in Nursing (PECAN) intervention, a complex intervention to enable nurses to promote participation in nursing home residents with joint contractures. The aim of this study was to examine the feasibility of the implementation strategy and to identify enablers and barriers for a successful implementation.

Methods

The implementation of PECAN was investigated in a 6-month pilot cluster-randomised controlled trial (c-RCT). As a key component of the implementation strategy, nominated nurses were trained as facilitators in a one-day workshop and supported by peer-mentoring (visit, telephone counselling). A mixed-methods approach was conducted in conjunction with the pilot trial and guided by a framework for process evaluations of c-RCTs. Data were collected using standardised questionnaires (nursing staff), documentation forms, problem-centred qualitative interviews (facilitators, therapists, social workers, relatives, peer-mentors), and a group discussion (facilitators). A set of predefined criteria on the nursing home level was examined. Quantitative data were analysed using descriptive statistics. Qualitative data were analysed using directed content analysis.

Results

Seven nursing homes (n = 4 intervention groups, n = 3 control groups) in two regions of Germany took part in the study. Facilitators responded well to the qualification measures (workshop participation: 14/14; workshop rating: “good”; peer-mentor visit participation: 10/14). The usage of peer-mentoring via telephone varied (one to seven contacts per nursing home). Our implementation strategy was not successful in connection with supplying the intervention to all the nurses. The clear commitment of the entire nursing home and the respect for the expertise of different healthcare professionals were emphasised as enablers, whereas a lack of impact on organisational conditions and routines and a lack of time and staff competence were mentioned as barriers.

Conclusion

The PECAN intervention was delivered as planned to the facilitators but was unable to produce comprehensive changes in the nursing homes and subsequently for the residents. Strategies to systematically include the management and the nursing team from the beginning are needed to support the facilitators during implementation in the main trial.

Trial registration

German clinical trials register, DRKS00010037. Registered 12 February 2016.

A mixed methods approach to exploring the moderating factors of implementation fidelity of the integrated chronic disease management model in South Africa

Background

Chronic care models like the Integrated Chronic Disease Management (ICDM) model strive to improve the efficiency and quality of care for patients with chronic diseases. However, there is a dearth of studies assessing the moderating factors of fidelity during the implementation of the ICDM model. The aim of this study is to assess moderating factors of implementation fidelity of the ICDM model.

Methods

This was a cross-sectional mixed method study conducted in two health districts in South Africa. The process evaluation and implementation fidelity frameworks were used to guide the assessment of moderating factors influencing implementation fidelity of the ICDM model. We interviewed 30 purposively selected healthcare workers from four facilities (15 from each of the two facilities with lower and higher levels of implementation fidelity of the ICDM model). Data on facility characteristics were collected by observation and interviews. Linear regression and descriptive statistics were used to analyse quantitative data while qualitative data were analysed thematically.

Results

The median age of participants was 36.5 (IQR: 30.8–45.5) years, and they had been in their roles for a median of 4.0 (IQR: 1.0–7.3) years. The moderating factors of implementation fidelity of the ICDM model were the existence of facilitation strategies (training and clinical mentorship); intervention complexity (healthcare worker, time and space integration); and participant responsiveness (observing operational efficiencies, compliance of patients and staff attitudes). One feature of the ICDM model that seemingly compromised fidelity was the inclusion of tuberculosis patients in the same stream (waiting areas, consultation rooms) as other patients with non-communicable diseases and those with HIV/AIDS with no clear infection control guidelines. Participants also suggested that poor adherence to any one component of the ICDM model affected the implementation of the other components. Contextual factors that affected fidelity included supply chain management, infrastructure, adequate staff, and balanced patient caseloads.

Conclusion

There are multiple (context, participant responsiveness, intervention complexity and facilitation strategies) interrelated moderating factors influencing implementation fidelity of the ICDM model. Augmenting facilitation strategies (training and clinical mentorship) could further improve the degree of fidelity during the implementation of the ICDM model.

Fidelity and sustainability of Mouth Care Without a Battle and lessons for other innovations in care

There are countless efficacious interventions that improve outcomes when conducted in controlled situations. Many fewer are effective when implemented in real-world situations, largely because they are not implemented with fidelity. Still fewer are sustained over time, for reasons including lack of institutional support and fit with existing values, among others. It is especially important to examine fidelity and sustainability when efficacious interventions are being implemented, because these interventions are the ones that hold the most promise. This project examined the fidelity and sustainability of Mouth Care Without a Battle (MCWB), an evidence-based program conducted in a two-year cluster randomized trial in 14 nursing homes. Results that triangulated two sources of data indicated that fidelity decreased after the first year; they provide guidance to promote fidelity and sustainability of this and other new care practices in nursing homes, including ongoing education, coaching, evaluation, feedback, and sufficient resources.

Investigation of ward fidelity to a multicomponent delirium prevention intervention during a multicentre, pragmatic, cluster randomised, controlled feasibility trial

BACKGROUND

delirium is a frequent complication of hospital admission for older people and can be reduced by multicomponent interventions, but implementation and delivery of such interventions is challenging.

OBJECTIVE

to investigate fidelity to the prevention of delirium system of care within a multicentre, pragmatic, cluster randomised, controlled feasibility trial.

SETTING

five care of older people and three orthopaedic trauma wards in eight hospitals in England and Wales.

DATA COLLECTION

research nurse observations of ward practice; case note reviews and examination of documentation.

ASSESSMENT

10 health care professionals with experience in older people's care assessed the fidelity to 21 essential implementation components within four domains: intervention installation (five items; maximum score = 5); intervention delivery (12 items; maximum score = 48); intervention coverage (three items; maximum score = 16); and duration of delivery (one item; maximum score = 1).

RESULTS

the mean score (range) for each domain was: installation 4.5 (3.5-5); delivery 32.6 (range 27.3-38.3); coverage 7.9 (range 4.2-10.1); and duration 0.38 (0-1). Of the 10 delirium risk factors, infection, nutrition, hypoxia and pain were the most and cognitive impairment, sensory impairment and multiple medications the least consistently addressed. Overall fidelity to the intervention was assessed as high (≥80%) in two wards, medium (51-79%) in five wards and low (≤50%) in one ward.

CONCLUSION

the trial was designed as a pragmatic evaluation, and the findings of medium intervention fidelity are likely to be generalisable to delirium prevention in routine care and provide an important context to interpret the trial outcomes.

A typology of useful evidence: approaches to increase the practical value of intervention research

BACKGROUND

Too often, studies of evidence-based interventions (EBIs) in preventive, community, and health care are not sufficiently useful to end users (typically practitioners, patients, policymakers, or other researchers). The ways in which intervention studies are conventionally conducted and reported mean that there is often a shortage of information when an EBI is used in practice. The paper aims to invite the research community to consider ways to optimize not only the trustworthiness but also the research's usefulness in intervention studies. This is done by proposing a typology that provides some approaches to useful EBIs for intervention researchers. The approaches originate from different research fields and are summarized to highlight their potential benefits from a usefulness perspective.

MAIN MESSAGE

The typology consists of research approaches to increase the usefulness of EBIs by improving the reporting of four features in intervention studies: (1) the interventions themselves, including core components and appropriate adaptations; (2) strategies to support-high-quality implementation of the interventions; (3) generalizations about the evidence in a variety of contexts; and (4) outcomes based on end users' preferences and knowledge. The research approaches fall into three levels: Description, Analysis, and Design. The first level, Description, outlines what types of information about the intervention and its implementation, context, and outcomes can be helpful for end users. Research approaches under analysis offers alternative ways of analyzing data, increasing the precision of information provided to end users. Approaches summarized under design involve more radical changes and far-reaching implications for how research can provide more useful information. These approaches partly flip the order of efficacy and effectiveness, focusing not on whether an intervention works in highly controlled and optimal circumstances, but first and foremost whether an intervention can be implemented and lead to anticipated outcomes in everyday practice.

CONCLUSIONS

The research community, as well as the end users of research, are invited to consider ways to optimize research's usefulness as well as its trustworthiness. Many of the research approaches in the typology are not new, and their contributions to quality have been described for generations - but their contributions to useful knowledge need more attention.

Shifting care from hospital to community, a strategy to integrate care in Singapore: process evaluation of implementation fidelity

Background

Accessibility to efficient and person-centered healthcare delivery drives healthcare transformation in many countries. In Singapore, specialist outpatient clinics (SOCs) are commonly congested due to increasing demands for chronic care. To improve this situation, the National University Health System (NUHS) Regional Health System (RHS) started an integrated care initiative,the Right-Site Care (RSC) program in 2014. Through collaborations between SOCs at the National University Hospital and primary and community care (PCC) clinics in the western region of the county, the program was designed to facilitate timely discharge and appropriate transition of patients, who no longer required specialist care, to the community. The aim of this study was to evaluate the implementation fidelity of the NUHS RHS RSC program using the modified Conceptual Framework for Implementation Fidelity (CFIF), at three distinct levels; providers, organizational, and system levels to explain outcomes of the program and to inform further development of (similar) programs.

Methods

A convergent parallel mixed methods study using the realist evaluation approach was used. Data were collected between 2016 and 2018 through non-participatory observations, reviews of medical records and program database, together with semi-structured interviews with healthcare providers. Triangulation of data streams was applied guided by the modified CFIF.

Results

Our findings showed four out of six program components were implemented with low level of fidelity, and 9112 suitable patients were referred to the program while 3032 (33.3%) declined to be enrolled. Moderating factors found to influence fidelity included: (i) complexity of program, (ii) evolving providers’ responsiveness, (iii) facilitation through synergistic partnership, training of PCC providers by specialists and supportive structures: care coordinators, guiding protocols, shared electronic medical record and shared pharmacy, (iv) lack of organization reinforcement, and (v) mismatch between program goals, healthcare financing and providers’ reimbursement.

Conclusion

Functional integration alone is insufficient for a successful right-site care program implementation. Improvement in relationships between providers, organizations, and patients are also warranted for further development of the program.

Assessment of the implementation fidelity of a strategy to scale up integrated care in five European regions: a multimethod study

OBJECTIVE

The SCaling IntegRated Care in COntext (SCIROCCO) project tested a step-based scaling up strategy to explore what and how to scale up integrated care initiatives in five European regions. To gain a profound understanding of which factors influence the implementation of this strategy, the objective of this study was to assess the extent to which the SCIROCCO strategy was implemented as planned.

DESIGN

Multimethod study.

METHODS

The extended version of the conceptual framework for implementation fidelity was used to evaluate what factors influence the implementation of the scaling up strategy. Data were collected in the five participating European regions during the intervention period. Data collection methods included: key informant interviews, focus groups, questionnaire studies and project documents.

RESULTS

All three main steps of the scaling up strategy were implemented with acceptable fidelity. Variations were observed in the duration of implementing the steps in the regions. Also, variations were observed in the coverage of experts to participate in the steps of the strategy. Several factors were observed to influence the implementation: facilitation conditions (ie, good coordination for implementation) and participant responsiveness (ie, a positive experience of participants in the organised study visits). Factors that may have moderated adherence to the original plan of the strategy were found in facilitating conditions (ie, in the flexible approach), participant recruitment factors (ie, adaptions of the procedure by the regions) and contextual factors (ie, the level of development of integrated care).

CONCLUSION

This was the first study to assess implementation fidelity of a European project that used a step-based scaling up strategy in five European regions. Similar European projects that are based on collaboration between several European regions can learn from the lessons captured in SCIROCCO and can become more aware of the facilitating factors and pitfalls of implementing such projects.

Focusing on fidelity: narrative review and recommendations for improving intervention fidelity within trials of health behaviour change interventions

Background: Interventions to change behaviour have substantial potential to impact positively on individual and overall public health. Despite an increasing focus on health behaviour change intervention research, interventions do not always have the desired effect on outcomes, while others have diluted effects once implemented into real-life settings. There is little investment into understanding how or why such interventions work or do not work. Methodological inadequacies of trials of behavioural interventions have been previously suggested as a barrier to the quality and advancement of behavioural research, with intervention fidelity acknowledged as a key area for improvement. However, there is much ambiguity regarding the terminology and conceptualisation of intervention fidelity and a lack of practical guidance regarding how to address it sufficiently, particularly within trials of complex behavioural interventions. Objectives: This article outlines specific issues concerning intervention fidelity within trials of health behaviour change interventions and suggests practical considerations and specific recommendations for researchers, with examples from the literature presented. Conclusions: Recommendations pertain to (1) clarifying how fidelity is defined and conceptualised, (2) considering fidelity beyond intervention delivery, (3) considering strategies to both enhance and assess fidelity, (4) making use of existing frameworks and guidance, (5) considering the quality and comprehensiveness of fidelity assessment strategies, (6) considering the balance between fidelity and adaptation and (7) reporting the use of fidelity enhancement and assessment strategies and their results. Suggestions for future research to improve our understanding of, and ability to, address fidelity in behaviour change interventions are also provided.

Sustainable, healthy cities: protocol of a mixed methods evaluation of a cluster randomized controlled trial for Aedes control in Brazil using a community mobilization approach

Background

Dengue is increasing in its global presence with an estimated 4 billion people at-risk of infection in at least 128 countries. Despite the promising results of EcoHealth and community mobilization approaches to Aedes reduction, more evidence of their efficacy on reducing dengue risk is needed. The principal research question is to determine if interventions based upon community mobilization reduce the risk of dengue virus infection among children 3 to 9 years old compared to usual dengue control practice in Fortaleza, Brazil.

Methods

The present study will follow a pragmatic cluster randomized controlled trial (cRCT) design with randomization at the census tract level with equal allocation to the two arms. In each arm, there will be 34 clusters of 86 children between 3 to 9 years old for an expected total of 5848 children enrolled in the study, assuming a risk reduction of 29.5% based upon findings from a previous multi-site cRCT. The primary outcomes are rates of anti-dengue Immunoglobulin G (IgG) seroconversion and adult female Aedes density. The intervention is based upon a participatory health research approach, Socializing Evidence for Participatory Action (SEPA), where the research evidence is used to foster community engagement and ownership of the health issue and solution. Following allocation, intervention communities will develop and implement their own solutions that will likely include a wide variety of collective events and media approaches. Data collection activities over a period of 3 years include household visits for blood collection, household surveys, and entomological surveys; and qualitative activities including focus groups, in-depth interviews, and document analysis to evaluate the process, acceptability, fidelity, and sustainability of the intervention. Study participants will be aware of their assignment and all research staff will be blinded although the intervention assignment will likely be revealed to field staff through interaction with participants.

Discussion

The results of our study will provide evidence on community mobilization as an intervention for dengue control. We anticipate that if community mobilization is effective in Fortaleza, the results of this study will help develop evidence-based vector control programs in Brazil, and also in other countries struggling with Aedes-transmitted diseases.

Trial registration

ISRCTN66131315, registration date: 1 October 2018.

Sustainment, Sustainability, and Spread Study (SSaSSy): protocol for a study of factors that contribute to the sustainment, sustainability, and spread of practice changes introduced through an evidence-based quality-improvement intervention in Canadian nursing homes

Background

Implementation scientists and practitioners, alike, recognize the importance of sustaining practice change, however post-implementation studies of interventions are rare. This is a protocol for the Sustainment, Sustainability and Spread Study (SSaSSy). The purpose of this study is to contribute to knowledge on the sustainment (sustained use), sustainability (sustained benefits), and spread of evidence-based practice innovations in health care. Specifically, this is a post-implementation study of an evidence-informed, Care Aide-led, facilitation-based quality-improvement intervention called SCOPE (Safer Care for Older Persons (in long-term care) Environments). SCOPE has been implemented in nursing homes in the Canadian Provinces of Manitoba (MB), Alberta (AB) and British Columbia (BC). Our study has three aims: (i) to determine the role that adaptation/contextualization plays in sustainment, sustainability and spread of the SCOPE intervention; (ii) to study the relative effects on sustainment, sustainability and intra-organizational spread of high-intensity and low-intensity post-implementation “boosters”, and a “no booster” condition, and (iii) to compare the relative costs and impacts of each booster condition.

Methods/design

SSaSSy is a two-phase mixed methods study. The overarching design is convergent, with qualitative and quantitative data collected over a similar timeframe in each of the two phases, analyzed independently, then merged for analysis and interpretation. Phase 1 is a pilot involving up to 7 units in 7 MB nursing homes in which SCOPE was piloted in 2016 to 2017, in preparation for phase 2. Phase 2 will comprise a quasi-experiment with two treatment groups of low- and high-intensity post-implementation “boosters”, and an untreated control group (no booster), using pretests and post-tests of the dependent variables relating to sustained care and management practices, and resident outcomes. Phase 2 will involve 31 trial sites in BC (17 units) and AB (14 units) nursing homes, where the SCOPE trial concluded in May 2019.

Discussion

This project stands to advance understanding of the factors that influence the sustainment of practice changes introduced through evidence-informed practice change interventions, and their associated sustainability. Findings will inform our understanding of the nature of the relationship of fidelity and adaptation to sustainment and sustainability, and afford insights into factors that influence the intra-organizational spread of practice changes introduced through complex interventions.

Process evaluation of implementation fidelity of the integrated chronic disease management model in two districts, South Africa

BACKGROUND

The Integrated Chronic Disease Management (ICDM) model has been implemented in South Africa to enhance quality of clinical services in Primary Healthcare (PHC) clinics in a context of a high prevalence of chronic conditions and multi-morbidity. This study aimed to assess the implementation fidelity (adherence to guidelines) of the ICDM model.

METHODS

A cross-sectional study in 16 PHC clinics in two health districts in South Africa: Dr. Kenneth Kaunda (DKK) and West Rand (WR). A fidelity assessment tool with 89 activities and maximum score of 158 was developed from the four interrelated ICDM model components: facility re-organization, clinical supportive management, assisted self-management and strengthening of support systems. Value stream mapping of patient flow was conducted to analyse waiting time and identify operational inefficiencies. ICDM items were scored based on structured observations, facility document reviews and structured questionnaires completed by healthcare workers. Fidelity scores were summarized using medians and proportions and compared by facilities and districts using Chi-Square and Kruskal Wallis test.

RESULTS

The monthly patient headcount over a six-month period in these 16 PHC clinics was a median of 2430 (IQR: 1685-2942) individuals over 20 years. The DKK district had more newly diagnosed TB patients per month [median 5.5 (IQR: 4.00-9.33) vs 2.0 (IQR: 1.67-2.92)], and fewer medical officers per clinic [median 1 (IQR: 1-1) vs 3.5 (IQR:2-4.5)] compared to WR district. The median fidelity scores in both districts for facility re-organization, clinical supportive management, assisted self-management and strengthening of support systems were 78% [29/37, IQR: 27-31)]; 77% [30/39 (IQR: 27-34)]; 77% [30/39 (IQR: 28-34)]; and 80% [35/44 (IQR: 30-37)], respectively. The overall median implementation fidelity of the ICDM model was 79% (125/158, IQR, 117-132); WR was 80% (126/158, IQR, 123-132) while DKK was 74% (117/158, IQR, 106-130), p = 0.1409. The lowest clinic fidelity score was 66% (104/158), while the highest was 86% (136/158). A patient flow analysis showed long (2-5 h) waiting times and one stream of care for acute and chronic services.

CONCLUSION

There was some variability of scores on components of the ICDM model by PHC clinics. More research is needed on contextual adaptations of the model.

Implementation and Long-Term Outcomes of Organisational Health Literacy Interventions in Ireland and The Netherlands: A Longitudinal Mixed-Methods Study

Organisational Health Literacy (OHL)-interventions are needed to overcome health inequality. OHL-interventions have successfully identified communication barriers at the organisational level, but evidence is limited on the extent to which this leads to sustainable organisational change. This study aims to assess the implementation fidelity, moderators (barriers and facilitators), and long-term impact of OHL-interventions in hospitals in Ireland and The Netherlands. We used a longitudinal mixed-methods approach to assess two similar OHL-interventions in one Irish and three Dutch hospitals. The OHL-interventions concerned the improvement of navigation and implementation of health literacy-friendly communication throughout organisations. Participants were 24 hospital employees and 40 older adults who use hospital services. At six, eight, and eighteen months, we assessed the level of implementation, barriers and facilitators, and impact through questionnaires and in-depth semi-structured interviews. After older adults and professionals had identified a number of communication problems, we found that professionals had successfully implemented OHL-interventions to promote navigation and comprehensible communication. Limited resources and variation in organisational structures and procedures were perceived as barriers to implementation. The participation of service users, leadership support, and a stepwise implementation of interventions were perceived to facilitate implementation. In the long term, the OHL-interventions led to system-wide improvements, as shown by better embedding of health literacy policies, enhanced patient engagement, provision of plain language training and comprehensible information. Findings were similar for the two countries. Embedded OHL-interventions resulted in sustainable and system-wide health literacy changes in all four hospitals. Following implementation, OHL-interventions have the potential to promote health equity and empowerment among health service users.

Systematic review of the use of process evaluations in knowledge translation research

BACKGROUND

Experimental designs for evaluating knowledge translation (KT) interventions can provide strong estimates of effectiveness but offer limited insight into how the intervention worked. Consequently, process evaluations have been used to explore the causal mechanisms at work; however, there are limited standards to guide this work. This study synthesizes current evidence of KT process evaluations to provide future methodological recommendations.

METHODS

Peer-reviewed search strategies were developed by a health research librarian. Studies had to be in English, published since 1996, and were not excluded based on design. Studies had to (1) be a process evaluation of a KT intervention study in primary health, (2) be a primary research study, and (3) include a licensed healthcare professional delivering or receiving the intervention. A two-step, two-person hybrid screening approach was used for study inclusion with inter-rater reliability ranging from 94 to 95%. Data on study design, data collection, theoretical influences, and approaches used to evaluate the KT intervention, analysis, and outcomes were extracted by two reviewers. Methodological quality was assessed with the Mixed Methods Appraisal Tool (MMAT).

RESULTS

Of the 20,968 articles screened, 226 studies fit our inclusion criteria. The majority of process evaluations used qualitative forms of data collection (43.4%) and individual interviews as the predominant data collection method. 72.1% of studies evaluated barriers and/or facilitators to implementation. 59.7% of process evaluations were stand-alone evaluations. The timing of data collection varied widely with post-intervention data collection being the most frequent (46.0%). Only 38.1% of the studies were informed by theory. Furthermore, 38.9% of studies had MMAT scores of 50 or less indicating poor methodological quality.

CONCLUSIONS

There is widespread acceptance that the generalizability of quantitative trials of KT interventions would be significantly enhanced through complementary process evaluations. However, this systematic review found that process evaluations are of mixed quality and lack theoretical guidance. Most process evaluation data collection occurred post-intervention undermining the ability to evaluate the process of implementation. Strong science and methodological guidance is needed to underpin and guide the design and execution of process evaluations in KT science.

REGISTRATION

This study is not registered with PROSPERO.

The Elusive Search for Success: Defining and Measuring Implementation Outcomes in a Real-World Hospital Trial

Objective and Study Setting: Research efforts to identify factors that influence successful implementation are growing. This paper describes methods of defining and measuring outcomes of implementation success, using a cluster randomized controlled trial with 12 cancer services in Australia comparing the effectiveness of implementation strategies to support adherence to the Australian Clinical Pathway for the Screening, Assessment and Management of Anxiety and Depression in Adult Cancer Patients (ADAPT CP). Study Design and Methods: Using the StaRI guidelines, a process evaluation was planned to explore participant experience of the ADAPT CP, resources and implementation strategies according to the Implementation Outcomes Framework. This study focused on identifying measurable outcome criteria, prior to data collection for the trial, which is currently in progress. Principal Findings: We translated each implementation outcome into clearly defined and measurable criteria, noting whether each addressed the ADAPT CP, resources or implementation strategies, or a combination of the three. A consensus process defined measures for the primary outcome (adherence) and secondary (implementation) outcomes; this process included literature review, discussion and clear measurement parameters. Based on our experience, we present an approach that could be used as a guide for other researchers and clinicians seeking to define success in their work. Conclusions: Defining and operationalizing success in real-world implementation yields a range of methodological challenges and complexities that may be overcome by iterative review and engagement with end users. A clear understanding of how outcomes are defined and measured, based on a strong theoretical framework, is crucial to meaningful measurement and outcomes. The conceptual approach described in this article could be generalized for use in other studies. Trial Registration: The ADAPT Program to support the management of anxiety and depression in adult cancer patients: a cluster randomized trial to evaluate different implementation strategies of the Clinical Pathway for Screening, Assessment and Management of Anxiety and Depression in Adult Cancer Patients was prospectively registered with the Australian New Zealand Clinical Trials Registry Registration Number: ACTRN12617000411347.

Integrated Care for Frail Elderly: A Qualitative Study of a Promising Approach in The Netherlands

INTRODUCTION

Increasingly, frail elderly need to live at home for longer, relying on support from informal caregivers and community-based health- and social care professionals. To align care and avoid fragmentation, integrated care programmes are arising. A promising example of such a programme is the Care Chain Frail Elderly (CCFE) in the Netherlands, which supports elderly with case and care complexity living at home with the best possible health and quality of life. The goal of the current study was to gain a deeper understanding of this programme and how it was successfully put into practice in order to contribute to the evidence-base surrounding complex integrated care programmes for persons with multi-morbidity.

METHODS

Document analyses and semi-structured interviews with stakeholders were used to create a 'thick description' that provides insights into the programme.

RESULTS

Through case finding, the CCFE-programme targets the frailest primary care population. The person-centred care approach is reflected by the presence of frail elderly at multidisciplinary team meetings. The innovative way of financing by bundling payments of multiple providers is one of the main facilitators for the success of this programme. Other critical success factors are the holistic assessment of unmet health and social care needs, strong leadership by the care groups, close collaboration with the healthcare insurer, a shared ICT-system and continuous improvements.

CONCLUSION

The CCFE is an exemplary initiative to integrate care for the frailest elderly living at home. Its innovative components and critical success factors are likely to be transferable to other settings when providers can take on similar roles and work closely with payers who provide integrated funding.

Self-Management education for adults with poorly controlled epILEpsy [SMILE (UK)]: a randomised controlled trial

BACKGROUND

Epilepsy is a common neurological condition resulting in recurrent seizures. Research evidence in long-term conditions suggests that patients benefit from self-management education and that this may improve quality of life (QoL). Epilepsy self-management education has yet to be tested in a UK setting.

OBJECTIVES

To determine the effectiveness and cost-effectiveness of Self-Management education for people with poorly controlled epILEpsy [SMILE (UK)].

DESIGN

A parallel pragmatic randomised controlled trial.

SETTING

Participants were recruited from eight hospitals in London and south-east England.

PARTICIPANTS

Adults aged ≥ 16 years with epilepsy and two or more epileptic seizures in the past year, who were currently being prescribed antiepileptic drugs.

INTERVENTION

A 2-day group self-management course alongside treatment as usual (TAU). The control group received TAU.

MAIN OUTCOME MEASURES

The primary outcome is QoL in people with epilepsy at 12-month follow-up using the Quality Of Life In Epilepsy 31-P (QOLIE-31-P) scale. Other outcomes were seizure control, impact of epilepsy, medication adverse effects, psychological distress, perceived stigma, self-mastery and medication adherence. Cost-effectiveness analyses and a process evaluation were undertaken.

RANDOMISATION

A 1 : 1 ratio between trial arms using fixed block sizes of two.

BLINDING

Participants were not blinded to their group allocation because of the nature of the study. Researchers involved in data collection and analysis remained blinded throughout.

RESULTS

The trial completed successfully. A total of 404 participants were enrolled in the study [SMILE (UK), n = 205; TAU, n = 199] with 331 completing the final follow-up at 12 months [SMILE (UK), n = 163; TAU, n = 168]. In the intervention group, 61.5% completed all sessions of the course. No adverse events were found to be related to the intervention. At baseline, participants had a mean age of 41.7 years [standard deviation (SD) 14.1 years], and had epilepsy for a median of 18 years. The mean QOLIE-31-P score for the whole group at baseline was 66.0 out of 100.0 (SD 14.2). Clinically relevant levels of anxiety symptoms were reported in 53.6% of the group and depression symptoms in 28.0%. The results following an intention-to-treat analysis showed no change in any measures at the 12-month follow-up [QOLIE-31-P: SMILE (UK) mean: 67.4, SD 13.5; TAU mean: 69.5, SD 14.8]. The cost-effectiveness study showed that SMILE (UK) was possibly cost-effective but was also associated with lower QoL. The process evaluation with 20 participants revealed that a group course increased confidence by sharing with others and improved self-management behaviours.

CONCLUSIONS

For people with epilepsy and persistent seizures, a 2-day self-management education course is cost-saving, but does not improve QoL after 12-months or reduce anxiety or depression symptoms. A psychological intervention may help with anxiety and depression. Interviewed participants reported attending a group course increased their confidence and helped them improve their self-management.

FUTURE WORK

More research is needed on self-management courses, with psychological components and integration with routine monitoring.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN57937389.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 22, No. 21. See the NIHR Journals Library website for further project information.

A pedometer-based walking intervention in 45- to 75-year-olds, with and without practice nurse support: the PACE-UP three-arm cluster RCT

BACKGROUND

Guidelines recommend walking to increase moderate to vigorous physical activity (MVPA) for health benefits.

OBJECTIVES

To assess the effectiveness, cost-effectiveness and acceptability of a pedometer-based walking intervention in inactive adults, delivered postally or through dedicated practice nurse physical activity (PA) consultations.

DESIGN

Parallel three-arm trial, cluster randomised by household.

SETTING

Seven London-based general practices.

PARTICIPANTS

A total of 11,015 people without PA contraindications, aged 45-75 years, randomly selected from practices, were invited. A total of 6399 people were non-responders, and 548 people self-reporting achieving PA guidelines were excluded. A total of 1023 people from 922 households were randomised to usual care (n = 338), postal intervention (n = 339) or nurse support (n = 346). The recruitment rate was 10% (1023/10,467). A total of 956 participants (93%) provided outcome data.

INTERVENTIONS

Intervention groups received pedometers, 12-week walking programmes advising participants to gradually add '3000 steps in 30 minutes' most days weekly and PA diaries. The nurse group was offered three dedicated PA consultations.

MAIN OUTCOME MEASURES

The primary and main secondary outcomes were changes from baseline to 12 months in average daily step counts and time in MVPA (in ≥ 10-minute bouts), respectively, from 7-day accelerometry. Individual resource-use data informed the within-trial economic evaluation and the Markov model for simulating long-term cost-effectiveness. Qualitative evaluations assessed nurse and participant views. A 3-year follow-up was conducted.

RESULTS

Baseline average daily step count was 7479 [standard deviation (SD) 2671], average minutes per week in MVPA bouts was 94 minutes (SD 102 minutes) for those randomised. PA increased significantly at 12 months in both intervention groups compared with the control group, with no difference between interventions; additional steps per day were 642 steps [95% confidence interval (CI) 329 to 955 steps] for the postal group and 677 steps (95% CI 365 to 989 steps) for nurse support, and additional MVPA in bouts (minutes per week) was 33 minutes per week (95% CI 17 to 49 minutes per week) for the postal group and 35 minutes per week (95% CI 19 to 51 minutes per week) for nurse support. Intervention groups showed no increase in adverse events. Incremental cost per step was 19p and £3.61 per minute in a ≥ 10-minute MVPA bout for nurse support, whereas the postal group took more steps and cost less than the control group. The postal group had a 50% chance of being cost-effective at a £20,000 per quality-adjusted life-year (QALY) threshold within 1 year and had both lower costs [-£11M (95% CI -£12M to -£10M) per 100,000 population] and more QALYs [759 QALYs gained (95% CI 400 to 1247 QALYs)] than the nurse support and control groups in the long term. Participants and nurses found the interventions acceptable and enjoyable. Three-year follow-up data showed persistent intervention effects (nurse support plus postal vs. control) on steps per day [648 steps (95% CI 272 to 1024 steps)] and MVPA bouts [26 minutes per week (95% CI 8 to 44 minutes per week)].

LIMITATIONS

The 10% recruitment level, with lower levels in Asian and socioeconomically deprived participants, limits the generalisability of the findings. Assessors were unmasked to the group.

CONCLUSIONS

A primary care pedometer-based walking intervention in 45- to 75-year-olds increased 12-month step counts by around one-tenth, and time in MVPA bouts by around one-third, with similar effects for the nurse support and postal groups, and persistent 3-year effects. The postal intervention provides cost-effective, long-term quality-of-life benefits. A primary care pedometer intervention delivered by post could help address the public health physical inactivity challenge.

FUTURE WORK

Exploring different recruitment strategies to increase uptake. Integrating the Pedometer And Consultation Evaluation-UP (PACE-UP) trial with evolving PA monitoring technologies.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN98538934.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 22, No. 37. See the NIHR Journals Library website for further project information.