Process evaluation of the maudsley model for treatment of adults with anorexia nervosa trial. Part II: Patient experiences of two psychological therapies for treatment of anorexia nervosa

OBJECTIVE

This study is the second part of a process evaluation, embedded in the MOSAIC study, a large randomised controlled trial comparing two different psychological therapies, the Maudsley Model for Treatment of Adults with Anorexia Nervosa (MANTRA) and Specialist Supportive Clinical Management (SSCM). The study adopted a qualitative approach to examine patient experiences of the two treatments.

METHOD

Seventeen semi-structured interviews were conducted with Anorexia Nervosa and Eating Disorder Not Otherwise Specified-Anorexia Nervosa type patients, and transcripts were analysed thematically.

RESULTS

Patient responses yielded five main themes: positive and helpful aspects, beneficial outcomes, less helpful aspects, possible improvements to the treatments, and the therapeutic and external environment. The findings show clear differences and some overlaps between patients' views on MANTRA and SSCM.

DISCUSSION

Both therapies were experienced by patients as credible and largely helpful, albeit in different ways. These results are in agreement with those of therapists' views on these treatments.

Adaptation and Implementation of the STarT Back Risk Stratification Strategy in a US Health Care Organization: A Process Evaluation

OBJECTIVE

To support implementation of effective treatments for back pain that can be delivered to a range of people, we summarize learnings from our process evaluation of the MATCH trial's implementation of an adaptation of the STarT Back risk-stratified care model.

DESIGN

Our logic model-driven evaluation focused primarily on qualitative data sources.

SETTING

This study took place in a US-based health care delivery system that had adapted and implemented the STarT Back stratified care approach. This was the first formal test of the strategy in a US setting.

METHODS

Data collection included observation of implementation activities, staff/provider interviews, and post-training evaluation questionnaires. Data were analyzed using thematic analysis of qualitative data and descriptive statistics for questionnaire data.

RESULTS

We found that both primary care teams and physical therapists at intervention clinics gave the training high scores on evaluation questionnaires and reported in the interviews that they found the training engaging and useful. However, there was significant variation in the extent to which the risk stratification strategy was incorporated into care. Some primary care providers reported that the intervention changed their conversations with patients and increased their confidence in working with patients with back pain. Providers using the STarT Back tool did not change referral rates for recommended matched treatments.

CONCLUSIONS

These insights provide guidance for future efforts to adapt and implement the STarT Back strategy and other complex practice change interventions. They emphasize the need for primary care-based interventions to minimize complexity and the need for ongoing monitoring and feedback.

Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: case study evaluation of adoption and maintenance of a complex intervention to reduce high-risk primary care prescribing

OBJECTIVE

To explore how different practices responded to the Data-driven Quality Improvement in Primary Care (DQIP) intervention in terms of their adoption of the work, reorganisation to deliver the intended change in care to patients, and whether implementation was sustained over time.

DESIGN

Mixed-methods parallel process evaluation of a cluster trial, reporting the comparative case study of purposively selected practices.

SETTING

Ten (30%) primary care practices participating in the trial from Scotland, UK.

RESULTS

Four practices were sampled because they had large rapid reductions in targeted prescribing. They all had internal agreement that the topic mattered, made early plans to implement including assigning responsibility for work and regularly evaluated progress. However, how they internally organised the work varied. Six practices were sampled because they had initial implementation failure. Implementation failure occurred at different stages depending on practice context, including internal disagreement about whether the work was worthwhile, and intention but lack of capacity to implement or sustain implementation due to unfilled posts or sickness. Practice context was not fixed, and most practices with initial failed implementation adapted to deliver at least some elements. All interviewed participants valued the intervention because it was an innovative way to address on an important aspect of safety (although one of the non-interviewed general practitioners in one practice disagreed with this). Participants felt that reviewing existing prescribing did influence their future initiation of targeted drugs, but raised concerns about sustainability.

CONCLUSIONS

Variation in implementation and effectiveness was associated with differences in how practices valued, engaged with and sustained the work required. Initial implementation failure varied with practice context, but was not static, with most practices at least partially implementing by the end of the trial. Practices organised their delivery of changed care to patients in ways which suited their context, emphasising the importance of flexibility in any future widespread implementation.

TRIAL REGISTRATION NUMBER

NCT01425502.

It Is Not That Simple nor Compelling! Comment on "Translating Evidence Into Healthcare Policy and Practice: Single Versus Multi-faceted Implementation Strategies - Is There a Simple Answer to a Complex Question?"

Healthcare decisions are often made under pressure, with varying levels of information in a changing clinical context. With limited resources and a focus on improving patient outcomes, healthcare managers and health professionals strive to implement both clinical and cost-effective care. However, the gap between research evidence and health policy/clinical practice persists despite our best efforts. In an attempt to close the gap through behaviour change interventions, there has been a strong held belief that 'more is better,' without understanding the mechanisms and circumstances of knowledge translation (KT). We argue that even a single intervention or strategy in translating evidence into healthcare policy or practice is rarely simple to implement. Nor is the evidence compelling on the best approach. As Harvey and Kitson argued, designing and evaluating KT interventions requires flexibility and responsiveness. If we are to move forward in translation science then we need to use rigorous designs such as randomised controlled trials to test effectiveness of interventions or strategies with embedded process evaluations to understand the reason interventions do or do not work!

INterpreting the Processes of the UMPIRE Trial (INPUT): protocol for a qualitative process evaluation study of a fixed-dose combination (FDC) strategy to improve adherence to cardiovascular medications

INTRODUCTION

This paper describes a planned process evaluation of the Use of a Multidrug Pill In Reducing Cardiovascular Events (UMPIRE) trial, one of several randomised clinical trials taking place globally to assess the potential of cardiovascular drugs as a fixed-dose combination (polypill) in cardiovascular disease prevention. A fixed-dose combination may be a promising strategy for promoting adherence to medication; alleviating pill burden through simplifying regimens and reducing cost. This process evaluation will complement the UMPIRE trial by using qualitative research methods to inform understanding of the complex interplay of factors that underpin trial outcomes.

METHODS

A series of semistructured, in-depth interviews with local health professionals and UMPIRE trial participants in India and the UK will be undertaken. The aim is to understand their views and experiences of the trial context and of day-to-day use of medications more generally. The grounded theory approach will be used to analyse data and help inform the processes of the UMPIRE trial.

ETHICS AND DISSEMINATION

The study has received ethical approval for all sites in the UK and India where trial participant interviews will be undertaken. The process evaluation will help inform and enhance the understanding of the UMPIRE trial results and its applicability to clinical practice as well as shaping policy regarding strategies for improving cardiovascular medication adherence.

A Continuous Quality Improvement Intervention to Improve Antenatal HIV Care Testing in Rural South Africa: Evaluation of Implementation in a Real-World Setting

Background: We evaluated continuous quality improvement (CQI) targeting antenatal HIV care quality in rural South Africa using a stepped-wedge cluster-randomised controlled trial (Management and Optimisation of Nutrition, Antenatal, Reproductive, Child health, MONARCH) and an embedded process evaluation. Here, we present results of the process evaluation examining determinants of CQI practice and ‘normalisation.’

Methods: A team of CQI mentors supported public-sector health workers in seven primary care clinics to (1) identify root causes of poor HIV viral load (VL) monitoring among pregnant women living with HIV and repeat HIV testing among pregnant women not living with HIV, and (2) design and iteratively test their own solutions. We used a mixed methods evaluation with field notes from CQI mentors (‘dose’ and ‘reach’ of CQI, causes of poor HIV care testing rates, implemented change ideas); patient medical records (HIV care testing by clinic and time step); and semi-structured interviews with available health workers. We analysed field notes and semi-structured interviews for determinants of CQI implementation and ‘normalisation’ using Normalisation Process Theory (NPT) and Tailored Implementation of Chronic Diseases (TICD) frameworks.

Results: All interviewed health workers found the CQI mentors and methodology helpful for quality improvement. Total administered ‘dose’ was higher than planned but ‘reach’ was limited by resource constraints, particularly staffing shortages. Simple workable improvements to identified root causes were implemented, such as a patient tracking notebook and results filing system. VL monitoring improved over time, but not repeat HIV testing. Besides resource constraints, gaps in knowledge of guidelines, lack of leadership, poor clinical documentation, and data quality gaps reduced CQI implementation fidelity and normalisation.

Conclusion: While CQI holds promise, we identified several health system challenges. Priorities for policy makers include improving staffing and strategies to improve clinical documentation. Additional support with implementing clinical guidelines and improving routine data quality are needed. Normalising CQI may be challenging without concurrent health system improvements.

How European Fans in Training (EuroFIT), a lifestyle change program for men delivered in football clubs, achieved its effect: a mixed methods process evaluation embedded in a randomised controlled trial

BACKGROUND

A randomised trial of European Fans in Training (EuroFIT), a 12-week healthy lifestyle program delivered in 15 professional football clubs in the Netherlands, Norway, Portugal, and the United Kingdom, successfully increased physical activity and improved diet but did not reduce sedentary time. To guide future implementation, this paper investigates how those effects were achieved. We ask: 1) how was EuroFIT implemented? 2) what were the processes through which outcomes were achieved?

METHODS

We analysed qualitative data implementation notes, observations of 29 of 180 weekly EuroFIT deliveries, semi-structured interviews with 16 coaches and 15 club representatives, and 30 focus group discussions with participants (15 post-program and 15 after 12 months). We descriptively analysed quantitative data on recruitment, attendance at sessions and logs of use of the technologies and survey data on the views of participants at baseline, post program and after 12 months. We used a triangulation protocol to investigate agreement between data from difference sources, organised around meeting 15 objectives within the two research questions.

RESULTS

We successfully recruited clubs, coaches and men to EuroFIT though the draw of the football club seemed stronger in the UK and Portugal. Advertising that emphasized getting fitter, club-based deliveries, and not 'standing out' worked and attendance and fidelity were good, so that coaches in all countries were able to deliver EuroFIT flexibly as intended. Coaches in all 15 clubs facilitated the use of behaviour change techniques and interaction between men, which together enhanced motivation. Participants found it harder to change sedentary time than physical activity and diet. Fitting changes into daily routines, planning for setbacks and recognising the personal benefit of behaviour change were important to maintain changes. Bespoke technologies were valued, but technological hitches frustrated participants.

CONCLUSION

EuroFIT was delivered as planned by trained club coaches working flexibly in all countries. It worked as expected to attract men and support initiation and maintenance of changes in physical activity and diet but the use of bespoke, unstable, technologies was frustrating. Future deliveries should eliminate the focus on sedentary time and should use only proven technologies to support self-monitoring and social interaction.

TRIAL REGISTRATION

ISRCTN81935608, registered 16/06/2015.

Implementation and Evaluation of the Graded Repetitive Arm Supplementary Program (GRASP) for People With Stroke in a Real World Community Setting: Case Report

OBJECTIVE

Exercise programs to improve upper extremity function following stroke in the community setting are needed as the length of hospital stay continues to decrease. However, little has been done to increase understanding of how to translate an evidence-based rehabilitation intervention to real-world programs. The purpose of this case report was to describe a process evaluation of the implementation of an evidence-based upper extremity rehabilitation intervention for stroke, the Graded Repetitive Arm Supplementary Program, in a community setting.

METHODS (CASE DESCRIPTION)

A partnership between a nonprofit support organization and a local community center was established to deliver the program in the community. The Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework with mixed methods was used to evaluate the implementation.

RESULTS

Reach: Twenty people were screened, 14 people met eligibility requirements, and 13 consented to participate. The program reached approximately 1.25% of the potential target population. Effectiveness: Participants with stroke demonstrated significant improvement in upper extremity function and quality of life as measured by the Fugl-Meyer Assessment for upper extremity, Action Research Arm Test, Rating of Everyday Arm-use in the Community and Home Scale, and Stroke Impact Scale. Adoption: Factors that facilitated program uptake were the well-planned implementation and the workplace coaching based on the audit results. Factors contributing to ongoing participation were the social support within the group environment and the instructor's capability of engaging the group. Implementation: A partnership between a nonprofit organization and a local community center was successfully established. The program was implemented as intended as verified by a fidelity checklist. Participant adherence was high as confirmed by the average attendance and practice time. Maintenance: Both the partner organization and community center continued to offer the program.

CONCLUSION

The Graded Repetitive Arm Supplementary Program had good fidelity of the critical principles and core components and was effective in improving upper extremity function and quality of life.

IMPACT

This partnership model may serve as the first step for future larger-scale implementation and could be used to move other stroke rehabilitation interventions into community settings.

Process evaluation of the MOSAIC Trial, Part I: Therapist experiences of delivering two psychological therapies for treatment of anorexia nervosa

OBJECTIVES

Forming part of a process evaluation of a large randomised controlled trial (the Maudsley Outpatient Study of Treatments for Anorexia Nervosa and Related conditions, MOSAIC) comparing two outpatient therapies for Anorexia Nervosa (AN), the Maudsley Model for Treatment of Adults with Anorexia Nervosa (MANTRA) and Specialist Supportive Clinical Management (SSCM), this study adopted a qualitative approach to examine therapist experiences of treatment delivery.

METHOD

Twenty MOSAIC therapists completed semi-structured interviews. Interviews were recorded, transcribed and analysed thematically.

RESULTS

Themes of positive aspects, challenges and therapeutic fit emerged. MANTRA was seen as structured and flexible but could feel demanding on therapist time and skill. The slow pace and narrower focus of SSCM gave patients space to talk, but the lack of psychological tools and nutritional emphasis could create frustration. Views on the therapeutic relationship and patient-therapy fit differed across treatments.

DISCUSSION

Findings provide testable hypotheses about what works for whom, ideas for therapist training, treatment development and delivery.

Process evaluation and impact results of the worksite component of a multilevel, multicomponent obesity prevention trial in six Native American communities

Native Americans (NA) have higher obesity rates compared to other populations. Employed adults spend a significant amount of time at work. OPREVENT2, an obesity prevention trial in 6 NA communities, included a worksite component that incorporated nutrition and physical activity educational media, competitions, tastes tests, and coffee station makeovers. Process evaluation results indicate a well-implemented worksite component based on team standards. Statistically significant improvements of coffee stations healthy offerings (p=0.006), but none in health policies and resources, were observed. Partnering with businesses to create healthier working environments was successful and future trials should investigate the effects on an individual level.

M-SAKHI - Mobile health solutions to help community providers promote maternal and infant nutrition and health: a description of development of the Program Impact Pathway using Theory of Change

OBJECTIVE

Behaviour Change Communication (BCC) intervention programmes often lack documentation of successful processes. This manuscript aims to describe the development of Program Impact Pathway (PIP) using Theory of Change (ToC) approach for a mHealth BCC intervention titled 'Mobile Solutions Aiding Knowledge for Health Improvement (M-SAKHI)' aimed at reducing stunting in infants at 18 months of age.

DESIGN

The PIP was developed using ToC to design the intervention and plan its implementation. Literature review and data from previous pilots helped to identify health service gaps that needed to be addressed by the PIP of this intervention.

SETTING

M-SAKHI was implemented in 244 villages under governance of forty primary health centres of Nagpur and Bhandara districts of eastern Maharashtra in central India.

PARTICIPANTS

The study investigators and the public health stakeholders participated in developing the PIP. M-SAKHI evaluation study recruited 2501 pregnant women who were followed up through delivery until their infants were 18 months old.

RESULTS

The PIP was developed, and it identified the following pathways for the final impact: (1) improving maternal and infant nutrition, (2) early recognition of maternal and infant danger signs, (3) improving access and utilisation to healthcare services, (4) improving hygiene, sanitation and immunisation practices, and (5) improving implementation and service delivery of community health workers through their training, monitoring and supervision in real time.

CONCLUSION

This paper will illustrate the significance of development of PIP for M-SAKHI. It can aid other community-based programmes to design their PIP for nutrition-based BCC interventions.

Implementing the Effective Coach2Move Approach for Community-Dwelling Older Adults With Mobility Limitations in Physical Therapist Practice: A Multi-Methods Process Evaluation

OBJECTIVE

The aims of this study were to evaluate the adoption of and fidelity to the Coach2Move approach; identify differences between physical therapists with a specialization in geriatrics and physical therapists without a specialization; explore if level of adoption and specialization explain variances in effectiveness; and explore group differences in attitudes and experiences with implementation.

METHODS

A multi-methods process evaluation of Coach2Move implementation through 2 education days, 3 peer-assessment meetings, and an adapted electronic health record was performed alongside a cluster randomized stepped-wedge trial comparing regular care physical therapy with Coach2Move. Participants were 36 physical therapists with a specialization (n = 17) and without a specialization (n = 19) who treated 292 community-dwelling older adults. Level of adoption and fidelity were analyzed by comparing preimplementation scores with scores 1 year later. Coach2Move adoption was measured by e-assessment scores, and fidelity through health record indicators. The impact of specialization and adoption on health outcomes was examined using a mixed-model analysis of variance. Therapists' attitudes and experiences were collected through a survey based on semistructured interviews.

RESULTS

Mean (SD) total indicator scores on the e-assessment (adoption of a Coach2Move mindset) increased from baseline 17% (5%) to 47% (9%) at follow-up. Physical therapists with a specialization in geriatrics scored (mean [SD]) higher than physical therapists without (54% [6%] vs 41% [6%]). Mean (SD) indicator scores on health records (fidelity) increased from 35% (12%) at baseline to 47% (15%) at follow-up. Mean scores of physical therapists with a specialization in geriatrics were higher. Level of adoption and specialization (yes/no) did not explain the variance in effectiveness. Physical therapists identified important facilitators, including tailored education and peer-assessment meetings, whereas adequate reimbursement for the extra time investment was considered a necessity. Different workflows in practices were perceived as a barrier.

CONCLUSION

Implementation led to increased adoption and fidelity of the Coach2Move intervention by physical therapists but shows room for improvement. Attitudes toward the approach and its implementation were positive. Future implementation efforts on adoption could be improved by focusing on a fair compensation structure by third-party payers and insurance companies and optimizing organizational and financial context within practices.

IMPACT

This study evaluated the implementation of Coach2Move, a clinical reasoning approach designed to increase physical activity and improve self-efficacy in older adults. Overall, the study demonstrates the potential of Coach2Move to be effectively adopted by physical therapists. However, addressing therapist compensation and adapting to practice workflows are important considerations for successful large-scale implementation.

A Mixed Methods Process Evaluation of a Food Hygiene Intervention in Low-Income Informal Neighbourhoods of Kisumu, Kenya

OBJECTIVES

Diarrhoea is a leading cause of infant mortality with the main transmission pathways being unsafe water and contaminated food, surfaces and hands. The 'Safe Start' trial evaluated a food hygiene intervention implemented in a peri-urban settlement of Kisumu, Kenya, with the aim of reducing diarrhoeagenic enteric infections among infants. Four food hygiene behaviours were targeted: handwashing with soap before preparation and feeding, boiling infant food before feeding, storing infant food in sealed containers, and exclusive use of designated utensils during feeding.

METHODS

A process evaluation of the intervention was guided by a theory of change describing the hypothesised implementation and receipt of the intervention, mechanisms of change, and the context. These were assessed by qualitative and quantitative data that included debriefing sessions with the delivery teams and Community Health Volunteers (CHVs), and structured observations during food preparation.

RESULTS

The intervention achieved high coverage and fidelity with over 90% of 814 eligible caregivers participating in the intervention. Caregivers in the intervention arm demonstrated an understanding of the intervention messages, and had 1.38 (95% CI: 1.02-1.87) times the odds of washing hands before food preparation and 3.5 (95% CI: 1.91-6.56) times the odds of using a feeding utensil compared to caregivers in the control group. Contextual factors, especially the movement of caregivers within and outside the study area and time constraints faced by caregivers influenced uptake of some intervention behaviours.

CONCLUSION

Future interventions should seek to explicitly target contextual factors such as secondary caregivers and promote food hygiene interventions as independent of each other.