Using Realist Evaluation to Understand Process Outcomes in a COVID-19-Impacted Yoga Intervention Trial: A Worked Example

Realist process evaluation of the knowledge translation programme of a patient-oriented research network

RATIONALE

The Knowledge Translation (KT) Programme of a pan-Canadian strategic patient-oriented research network focused on brain-based developmental disabilities aimed to mobilize knowledge relevant to the network members. The programme also promotes and studies integrated Knowledge Translation (iKT) approaches involving different interested parties, such as researchers, patient-partners and decision-makers, in all parts of the knowledge creation process.

AIMS AND OBJECTIVES

The objective of this study is to advance research programme evaluation methods through a realist evaluation of the process of implementing iKT activities.

METHODS

Realist process evaluation included: (1) development of initial programme theories (using the partnership synergy theory); (2) data collection and analysis; (3) synthesis and refinement of theories through engagement with literature; and (4) presentation of findings in context-mechanism-outcome (C-M-O) configurations. A range of project documentation records were reviewed for analysis, and three co-leads, a programme coordinator, and a senior research associate were consulted to contextualize the implementation process of relevant KT activities.

RESULTS

Based on the developed C-M-O configurations, we identified five key mechanisms of generating synergy in the iKT processes: (1) Visible shared leadership that embodies what iKT looks like; (2) Researchers' readiness for iKT; (3) Adaptation and flexible allocation of resources to emerging needs; (4) Power sharing to create practical and creative knowledge; and (5) Collective voice for potential transformative impacts at the policy level.

CONCLUSIONS

The current realist evaluation demonstrated how partnerships between researchers, patient-partners and other interested parties can synergistically generate new ways of thinking among all interested parties, actionable strategies to integrate users in research, and solutions to disseminate knowledge. In particular, we identified a pivotal role for patient-partners to act as equal decision-maker helps building and maintaining partnerships and consolidating KT strategies.

Realist process evaluation of occupational performance coaching: protocol

INTRODUCTION

A cluster randomised controlled trial, the Meaning, Agency and Nurturing Autonomy (MANA) study, is underway comparing the effects of occupational performance coaching (OPC) and usual care on the social participation, health and well-being of children with neurodisability and their caregivers. This protocol presents the realist process evaluation which is occurring in parallel with the trial to allow testing and further refinement of OPC programme theory, as represented in its logic model. The aim of this realist evaluation is to examine what works, for whom, in the implementation of OPC with caregivers of children with neurodisability (in particular, Māori and Pasifika) in current service delivery contexts.

METHODS AND ANALYSIS

Guided by OPC programme theory and realist evaluation processes, mixed-methods data collected from the MANA study OPC group will be analysed to elucidate when OPC works (outcomes), for whom, how (mechanisms) and under what circumstances (contexts). This will culminate in the synthesis of Intervention-Actor Context-Mechanism-Outcome configurations. Descriptive analyses will be reported for quantitative measures of treatment fidelity (OPC-Fidelity Measure), caregiver emotional response to OPC (Session Rating Scale) preintervention emotional state (Depression Stress and Anxiety Scale) and client outcomes (Canadian Occupational Performance Measure). Reflexive thematic analysis will be undertaken to analyse realist interviews with therapists who implemented OPC above and below fidelity thresholds and culturally focused interviews with clients of Māori or Pasifika ethnicity, informing understanding of the contexts influencing therapists' implementation of OPC with fidelity, and the mechanisms triggered within therapists or caregivers to elicit a response to the intervention. The MANA study trial outcomes will be reported separately.

ETHICS AND DISSEMINATION

Ethical approval for this study was granted by the New Zealand Health and Disability Ethics Committee (20/STH/93). In all participating jurisdictions local area approval was obtained, involving a process of local Māori consultation. Results will be disseminated to all participants, and more broadly to clinicians and policy-makers through conference presentations and peer-reviewed journal publications, which will inform decision-making about resourcing and supporting effective delivery of OPC to optimise outcomes for children and caregivers.

TRIAL REGISTRATION NUMBER

ACTRN12621000519853.

"It doesn't feel like exercise": a realist process evaluation of factors that support long-term attendance at dance classes designed for healthy ageing

Introduction

Dance can positively impact older people's health and wellbeing across cultures and socioeconomic groups, countering age-related physical, sensorimotor and cognitive decline.

Background/objectives

The RIPE (Really Is Possible for Everyone) Dance program aims to improve older people's physical, mental, cognitive and social wellbeing by integrating engaging dance sequences with evidence-based fall prevention exercises. We sought to identify what mechanisms support observed long-term participation in this program, including by people living with challenging health conditions.

Methods

Following a realist evaluation approach, we co-developed and tested program theories iteratively with participant interviewees (n = 20), dance teachers (n = 2) and via observation of a dance class. Initial data were dual-coded and emergent findings were interrogated by the research team. Findings were organised to express Program activities + Context + Mechanism = Process outcomes configurations.

Results

We identified four program theories comprising 14 mechanisms which explained long-term attendance: 1. RIPE Dance benefits my body and mind (trust in the program, belief in health benefits), 2. RIPE Dance helps me feel good about myself (self-efficacy, pride in achievement, psychological safety, defying expectations, feeling valued), 3. RIPE Dance creates camaraderie (social connection, mutual support, rapport with the teacher), and 4. RIPE Dance is uplifting (raised spirits, fun, synchrony, musical reactivity).

Conclusion

The RIPE Dance program provides effective and enjoyable 'exercise in disguise' for older people with diverse mobility profiles.

Significance/implications

This research confirms that participation in dance can contribute significantly to healthy, happy ageing. Findings detail program activities that were most strongly associated with process outcomes, offering guidance for further program development, implementation and scaling up.

The Value of Mind-Body Connection in Physical Activity for Older People

Exercise that targets balance and strength is proven to prevent falls in older age. The Successful AGEing yoga trial is the first large randomized controlled trial to assess the impact of yoga on falls in people aged ≥60 years. We conducted a realist process evaluation to explain the strong participant engagement observed using interviews (21 participants and three yoga instructors) and focus groups (12 participants and four yoga instructors). Results showed that relaxation, breathing, and yoga's mind-body connection created a satisfying internal focus on bodily sensation which was valued by participants. The mechanisms of mindfulness and embodiment appeared to facilitate this. Mindfulness and embodiment are also linked to, and enhance engagement with, other forms of physical activity. By focusing creatively on these mechanisms, we can develop a range of programs that target improvements in physical and mental health (including reducing falls and fear of falls) and appeal to older people.

'My words become my hands': Yoga instructors' experiences of adapting teleyoga in the SAGE fall prevention trial-A qualitative analysis

Objective

This research identifies practical lessons regarding the delivery of teleyoga. Our objectives are to (1) describe challenges and opportunities experienced by yoga instructors when moving the Successful AGEing (SAGE) yoga programme online, and (2) describe how yoga instructors adapted to manage the challenges and leverage opportunities presented by teleyoga.

Methods

This study is a secondary analysis of the data from a previous realist process evaluation of the SAGE yoga trial. The SAGE yoga trial is testing the effect of a yoga-based exercise programme on falls among 700 community-dwelling people aged 60+ years. We draw on focus groups and interviews with four SAGE yoga instructors which we analysed using previously developed programme theories combined with inductive coding and an analytical workshop.

Results

The concerns of the yoga instructors about teleyoga can be characterised into four broad issues: threats to safety, altered interpersonal dynamics, facilitating mind-body connection and difficulties with technology. The SAGE instructors identified eight modifications they used to manage these challenges: a 1:1 participant interview prior to programme commencement, more descriptive verbal instructions, increased focus on interoception, increased attention and support, slower more structured class flow, simplifying poses, adapting the studio environment and IT support.

Conclusions

We have created a typology of strategies for addressing challenges in the delivery of teleyoga for older people. As well as maximising engagement with teleyoga, these manageable strategies could be applied by other instructors to a wide range of telehealth classes, improving the uptake and adherence of beneficial online programmes and services.

"I wouldn't have joined if it wasn't online": understanding older people's engagement with teleyoga classes for fall prevention

BACKGROUND

Yoga-based exercise is a promising strategy for promoting healthy ageing, with the potential to reduce falls and increase physical, cognitive and psychological wellbeing. Teleyoga (real-time yoga provided via interactive videoconferencing) can deliver yoga programs at scale, potentially reducing costs, increasing convenience, and reaching people who cannot attend studio-based classes. But better understanding of how older people perceive and engage with teleyoga is needed to optimise its design, implementation and promotion.

METHODS

This study built on a previous realist process evaluation of the SAGE yoga trial which is testing the effect of a yoga-based exercise program on falls among 700 community-dwelling people aged 60 + years. In this second phase of evaluation we conducted focus groups with participants who had completed the SAGE program online and with the yoga instructors who were delivering it. We also conducted interviews with participants who had withdrawn from the trial. Six program theories developed in the earlier evaluation provided a framework for data analysis, supplemented by inductive coding and an analytical workshop.

RESULTS

Participants described physical and psychological benefits from the SAGE teleyoga program. While noting that teleyoga cannot facilitate hands-on correction or the same quality of observation or interaction as studio classes, participants were highly appreciative of their yoga instructors' strategies for optimising visibility, instruction, social connection and therapeutic alliance, and for adapting to constrained home environments. Some participants argued that teleyoga was superior to studio classes due to its accessibility and convenience, its lower exposure to potential embarrassment about physical appearance or capabilities, and a reduced sense of peer competition and distraction. Our program theories applied across studio and online modes of delivery.

CONCLUSION

Teleyoga increases accessibility for people in diverse locations and circumstances; it provides a psychologically safer space which combats self-consciousness and unwanted competitiveness; it may enhance embodiment and mindfulness for some; and it has the potential to be offered relatively cheaply at scale which could support free or reduced price classes for people on low incomes and pensions, thereby encouraging a wider population to engage in yoga for healthy ageing and fall prevention.

What helps older people persevere with yoga classes? A realist process evaluation of a COVID-19-affected yoga program for fall prevention

Background

Falls among older people are a major global health concern. This process evaluation investigates the experience of participants aged 60+ in a yoga program aimed at preventing falls which transitioned from studio-based classes to online classes in response to COVID-19 restrictions. We sought to understand how the Successful AGEing (SAGE) yoga program functioned in both settings and as a hybrid program, and to explain why it worked well for most participants.

Methods

Realist process evaluation was used to explore the factors that facilitated a successful transition for most participants, and to consider why it did not work for a minority. This approach develops program theories that describe which mechanisms an intervention is (or is not) activating, and how this is mediated by context to generate process outcomes. Data included interviews with participants (n = 21) and yoga instructors (n = 3), self-report feedback forms (n = 46), observation of classes and routine process measures.

Results

Factors that facilitated a successful transition for most participants included the quality of yoga instruction, the program format and inherent characteristics of yoga. Gains in transitioning online included continuity and greater convenience. Losses included perceived reduction in the effectiveness of yoga instruction. There were greater challenges for people struggling with pain and in disadvantageous home environments. We identified six program theories configured around 16 mechanisms: 1. It’s worth the effort and 2. In expert hands (these had the same mechanisms: value expectancy, therapeutic alliance and achievement/mastery), 3. A communal experience (these mechanisms were shared experience, social connection, social comparison and peer checking), 4. Putting yoga within reach (accessibility, convenience, gratitude), 5. Building yoga habits (purposeful structure, momentum, accountability and continuity), and 6. Yoga’s special properties (embodiment and mindfulness).

Conclusions

This study showed that online delivery of a yoga program for people aged 60+ retained much of the value of a face-to-face program for the majority of participants, and increased the value for some. The structured, communal nature of an organised group program delivered by a skilled instructor, together with yoga’s intrinsic focus on mindfulness, facilitated continued engagement and perceived health benefits, despite the change in delivery mode.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-12818-5.

Do the implementation processes of a school-based daily physical activity (DPA) program vary according to the socioeconomic context of the schools? a realist evaluation of the Active at school program

Background

Less than half of Canadian children meet the Canadian Physical Activity (PA) Guidelines, and the proportion is even lower among children living in underprivileged neighbourhoods. Regular PA supports physical, cognitive, and psychological/social health among school-aged children. Successful implementation of school-based daily physical activity (DPA) programs is therefore important for all children and crucial for children who attend schools in lower socioeconomic settings. The purpose of this study is to uncover what worked, for whom, how, and why during the three-year implementation period of a new “flexible” DPA program, while paying particular attention to the socioeconomic setting of the participating schools.

Methods

This study is a realist evaluation using mixed methods for data generation. Longitudinal data were collected in 415 schools once a year during the three-year implementation period of the program using questionnaires. Data analysis was completed in three steps and included qualitative thematic analysis using a mixed inductive and deductive method and chi-square tests to test and refine context-mechanism-outcome (CMO) configurations.

Results

Giving the school teams autonomy in the choice of strategies appropriate to their context have allowed schools to take ownership of program implementation by activating a community empowerment process, which resulted in a cultural shift towards a sustainable DPA provision in most settings. In rural underprivileged settings, the mobilization of local resources seems to have successfully created the conditions necessary for implementing and maintaining changes in practice. In disadvantaged urban settings, implementing local leadership structures (leader, committee, and meetings) provided pivotal assistance to members of the school teams in providing new DPA opportunities. However, without continued external funding, those schools seem unable to support local leadership structures on their own, jeopardizing the sustainability of the program for children living in disadvantaged urban areas.

Conclusion

By exploring CMO configurations, we have been able to better understand what worked, for whom, how and why during the three-year implementation period of the Active at School! program. When implementing DPA policies, decision makers should consider adjusting resource allocations to meet the actual needs of schools from different backgrounds to promote equal PA opportunities for all children.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-12797-7.