Translating global evidence into local implementation through technical assistance: a realist evaluation of the Bloomberg philanthropies initiative for global Road safety

BACKGROUND

Traffic-related crashes are a leading cause of premature death and disability. The safe systems approach is an evidence-informed set of innovations to reduce traffic-related injuries and deaths. First developed in Sweden, global health actors are adapting the model to improve road safety in low- and middle-income countries via technical assistance (TA) programs; however, there is little evidence on road safety TA across contexts. This study investigated how, why, and under what conditions technical assistance influenced evidence-informed road safety in Accra (Ghana), Bogotá (Colombia), and Mumbai (India), using a case study of the Bloomberg Philanthropies Initiative for Global Road Safety (BIGRS).

METHODS

We conducted a realist evaluation with a multiple case study design to construct a program theory. Key informant interviews were conducted with 68 government officials, program staff, and other stakeholders. Documents were utilized to trace the evolution of the program. We used a retroductive analysis approach, drawing on the diffusion of innovation theory and guided by the context-mechanism-outcome approach to realist evaluation.

RESULTS

TA can improve road safety capabilities and increase the uptake of evidence-informed interventions. Hands-on capacity building tailored to specific implementation needs improved implementers' understanding of new approaches. BIGRS generated novel, city-specific analytics that shifted the focus toward vulnerable road users. BIGRS and city officials launched pilots that brought evidence-informed approaches. This built confidence by demonstrating successful implementation and allowing government officials to gauge public perception. But pilots had to scale within existing city and national contexts. City champions, governance structures, existing political prioritization, and socio-cultural norms influenced scale-up.

CONCLUSION

The program theory emphasizes the interaction of trust, credibility, champions and their authority, governance structures, political prioritization, and the implement-ability of international evidence in creating the conditions for road safety change. BIGRS continues to be a vehicle for improving road safety at scale and developing coalitions that assist governments in fulfilling their role as stewards of population well-being. Our findings improve understanding of the complex role of TA in translating evidence-informed interventions to country-level implementation and emphasize the importance of context-sensitive TA to increase impact.

Protocol for an interview-based method for mapping mental models using causal-loop diagramming and realist interviewing

Causal-loop diagramming, a method from system dynamics, is increasingly used in evaluation to describe individuals' understanding of how policies or programs do or could work ("mental models"). The use of qualitative interviews to inform model development is common, but guidance for how to design and conduct these interviews to elicit causal information in participant mental models is scant. A key strength of semi-structured qualitative interviews is that they let participants speak freely; they are not, however, designed to elicit causal information. Moreover, much of human communication about mental models-particularly larger causal structures such as feedback loops-is implicit. In qualitative research, part of the skill and art of effective interviewing and analysis involves listening for information that is expressed implicitly. Similarly, a skilled facilitator can recognize and inquire about implied causal structures, as is commonly done in group model building. To standardize and make accessible these approaches, we have formalized a protocol for designing and conducting semi-structured interviews tailored to eliciting mental models using causal-loop diagramming. We build on qualitative research methods, system dynamics, and realist interviewing. This novel, integrative method is designed to increase transparency and rigor in the use of interviews for system dynamics and has a variety of potential applications.

A realist evaluation of devices used for the prevention of heel pressure ulcers: An ethnographic study of clinical practice

BACKGROUND

It is known that heel offloading devices are widely used in clinical practice for the prevention of heel pressure ulcers, even though there is a lack of robust, good quality evidence to inform their use.

OBJECTIVE

To explore how and why heel offloading devices are used (or not used) and reasoning behind their use in population at high risk of developing heel pressure ulcers.

METHODS

An ethnographic study was conducted as part of a realist evaluation in three orthopaedic wards in a large English hospital. Twelve observations took place, with 49 h and 35 min of patient care observed. A total of 32 patients were observed and 19 members of the nursing team were interviewed and in-depth interviews with the three ward managers were conducted.

RESULTS

Although the focus of the study was on offloading devices, constant low pressure heel specific devices were also observed in use for pressure ulcer prevention, whilst offloading devices were perceived to be for higher risk patients or those already with a heel pressure ulcer. Nursing staff viewed leadership from the ward manager and the influence of the Tissue Viability Nurse Specialists as key mechanisms for the proactive use of devices.

CONCLUSIONS

This study informs trial design as it has identified that a controlled clinical trial of both types of heel specific devices is required to inform evidence-based practice. Involving the ward managers and Tissue Viability Nurse Specialists during set up phase for clinical equipoise could improve recruitment. Tweetable abstract How, for whom, and in what circumstances do devices work to prevent heel pressure ulcers? Observations of clinical practice.

Centring participant experience: a realist evaluation of a menstruator-friendly facility design project in a refugee settlement, Lebanon

INTRODUCTION

Menstrual health in humanitarian contexts is a neglected topic. Its taboo nature presents difficulties for participants in menstrual health projects in these particularly challenging settings. Namely, their experiences may be concealed or overlooked in projects that are typically outcome focused. Realist Evaluation is a useful method to unearth and explore the hidden mechanisms and their causes, which lead to positive or negative participant experiences. The authors have applied this approach to a robust humanitarian menstrual health project to explore how to centre the emotional wellbeing of participants at all stages: prior to, during, and post-participation.

STUDY SETTING

The project studied was led by the International Federation of Red Cross and Red Crescent Societies who piloted their adaptable manual for menstruator-friendly water, sanitation and hygiene (WaSH) facility design in humanitarian contexts. It was conducted by the Lebanese Red Cross in an informal tented settlement hosting Syrian refugees in Qaa, Lebanon.

METHODS

The authors collected interview and focus group data on the contextual factors and processes within the project from nine project staff and 16 settlement inhabitants. They used a realist process of theory development, testing, and consolidation to understand how and under what circumstances the project inputs affected participants' wellbeing.

RESULTS

The contextual factors and causal mechanisms promoting participant experience comprised individual (choices influencing and experience during participation), interpersonal (group dynamics and the role of non-menstruators), and organisational (expertise and knowledge, relationship to participants and cultural differences) factors.

IMPLICATIONS

The research uses a case study from a renowned humanitarian organisation who provided a well-delivered project in a conducive environment to explore the mechanisms and contexts that can promote wider learning and refine understanding and programming in this under-researched and -theorised space. Specifically, it informs which contextual factors and project inputs must be present within a menstrual health project to ensure participant satisfaction whilst efficiently delivering well-designed menstruator-friendly WaSH facilities.

The use of written guides to empower breast cancer survivors in their management of chronic pain: A realist evaluation

OBJECTIVE

Many breast cancer survivors experience long-term complaints following treatment, such as pain, which are often not addressed in a sufficient way. To empower survivors in talking about their pain and related complaints and in searching for appropriate support when needed, we developed two written guides. With this study, we aimed to pilot test the guides and gain insight into the implementation process and influencing mechanisms through the perspective of a realist evaluation.

METHODS

Nine survivors were interviewed at two time points (i.e., post-intervention and at three-month follow-up). The data were thematically analysed and categorized into a context-implementation-mechanisms-outcomes hypothesis.

RESULTS

The guides empowered the participants to discuss pain with medical specialists and initiate support-seeking behaviour through underlying mechanisms such as awareness, acknowledgment, hope, reduced isolation, and motivation. Nonetheless, mechanisms and outcomes differed according to a survivor's unique context.

CONCLUSION

Written guides can offer a feasible way to empower survivors in their management of chronic pain. However, a one-size-fits-all approach is not desirable and other strategies might be necessary.

PRACTICE IMPLICATIONS

It is pivotal to engage survivors as well as professionals in adopting new interventions. As such, the role of nurses in introducing and endorsing the guides should be further explored.

The role of provincial health administration in supporting district health management teams in the Democratic Republic of Congo: eliciting an initial programme theory of a realist evaluation

BACKGROUND

In 2006, the Ministry of Health in the Democratic Republic of Congo designed a strategy to strengthen the health system by developing health districts. This strategy included a reform of the provincial health administration to provide effective technical support to district health management teams in terms of leadership and management. The provincial health teams were set up in 2014, but few studies have been done on how, for whom, and under what circumstances their support to the districts works. We report on the development of an initial programme theory that is the first step of a realist evaluation seeking to address this knowledge gap.

METHODS

To inform the initial programme theory, we collected data through a scoping review of primary studies on leadership or management capacity building of district health managers in sub-Saharan Africa, a review of policy documents and interviews with the programme designers. We then conducted a two-step data analysis: first, identification of intervention features, context, actors, mechanisms and outcomes through thematic content analysis, and second, formulation of intervention-context-actor-mechanism-outcome (ICAMO) configurations using a retroductive approach.

RESULTS

We identified six ICAMO configurations explaining how effective technical support (i.e. personalised, problem-solving centred and reflection-stimulating) may improve the competencies of the members of district health management teams by activating a series of mechanisms (including positive perceived relevance of the support, positive perceived credibility of provincial health administration staff, trust in provincial health administration staff, psychological safety, reflexivity, self-efficacy and perceived autonomy) under specific contextual conditions (including enabling learning environment, integration of vertical programmes, competent public health administration staff, optimal decision space, supportive work conditions, availability of resources and absence of negative political influences).

CONCLUSIONS

We identified initial ICAMO configurations that explain how provincial health administration technical support for district health management teams is expected to work, for whom and under what conditions. These ICAMO configurations will be tested in subsequent empirical studies.

Conditions for adherence to videoconference-based programs promoting adapted physical activity in cancer patients: a realist evaluation

BACKGROUND

Although moderate physical activity (PA) during cancer treatment has been associated with improved quality of life, reduced side effects, and even lower mortality, many barriers to successful implementation remain. Digital technology has been perceived as an effective lever for overcoming access and motivational issues but few studies have been performed to confirm this assumption. The "VISIO-AJUST" study explored the factors affecting the conditions of adherence to video-conference-based PA programs in patients undergoing cancer treatment.

METHODS

The VISIO-AJUST study was based on a qualitative successive case approach, guided by the principles of the realist evaluation, and applied to two French programs of PA, following three main steps: (1) Identification of factors likely to influence conditions of patients' adherence; (2) Elaboration and testing of explanatory "Context-Mechanism-Outcome (CMO)" configurations; (3) Refinement of CMO configurations, in order to understand what, how, for whom, and under what circumstances video-conference-based PA programs work.

RESULTS

Five main CMO configurations were found to be associated with adherence to video-conferencing-based PA programs, promoting (i) accessibility and a supportive environment for adapted physical activity, (ii) a setting conducive to sociability despite distance, (iii) Confidence and security of practice, (iv) a combination of several motivational levers in favor of continuity of effort and progress, (v) regularity of the sessions, progressiveness in the effort and evaluation of progress as a basis for the adoption of a regular physical practice.

CONCLUSION

This study provides original insights into the use of videoconferencing to enable patients to improve their PA during cancer treatment. Future research with long-term follow-up would allow for a better understanding of the key conditions promoting sustainable behavior change.

Key elements and contextual factors that influence successful implementation of large-system transformation initiatives in the New Zealand health system: a realist evaluation

BACKGROUND

Despite three decades of policy initiatives to improve integration of health care, delivery of health care in New Zealand remains fragmented, and health inequities persist for Māori and other high priority populations. An evidence base is needed to increase the chances of success with implementation of large-system transformation (LST) initiatives in a complex adaptive system.

METHODS

This research aimed to identify key elements that support implementation of LST initiatives, and to investigate contextual factors that influence these initiatives. The realist logic of enquiry, nested within the macro framing of complex adaptive systems, formed the overall methodology for this research and involved five phases: theory gleaning from a local LST initiative, literature review, interviews, workshop, and online survey. NVivo software programme was used for thematic analysis of the interview, workshop, and the survey data. We identified key elements and explained variations in success (outcomes) by identifying mechanisms triggered by various contexts in which LST initiatives are implemented.

RESULTS

The research found that a set of 10 key elements need to be present in the New Zealand health system to increase chances of success with implementation of LST initiatives. These are: (i) an alliancing way of working; (ii) a commitment to te Tiriti o Waitangi; (iii) an understanding of equity; (iv) clinical leadership and involvement; (v) involved people, whānau, and community; (vi) intelligent commissioning; (vii) continuous improvement; (viii) integrated health information; (ix) analytic capability; and (x) dedicated resources and time. The research identified five contextual factors that influenced implementation of LST initiatives: a history of working together, distributed leadership from funders, the maturity of Alliances, capacity and capability for improvement, and a continuous improvement culture. The research found that the key mechanism of trust is built and nurtured over time through sharing of power by senior health leaders by practising distributed leadership, which then creates a positive history of working together and increases the maturity of Alliances.

DISCUSSION

Two authors (KMS and PBJ) led the development and implementation of the local LST initiative. This prior knowledge and experience provided a unique perspective to the research but also created a conflict of interest and introduced potential bias, these were managed through a wide range of data collection methods and informed consent from participants. The evidence-base for successful implementation of LST initiatives produced in this research contains knowledge and experience of senior system leaders who are often in charge of leading these initiatives. This evidence base enables decision makers to make sense of complex processes involved in the successful implementation of LST initiatives.

CONCLUSIONS

Use of informal trust-based networks provided a critical platform for successful implementation of LST initiatives in the New Zealand health system. Maturity of these networks relies on building and sustaining high-trust relationships among the network members. The role of local and central agencies and the government is to provide the policy settings and conditions in which trust-based networks can flourish.

OTHER

This study was approved by the Victoria University of Wellington Human Ethics Committee (Ethics Approval Number 27,356). The research was supported by the Victoria University of Wellington research grant (222,809) and from the University of Auckland Department of Medicine research fund (H10779).

Audit and feedback to reduce unwarranted clinical variation at scale: a realist study of implementation strategy mechanisms

BACKGROUND

Unwarranted clinical variation in hospital care includes the underuse, overuse, or misuse of services. Audit and feedback is a common strategy to reduce unwarranted variation, but its effectiveness varies widely across contexts. We aimed to identify implementation strategies, mechanisms, and contextual circumstances contributing to the impact of audit and feedback on unwarranted clinical variation.

METHODS

Realist study examining a state-wide value-based healthcare program implemented between 2017 and 2021 in New South Wales, Australia. Three initiatives within the program included audit and feedback to reduce unwarranted variation in inpatient care for different conditions. Multiple data sources were used to formulate the initial audit and feedback program theory: a systematic review, realist review, program document review, and informal discussions with key program stakeholders. Semi-structured interviews were then conducted with 56 participants to refute, refine, or confirm the initial program theories. Data were analysed retroductively using a context-mechanism-outcome framework for 11 transcripts which were coded into the audit and feedback program theory. The program theory was validated with three expert panels: senior health leaders (n = 19), Agency for Clinical Innovation (n = 11), and Ministry of Health (n = 21) staff.

RESULTS

The program's audit and feedback implementation strategy operated through eight mechanistic processes. The strategy worked well when clinicians (1) felt ownership and buy-in, (2) could make sense of the information provided, (3) were motivated by social influence, and (4) accepted responsibility and accountability for proposed changes. The success of the strategy was constrained when the audit process led to (5) rationalising current practice instead of creating a learning opportunity, (6) perceptions of unfairness and concerns about data integrity, 7) development of improvement plans that were not followed, and (8) perceived intrusions on professional autonomy.

CONCLUSIONS

Audit and feedback strategies may help reduce unwarranted clinical variation in care where there is engagement between auditors and local clinicians, meaningful audit indicators, clear improvement plans, and respect for clinical expertise. We contribute theoretical development for audit and feedback by proposing a Model for Audit and Feedback Implementation at Scale. Recommendations include limiting the number of audit indicators, involving clinical staff and local leaders in feedback, and providing opportunities for reflection.

Evaluating the implementation of a person-centred transition programme for adolescents and young adults with long-term conditions: the role of context and organisational behaviour

PURPOSE

Drawing on the experiences of healthcare professionals in one paediatric hospital, this paper explores the influence of context and organisational behaviour on the implementation of a person-centred transition programme for adolescents and young adults (AYA) with long-term conditions.

DESIGN/METHODOLOGY/APPROACH

A single embedded qualitative case study design informed by a realist evaluation framework, was used. Participants who had experience of implementing the transition programme were recruited from across seven individual services within the healthcare organisation. The data were gathered through semi-structured interviews (n = 20) and analysed using thematic analysis.

FINDINGS

Implementation of the transition programme was influenced by the complex interaction of macro, meso and micro processes and contexts. Features of organisational behaviour including routines and habits, culture, organisational readiness for change and professional relationships shaped professional decision-making around programme implementation.

ORIGINALITY/VALUE

There exists a significant body of research relating to the role of context and its influence on the successful implementation of complex healthcare interventions. However, within the area of healthcare transition there is little published evidence on the role that organisational behaviour and contextual factors play in influencing transition programme implementation. This paper provides an in-depth understanding of how organisational behaviour and contextual factors affect transition programme implementation.

Intentional rounding: a realist evaluation using case studies in acute and care of older people hospital wards

BACKGROUND

In response to concerns about high hospital mortality rates, patient and carer complaints, a Mid Staffordshire NHS Foundation Trust public inquiry was conducted at the request of the UK government. This inquiry found serious failures in the quality of basic care provided and as a consequence, recommended that patients should have more regular visits, organised at predictable times from nursing staff. Intentional rounding, also known as nursing ward rounds, was widely adopted to meet this need.

OBJECTIVE

To test, refine or refute eight programme theories to understand what works, for whom, and in what circumstances.

SETTING

Six wards (older people and acute wards) in three NHS trusts in England.

PARTICIPANTS

Board level and senior nursing managers (N = 17), nursing ward staff (N = 33), allied health and medical professionals (N = 26), patients (N = 34) and relatives (N = 28) participated in an individual, in-depth interview using the realist method. In addition, ward-based nurses (N = 39) were shadowed whilst they conduced intentional rounds (240 rounds in total) and the direct care of patients (188 h of patient care in total) was observed.

METHODS

The mixed methods design included: Phase (1) Theory development - A realist synthesis was undertaken to identify any programme theories which were tested, refined and/or refuted, using data from phases 2 and 3; Phase (2) A survey of all English NHS acute Trusts; Phase (3) Six case studies of wards involving realist interviews, shadowing and non-participant observations, analysis of ward outcome and cost data; and Phase (4) Synthesis of findings from phases 1, 2 and 3.

RESULTS

The realist synthesis identified eight programme theories of intentional rounding: 'Consistency and comprehensiveness', 'Accountability', 'Visibility of nurses', 'Anticipation', 'Allocated time to care', 'Nurse-patient relationships', 'Multi-disciplinary teamwork and communication' and 'Patient empowerment'. Key findings showed that of the original eight programme theories of intentional rounding, only two partially explained how the intervention worked ('Consistency and comprehensiveness' and 'Accountability'). Of the remaining six programme theories, the evidence for two was inconclusive ('Visibility of nurses' and 'Anticipation') and there was no evidence for four ('Allocated time to care'; 'Nurse-patient relationships'; 'Multi-disciplinary teamwork and communication'; and 'Patient empowerment').

CONCLUSIONS

This first theory-informed evaluation of intentional rounding, demonstrates that the effectiveness of intentional rounding in the English healthcare context is very weak. Furthermore, the evidence collected in this study has challenged and refuted some of the underlying assumptions about how intentional rounding works. This study has demonstrated the crucial role context plays in determining the effectiveness of an intervention and how caution is needed when implementing interventions developed for the health system of one country into another.

How do navigation programs address the needs of those living in the community with advanced, life-limiting Illness? A realist evaluation of programs in Canada

BACKGROUND

We sought to identify innovative navigation programs across Canadian jurisdictions that target their services to individuals affected by life-limiting illness and their families, and articulate the principal components of these programs that enable them to address the needs of their clients who are living in the community.

METHODS

This realist evaluation used a two-phased approach. First, we conducted a horizon scan of innovative community-based navigation programs across Canadian jurisdictions to identify innovative community-based navigation programs that aim to address the needs of community-dwelling individuals affected by life-limiting illness. Second, we conducted semi-structured interviews with key informants from each of the selected programs. Informants included individuals responsible for managing and delivering the program and decision-makers with responsibility and/or oversight of the program. Analyses proceeded in an iterative manner, consistent with realist evaluation methods. This included iteratively developing and refining Context-Mechanism-Outcome (CMO) configurations, and developing the final program theory.

RESULTS

Twenty-seven navigation programs were identified from the horizon scan. Using specific eligibility criteria, 11 programs were selected for subsequent interviews and in-depth examination. Twenty-three participants were interviewed from these programs, which operated in five Canadian provinces. The programs represented a mixture of community (non-profit or volunteer), research-initiated, and health system programs. The final program theory was articulated as: navigation programs can improve client outcomes if they have supported and empowered staff who have the time and flexibility to personalize care to the needs of their clients.

CONCLUSIONS

The findings highlight key principles (contexts and mechanisms) that enable navigation programs to develop client relationships, personalize care to client needs, and improve client outcomes. These principles include staff (or volunteer) knowledge and experience to coordinate health and social services, having a point of contact after hours, and providing staff (and volunteers) time and flexibility to develop relationships and respond to individualized client needs. These findings may be used by healthcare organizations - outside of navigation programs - to work towards more person-centred care.

"You can create a little bit more closure in your own story when someone really connects with it": exploring how involvement in youth peer support work can promote peer development

BACKGROUND

Peer support relates to the provision of social/emotional support that is delivered by individuals with lived experience of a key characteristic that is shared with clients. Although the main objective of peer support is to enhance client outcomes, through their involvement, peers derive a secondary benefit to their own personal development. This study applied a hybrid participatory-realist approach to identify what works, for whom, why and in what circumstances within the LOFT Transitional Age Youth (TAY) peer services. This paper presents findings related to the processes and possible benefits of being involved in peer work for the peer supporters themselves.

METHODS

Semi-structured interviews and focus groups were completed with peer and non-peer staff from the TAY program. A qualitative analysis applied a retroductive approach that involved both inductive and deductive processes to identify relevant themes.

RESULTS

Four program theories and one over-arching context were identified through the analyses. Program theories were related to: (1) enhancing self-efficacy and self-determination through peer involvement in program design, (2) increasing peer resiliency and self-care through effective supervision, (3) developing professional skills and opportunities for career advancement through peer practice and (4) overcoming stigma through the recognition of the value of peer lived experience.

CONCLUSIONS

Peer practice holds significant potential for the enhancement of the mental health system as well as to increase our understanding of stigma. The findings from this study offer critical new insights into the dynamics of how professional peer practice can support the personal development of youth peers and how programming can be intentionally designed to enhance these benefits.

How do health system factors (funding and performance) impact on access to healthcare for populations experiencing homelessness: a realist evaluation

BACKGROUND

People experiencing long-term homelessness face significant difficulties accessing appropriate healthcare at the right time and place. This study explores how and why healthcare performance management and funding arrangements contribute to healthcare accessibility or the lack thereof using long-term homeless adults as an example of a population experiencing social exclusion.

METHODS

A realist evaluation was undertaken. Thirteen realist interviews were conducted after which data were transcribed, coded, and analysed.

RESULTS

Fourteen CMOCs were created based on analysis of the data collected. These were then consolidated into four higher-level CMOCs. They show that health systems characterised by fragmentation are designed to meet their own needs above the needs of patients, and they rely on practitioners with a special interest and specialised services to fill the gaps in the system. Key contexts identified in the study include: health system fragmentation; health service fragmentation; bio-medical, one problem at a time model; responsive specialised services; unresponsive mainstream services; national strategy; short health system funding cycles; and short-term goals.

CONCLUSION

When health services are fragmented and complex, the needs of socially excluded populations such as those experiencing homelessness are not met. Health systems focus on their own metrics and rely on separate actors such as independent NGOs to fill gaps when certain people are not accommodated in the mainstream health system. As a result, health systems lack a comprehensive understanding of the needs of all population groups and fail to plan adequately, which maintains fragmentation. Policy makers must set policy and plan health services based on a full understanding of needs of all population groups.

Focusing a realist evaluation of peer support for paediatric mental health

OBJECTIVE

Mental health problems are a leading and increasing cause of health-related burden in children across the world. Peer support interventions are increasingly used to meet this need using the lived experience of people with a history of mental health problems. However, much of the research underpinning this work has focused on adults, leaving a gap in knowledge about how these complex interventions may work for different children in different circumstances. Realist research may help us to understand how such complex interventions may trigger different mechanisms to produce different outcomes in children. This paper reports on an important first step in realist research, namely the construction of an embryonic initial programme theory to help 'focus' realist evaluation exploring how children's peer support services work in different contexts to produce different outcomes in the West Midlands (UK).

METHODS

A survey and preliminary semi-structured realist interviews were conducted with 10 people involved in the delivery of peer support services. Realist analysis was carried out to produce context-mechanism-outcome configurations (CMOC).

RESULTS

Analysis produced an initial programme theory of peer support for children's mental health. This included 12 CMOCs. Important outcomes identified by peer support staff included hope, service engagement, wellbeing, resilience, and confidence; each generated by different mechanisms including contextualisation of psychoeducation, navigating barriers to accessing services, validation, skill development, therapeutic relationship, empowerment, and reducing stigma.

CONCLUSION

These data lay the groundwork for designing youth mental health realist research to evaluate with nuance the complexities of what components of peer support work for whom in varying circumstances.

What works in advocating for food advertising policy change across an english region - a realist evaluation

BACKGROUND

With increasing recognition of the role of commercial determinants of health, local areas in England have sought to restrict the advertising of products high in fat, salt and sugar (HFSS) on council-owned spaces, as part of wider strategies to reduce obesity. While there is some evidence of the impact of such policy change on behaviour, little is known about what works in the process of implementing this policy change.

METHODS

Guided by a realist evaluation framework that explores the interaction between context, mechanism and outcomes, this study aims to investigate the factors that influence the restriction of outdoor advertising of HFSS products in one region in England. It refines a programme theory co-produced with stakeholders from 14 local authorities within a region and uses multiple data sources from each area with an in-depth examination of four case study sites. Data sources include longitudinal realist interviews, focus groups and surveys with policy advocates and policy stakeholders. Data were analysed retroductively to understand the causal link between context, mechanism and outcomes.

RESULTS

Outcomes were driven by five dominant mechanisms: a strategic and staggered approach to stakeholder engagement, gathering intelligence, identifying policy champions, building relationships, reframing the issue; and two secondary mechanisms of amplifying the issue and increasing public will. These led to varied outcomes with no changes in formal policy position within the evaluation period but draft policy guidance in place and changes in political will demonstrated. Dominant context factors influencing change included having a named and resourced policy advocate in place supported by an external Community of Improvement and having existing aligned local objectives. Organisational complexity and change, financial concerns, lack of local examples, ideological positions and the pandemic were also influencing contextual factors.

CONCLUSION

Effecting policy change in this area requires the commitment of an extended period and the valuing of short-term policy outcomes, such as increasing political will. The importance of a resourced and well-supported policy advocate to lead this work is fundamental and the commercially sensitive nature of this policy change means that a complex interplay of mechanisms is required which may be dominated by a strategically staggered approach to stakeholder engagement.

A realist evaluation of a London general practitioner trainer course

Clinicians with teaching and training roles should be adequately trained and assessed. However, some debate exists as to what the nature of this training should be. Historically, a postgraduate certificate in education was a pre-requisite to becoming a GP trainer but this is changing with growing concern that such a pre-requisite might act as a deterrent to potential GP trainers. This research examines the impact of a scheme designed to provide an alternative, more practical and focused, pathway to becoming a GP trainer. We interviewed 26 course participants and stakeholders of the London GP Training Course (LGPTC), observed teaching sessions, and analysed course materials. We asked what elements of the course were and weren't effective, for whom, and under what circumstances. Here, we present a summary of our main findings - that GP trainers want to know practically, not theoretically, how to be a trainer; formative assessment boosts trainees' confidence in their own skills and abilities; short, practical GP training courses can help enhance the numbers of GP trainers; important questions remain about the role and value of educational theory in education faculty development.

Initial programme theory for community-based ART delivery for key populations in Benue State, Nigeria: a realist evaluation study

BACKGROUND

The community-based antiretroviral therapy delivery (CBART) model was implemented in Benue State in Nigeria to increase access of key populations living with HIV (KPLHIV) to antiretroviral treatment. Key populations (KP) are female sex workers, men who have sex with men, persons who inject drugs, and transgender people. Evidence shows that the CBART model for KP (KP-CBART) can improve HIV outcomes along the cascade of HIV care and treatment in sub-Saharan Africa. However, how KP-CBART works, for whom, why, and under what circumstances it generates specific outcomes are not yet clear. Therefore, the aim of this study is to identify the initial programme theory (IPT) of the KP-CBART in Benue State using a realist approach.

METHOD

The study design is exploratory and qualitative, exploring the implementation of KP-CBART. We reviewed the intervention logic framework & guidelines for the KP-CBART in Nigeria, conducted a desk review of KP-CBART in Sub-Saharan Africa (SSA) and interviewed programme managers in the Benue HIV programme between November 2021 and April 2022. Findings were synthesized using the Context-Mechanism-Outcome (CMO) heuristic tool to explain the relationship between the different types of CBART models, contextual factors, actors, mechanisms and outcomes. Using a generative causality logic (retroduction and abduction), we developed, following a realist approach, CMO configurations (CMOc), summarized as an empirically testable IPT.

RESULT

We developed 7 CMOc and an IPT of the KP-CBART. Where KPLHIV receive ART in a safe place while living in a setting of punitive laws, harassment, stigma and discrimination, KP will adhere to treatment and be retained in care because they feel safe and trust the healthcare providers. Where KPLHIV are involved in the design, planning and implementation of HIV services; medication adherence and retention in care will improve because KP clients perceive HIV services to be KP-friendly and participate in KP-CBART.

CONCLUSION

Implementation of CBART model where KPLHIV feel safe, trust healthcare providers, and participate in HIV service delivery can improve medication adherence and retention in care. This programme hypothesis will be tested and refined in the next phase of the realist evaluation of KP-CBART.

Valuing adaptive programming: A study of resilience processes and outcomes

Recent decades have seen a growth in theoretical frameworks focusing on systems, context and the dynamic interplay of multiple variables, stimulating interest in complementary research and programme evaluation methods. With resilience theory now emphasising the complex and dynamic nature of resilience capacities, processes and outcomes, resilience programming stands to benefit from approaches such as design-based research and realist research/evaluation. The aim of this collaborative (researcher/practitioner) study was to explore how such benefits can be achieved when programme theory spans individual, community and institutional outcomes, with a focus on the reciprocal processes involved in effecting change across the social system. The context of the research was a regional (Middle East and North Africa) project operating in contexts with an escalated risk of marginalised young people being drawn into illegal/harmful activity. The project's youth engagement and development approach combined participatory learning, skills training, and collective social action, adapted for diverse localities and during the COVID-19 crisis. Quantitative measures of individual and collective resilience were at the centre of a set of realist analyses evidencing systemic connections in changes to individual, collective and community resilience. Findings demonstrated the value, challenges and limitations of the applied research approach for adaptive, contextualised programming.

Methods in realist evaluation: A mapping review

Realist evaluation is becoming increasingly popular as an evaluation methodology. Its main objective is to uncover the mechanisms that lead to observed outcomes following an intervention and the contextual conditions that enabled this. The focus is on explaining why, for whom and in what circumstances an intervention works. It is a theory-driven approach and is explicitly method neutral, meaning that both quantitative and qualitative data collection methods can be used to unearth the underlying mechanisms that cause the intervention outcomes. In this review, we aim to map the methods used in realist evaluation studies, to draw lessons from the findings and to reflect on ways forward. We found that qualitative methods and interviews specifically are most commonly used in realist evaluations; that theory is often absent behind the methods and sampling techniques used; and that more innovative methods remain underexplored. We conclude the review by proposing four ways forward: (1) developing realist surveys, (2) exploring the relevance of innovative methods, (3) increasing the attention paid to sampling procedures and (4) strengthening the theory-driven nature of method. We believe that these four action points can strengthen the practice of realist evaluation and its outcomes.

A community pharmacy practice-based research network: A protocol for a realist evaluation

BACKGROUND

Community pharmacists' active participation in research is essential to build a robust, translatable evidence base. Practice-based research networks (PBRNs) have been established to support collaborative research and knowledge translation in community pharmacies. However, PBRNs' effectiveness in supporting research engagement and knowledge translation remains unexplored. A new PBRN will be implemented in southeast Queensland, Australia. This realist evaluation seeks to explain whether, how, why, for whom, in what context and over what duration the PBRN supports community pharmacists to engage in research.

OBJECTIVES

1. to generate transferable knowledge about the different circumstances in which-and the mechanisms by which-a PBRN influences research engagement outcomes for different community pharmacists, in the form of a program theory. 2. To use the program theory to develop evidence-informed recommendations for use by PBRN stakeholders.

METHODS

A realist evaluation will be conducted in four iterative phases: (1) theory development, (2) hypothesis generation, (3) observations, and (4) theory refinement. A two-year multi-method study will be conducted, including interviews with pharmacists, surveys, participatory and observational data collection. The evidence will be used to confirm, refute, and/or refine the program theory. The evaluation will adhere to the Realist And Meta-Narrative Evidence Synthesis (RAMESES) publication and quality standards.

CONCLUSIONS

The evaluation will contribute to the body of knowledge by generating a realist program theory to explain how, why, for whom, in what contexts, to what extent, in what respects, and over what duration the PBRN supports community pharmacists to engage in research. The findings will support the broader implementation of PBRNs and future network activities.

Rurality and social innovation processes and outcomes: A realist evaluation of rural social enterprise activities

Although increasingly prominent in research, policy and practice, little is known about social innovation in a rural context. To address this knowledge gap, our paper explores how rurality might affect the social innovation process. Drawing on 68 interviews carried out with beneficiaries, service providers and external stakeholders of a rural social enterprise initiative in Scotland, the paper adopts a realist evaluation theory (Pawson and Tilley, 1997) approach combined with Calò et al.'s (2019) social innovation analytical framework to identify Context-Mechanism-Outcome configurations for rural social innovation. The findings highlight that specific characteristics of rural places can act as stimuli of social innovation. Positive outcomes of a social innovation can potentially be rooted in rural peculiarity and its problematic context. Push factors, born out of necessity, lead to reactive social innovation and pull factors, derived through harnessing perceived opportunities in the environment, lead to proactive social innovation. Importantly, push factors do not undermine the establishment of social innovation - indeed, they can actually promote social innovation and strengthen its validity. The paper also shows that outcomes of the social innovation process might not be specific to rural areas. Instead, the pathway to the desired outcomes is conditioned by rural factors, shaping the contexts and mechanisms of rural social innovation. As different rural locations might have different resources to address local challenges, social innovation processes vary from one case to another, although the challenges being addressed might be similar. As such, rural social innovation policies should not be 'over prescribed'. Context creates both challenges and solutions and influences the type and form of mechanisms used to achieve a desirable social innovation outcome.

Backward mapping from outcomes: Using a realist evaluation lens to evaluate an international aid program

In this paper, a backward mapping approach to realist evaluation is used to evaluate the efficacy of an international inclusive education short course program in transforming inclusive perspectives and practices of educators from Nepal who were considered leaders in inclusive education. The program, an Australia Awards South and West Asia (AASWA) short course, delivered by an Australian university, was designed to build on the existing capacity of the participants to lead the implementation of inclusive education reforms and system enhancements within the school system in Nepal. Data collected before, during and after the program provide insights into the ways in which participants responded to the program. Our findings indicate that a deep understanding of context is needed by program planners to identify how and why particular mechanisms are triggered that lead to transformative learning of the participants. We suggest that backward mapping as a methodology is a viable realist evaluation process to evaluate complex programs such as the short-term international aid program on IE. Our paper concludes with recommendations for developers of similar programs in the future.

How do stroke early supported discharge services achieve intensive and responsive service provision? Findings from a realist evaluation study (WISE)

BACKGROUND

Stroke Early Supported Discharge (ESD) involves provision of responsive and intensive rehabilitation to stroke survivors at home and it is recommended as part of the stroke care pathway. Core components have been identified to guide the delivery of evidence-based ESD, however, service provision in England is of variable quality. The study sought to understand how and in what conditions the adoption of these components drives the delivery of responsive and intensive ESD services in real world settings.

METHODS

This qualitative study was part of a wider multimethod realist evaluation project (WISE) conducted to inform large-scale ESD implementation. Overarching programme theories and related context-mechanism-outcome configurations were used as a framework to guide data collection and analysis. Six case study sites were purposively selected; interviews and focus groups with ESD staff members were conducted and analysed iteratively.

RESULTS

We interviewed 117 ESD staff members including clinicians and service managers. Staff highlighted the role of certain core components including eligibility criteria, capacity, team composition and multidisciplinary team (MDT) coordination in achieving responsive and intensive ESD. Regardless of the geographical setting, adhering to evidence-based selection criteria, promoting an interdisciplinary skillset and supporting the role of rehabilitation assistants, allowed teams to manage capacity issues and maximise therapy time. Gaps in the stroke care pathway, however, meant that teams had to problem solve beyond their remit to cater for the complex needs of patients with severe disabilities. Adjusting MDT structures and processes was seen as key in addressing challenges posed by travel times and rural geography.

CONCLUSIONS

Despite variations in the wider service model of operation and geographical location, the adoption of core components of ESD helped teams manage the pressures and deliver services that met evidence-based standards. Findings point to a well-recognised gap in service provision in England for stroke survivors who do not meet the ESD criteria and emphasise the need for a more integrated and comprehensive stroke service provision. Transferable lessons could be drawn to inform improvement interventions aimed at promoting evidence-based service delivery in different settings.

TRIAL REGISTRATION

ISRCTN: 15,568,163, registration date: 26 October 2018.

Strengthening open disclosure after incidents in maternity care: a realist synthesis of international research evidence

BACKGROUND

Open Disclosure (OD) is open and timely communication about harmful events arising from health care with those affected. It is an entitlement of service-users and an aspect of their recovery, as well as an important dimension of service safety improvement. Recently, OD in maternity care in the English National Health Service has become a pressing public issue, with policymakers promoting multiple interventions to manage the financial and reputational costs of communication failures. There is limited research to understand how OD works and its effects in different contexts.

METHODS

Realist literature screening, data extraction, and retroductive theorisation involving two advisory stakeholder groups. Data relevant to families, clinicians, and services were mapped to theorise the relationships between contexts, mechanisms, and outcomes. From these maps, key aspects for successful OD were identified.

RESULTS

After realist quality appraisal, 38 documents were included in the synthesis (22 academic, 2 training guidance, and 14 policy report). 135 explanatory accounts were identified from the included documents (with n = 41 relevant to families; n = 37 relevant to staff; and n = 37 relevant to services). These were theorised as five key mechanism sets: (a) meaningful acknowledgement of harm, (b) opportunity for family involvement in reviews and investigations, (c) possibilities for families and staff to make sense of what happened, (d) specialist skills and psychological safety of clinicians, and (e) families and staff knowing that improvements are happening. Three key contextual factors were identified: (a) the configuration of the incident (how and when identified and classified as more or less severe); (b) national or state drivers, such as polices, regulations, and schemes, designed to promote OD; and (c) the organisational context within which these these drivers are recieived and negotiated.

CONCLUSIONS

This is the first review to theorise how OD works, for whom, in what circumstances, and why. We identify and examine from the secondary data the five key mechanisms for successful OD and the three contextual factors that influence this. The next study stage will use interview and ethnographic data to test, deepen, or overturn our five hypothesised programme theories to explain what is required to strengthen OD in maternity services.

How transdisciplinary research teams learn to do knowledge translation (KT), and how KT in turn impacts transdisciplinary research: a realist evaluation and longitudinal case study

BACKGROUND

Transdisciplinary research and knowledge translation are increasingly regarded as key concepts underpinning applied research across the health and social sciences, due to their presumed potential in addressing complex, "wicked" problems and improving the use of research in practice and policy, respectively. Despite sharing an impact mandate, the relationship between transdisciplinary research collaboration and knowledge translation remains unclear. In response, we examined the relationship between transdisciplinary collaboration and knowledge translation to generate these understandings with a view towards maximizing the impact of collaborative efforts.

METHODS

We undertook a realist evaluation and longitudinal case study of a 5-year National Health and Medical Research Council-funded Centre of Research Excellence in Transdisciplinary Frailty Research. Data were collected between February 2017 and March 2020 over three rounds of theory development, refinement and testing using interviews, observation, document review and visual elicitation as data sources. The Human Research Ethics Committee of the University of Adelaide approved this study.

RESULTS

Iterative analysis of narrative interviews and visual data led to the development of three overarching programme theories explicating the reciprocal relationship between KT understandings and transdisciplinary team process. These programme theories revolve around the concept of a network, which we define in alignment with extant theoretical literature on network mechanisms and complex networks as graphically representable networks of agents/people (nodes) joined by social relationships (links). Our findings demonstrate that under the right contextual conditions, transdisciplinary team members respond through an improved ability to (1) navigate the network, (2) negotiate the network and (3) mobilize the network.

CONCLUSIONS

This research demonstrates the reciprocity and mutually supportive relationship between transdisciplinary research and knowledge translation. Our findings suggest that embedding a collaborative knowledge translation framework and providing resources such as facilitation and distributed leadership within a transdisciplinary team can improve collaboration and support transdisciplinary research objectives.

A realist evaluation of the feasibility of a randomised controlled trial of a digital music and movement intervention for older people living in care homes

BACKGROUND

Low physical activity in care home residents brings about negative mental health consequences, such as higher levels of depression and loneliness. With advancements in communication technology, particularly during the COVID-19 pandemic, the feasibility and effectiveness of a randomised controlled trial (RCT) of a digital Physical Activity (PA) resource in care homes deserve more research attention. A realist evaluation was used to uncover influencing factors of a feasibility study implementation to inform how a digital music and movement programme would work and under what circumstances this would be most effective.

METHODS

Participants were 49 older adults (aged 65 years +) recruited across ten care homes in Scotland. Surveys were administered at baseline and post-intervention comprising psychometric questionnaires on multidimensional health markers validated in older adults with possible cognitive impairment. The intervention comprised 12 weeks of four prescribed digitally delivered movement (n = 3) and music-only (n = 1) sessions per week. An activity coordinator delivered these online resources in the care home. Post-intervention focus groups with staff and interviews with a sub-sample of participants were conducted to gain qualitative data on the acceptability of the intervention.

RESULTS

Thirty three care home residents started the intervention, but only 18 residents (84% female) completed both pre- and post-intervention assessments. Activity coordinators (AC) offered 57% of the prescribed sessions, with an average residents' adherence of 60%. Delivery of the intervention did not go as planned due to Covid restrictions in care homes and delivery challenges, including (1) motivation and engagement, (2) changes in cognitive impairment and disabilities of the participants, (3) death or hospitalisation of the participants and (4) limited staffing and technology resources to deliver the programme as intended. Despite this, group participation and encouragement of residents supported the delivery and acceptance of the intervention, with ACs and residents reporting improved mood, physical health, job satisfaction and social support. Improvements with large effect sizes were found for anxiety, depression, loneliness, perceived stress and sleep satisfaction, but no changes in fear of falling, domains of general health or appetite.

CONCLUSION

This realist evaluation suggested that this digitally delivered movement and music intervention is feasible. From the findings, the initial programme theory was refined for future implementation of an RCT in other care homes but future research exploring how to tailor the intervention to those with cognitive impairment and/or lacking capacity to consent is needed.

TRIAL REGISTRATION

Retrospectively registered at ClinicalTrials.gov NCT05559203.

Theory-based evaluation and programme theories in nursing: A discussion on the occasion of the updated Medical Research Council (MRC) Framework

Developing and evaluating health interventions for the benefit of patients is notoriously difficult. This also applies to the discipline of nursing, owing to the complexity of nursing interventions. Following significant revision, the updated guidance of the Medical Research Council (MRC) adopts a pluralistic view to intervention development and evaluation, including a theory-based perspective. This perspective promotes the use of program theory, aiming to understand how and under what circumstances interventions lead to change. In this discussion paper, we reflect the recommended use of program theory in the context of evaluation studies addressing complex nursing interventions. First, we review the literature by investigating the question whether and how evaluation studies targeting complex interventions used theory and to what extent program theories may contribute to enhance the theoretical foundations of intervention studies in nursing. Second, we illustrate the nature of theory-based evaluation and program theories. Third, we argue how this may impact theory building in nursing in general. We finish by discussing which resources, skills and competencies are necessary to fulfill the demanding task of undertaking theory-based evaluations. We caution against an oversimplified interpretation of the updated MRC guidance regarding the theory-based perspective, e.g. by using simple linear logic models, rather than articulating program theories. Instead, we encourage researchers to embrace the corresponding methodology, i.e. theory-based evaluation. With the prevailing perspective of knowledge production in crisis, we might be on the verge of a paradigm shift in health intervention research. Viewed through this lens, the updated MRC guidance could lead to a renewed understanding of what constitutes useful knowledge in nursing. This may facilitate knowledge production and, thereby, contribute to improve nursing practice for the benefit of the patient.

How to collaborate for health throughout the project timeline - a longitudinal study reflecting on implemented strategies in three projects for a healthy living environment

BACKGROUND

When improving the health of local and regional populations, cross-sector collaboration between different policy domains, non-governmental organisations and citizens themselves is needed. Previously, enabling factors and strategies have been identified to improve cross-sector collaboration for health. However, few longitudinal studies have been conducted to understand how the implementation of strategies for cross-sector collaboration changes throughout the collaboration process. The aim of this study is therefore to learn more about the different strategies that were implemented throughout three cross-sector collaboration projects for a healthy living environment.

METHODS

The realist evaluation approach was used to understand how the implemented strategies worked, in which context, why and with what outcomes. Project partners were asked to reflect on their implemented strategies at two different moments in the project timelines, and quarterly updates with project leaders were held. In addition two reference panels were organised for data triangulation.

RESULTS

Three key insights for successful cross-sector collaboration throughout projects for a healthy living environment were identified, namely 1. Investing in trust among the partners and faith in the project has a positive influence on continuing the collaboration throughout the project; 2. Making stakeholders actively participate throughout the project requires additional strategies after the onset of the project, and 3. Defining roles, tasks, and other prerequisites at the start of the project helps in pursuing the project over time, but needs re-examination throughout the project. These key insights were based on multiple examples of implemented strategies, linked to context, mechanisms and outcomes.

CONCLUSIONS

This study shows the different strategies that can be employed as the collaboration in projects for a healthy living environment progresses. We found that 'trust' does not merely include the relationships built between the partners, but at the onset of projects can also be based on faith in the project itself. In addition, as it can be difficult to foresee the right investments and strategies at the onset of the project, frequent reflection moments to choose fitting strategies might benefit regional partners in their cross-sector collaboration for health.

Realist analysis of whether emergency departments with primary care services generate 'provider-induced demand'

Background

It is not known whether emergency departments (EDs) with primary care services influence demand for non-urgent care (‘provider-induced demand’). We proposed that distinct primary care services in EDs encourages primary care demand, whereas primary care integrated within EDs may be less likely to cause additional demand. We aimed to explore this and explain contexts (C), mechanisms (M) and outcomes (O) influencing demand.

Methods

We used realist evaluation methodology and observed ED service delivery. Twenty-four patients and 106 staff members (including Clinical Directors and General Practitioners) were interviewed at 13 EDs in England and Wales (240 hours of observations across 30 days). Field notes from observations and interviews were analysed by creating ‘CMO’ configurations to develop and refine theories relating to drivers of demand.

Results

EDs with distinct primary care services were perceived to attract demand for primary care because services were visible, known or enabled direct access to health care services. Other influencing factors included patients’ experiences of accessing primary care, community care capacity, service design and population characteristics.

Conclusions

Patient, local-system and wider-system factors can contribute to additional demand at EDs that include primary care services. Our findings can inform service providers and policymakers in developing strategies to limit the effect of potential influences on additional demand when demand exceeds capacity.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12873-022-00709-2.

Rivalries for attention: insights from a realist evaluation of a postgraduate competency-based medical education implementation in Canada

BACKGROUND

Implementing competency-based medical education (CBME) in post-graduate medical education (PGME) is a complex process that requires multiple systemic changes in a complex system that is simultaneously engaged in multiple initiatives. These initiatives often compete for attention during the implementation of CBME and produce unintended and unanticipated consequences. Understanding the impact of this context is necessary for evaluating the effectiveness of CBME. The purpose of the study was to identify factors, such as contexts and processes, that contribute to the implementation of CBME.

METHODS

We conducted a realist evaluation using data collected from 15 programs through focus groups with residents (2 groups, n = 16) and faculty (one group, n = 8), and semi-structured interviews with program directors (n = 18), and program administrators (n = 12) from 2018 to 2021. Data were analyzed using a template analysis based on a coding framework that was developed from a sample of transcripts, the context-mechanism-outcomes framework for realist evaluations, and the core components of CBME.

RESULTS

The findings demonstrate that simultaneous initiatives in the academic health sciences system creates a key context for CBME implementation - rivalries for attention - and specifically, the introduction of curricular management systems (CMS) concurrent to, but separate from, the implementation of CBME. This context influenced participants' participation, communication, and adaptation during CBME implementation, which led to change fatigue and unmet expectations for the collection and use of assessment data.

CONCLUSIONS

Rival initiatives, such as the concurrent implementation of a new CMS, can have an impact on how programs implement CBME and greatly affect the outcomes of CBME. Mitigating the effects of rivals for attention with flexibility, clear communication, and training can facilitate effective implementation of CBME.

How can community-based (re)engagement initiatives meet the needs of 'NEET' young people? Findings from the theory gleaning phase of a realist evaluation in Sweden

Objective

There has been a lack of systematic and theoretically underpinned evaluations, internationally and in Sweden, of local multi-component initiatives delivered outside public employment services and formal education systems to young people who are not in employment, education or training (‘NEETs’). To bridge this knowledge gap, the objective of this study was to present findings from the theory gleaning phase of a realist evaluation aimed at assessing how Swedish community-based initiatives may work to (re)engage vulnerable ‘NEET’ young people in education or employment, under what conditions and why.

Results

Based on insights gleaned and synthesised from various sources, three candidate programme theories were elicited drawing attention to the importance of community-based initiatives in Sweden adopting a ‘caring approach’, a ‘capability approach’ and a ‘collaborative approach’ to (re)engage ‘NEET’ young people in education or employment. While limited to the initial phase of theory gleaning, the study provides valuable insights into the potential functioning of (re)engagement initiatives directed towards vulnerable ‘NEETs’ in addition to increasing the transparency of a highly iterative research project.

Making implementation science more real

Implementation science in healthcare aims to understand how to get evidence into practice. Once this is achieved in one setting, it becomes increasingly difficult to replicate elsewhere. The problem is often attributed to differences in context that influence how and whether implementation strategies work. We argue that realist research paradigms provide a useful framework to express the effect of contextual factors within implementation strategy causal processes. Realist studies are theory-driven evaluations that focus on understanding how and why interventions work under different circumstances. They consider the interaction between contextual circumstances, theoretical mechanisms of change and the outcomes they produce, to arrive at explanations of conditional causality (i.e., what tends to work, for whom, under what circumstances). This Commentary provides example applications using preliminary findings from a large realist implementation study of system-wide value-based healthcare initiatives in New South Wales, Australia. If applied judiciously, realist implementation studies may represent a sound approach to help optimise delivery of the right care in the right setting and at the right time.

Contributing to collaborative health governance in Africa: a realist evaluation of the Universal Health Coverage Partnership

Background

Policy dialogue, a collaborative governance mechanism, has raised interest among international stakeholders. They see it as a means to strengthen health systems governance and to participate in the development of health policies that support universal health coverage. In this context, WHO has set up the Universal Health Coverage Partnership. This Partnership aims to support health ministries in establishing inclusive, participatory, and evidence-informed policy dialogue. The general purpose of our study is to understand how and in what contexts the Partnership may support policy dialogue and with what outcomes. More specifically, our study aims to answer two questions: 1) How and in what contexts may the Partnership initiate and nurture policy dialogue? 2) How do collaboration dynamics unfold within policy dialogue supported by the Partnership? 

Methods

We conducted a multiple-case study realist evaluation based on Emerson’s integrative framework for collaborative governance to investigate the role of the Partnership in policy dialogue on three policy issues in six sub-Saharan African countries: health financing (Burkina Faso and Democratic Republic of Congo), health planning (Cabo Verde, Niger, and Togo), and aid coordination for health (Liberia). We interviewed 121 key informants, analyzed policy documents, and observed policy dialogue events.

Results

The Partnership may facilitate the initiation of policy dialogue when: 1) stakeholders feel uncertain about health sector issues and acknowledge their interdependence in responding to such issues, and 2) policy dialogue coincides with their needs and interests. In this context, policy dialogue enables stakeholders to build a shared understanding of issues and of the need for action and encourages collective leadership. However, ministries’ weak ownership of policy dialogue and stakeholders’ lack of confidence in their capacity for joint action hinder their engagement and curb the institutionalization of policy dialogue.

Conclusions

Development aid actors wishing to support policy dialogue must do so over the long term so that collaborative governance becomes routine and a culture of collaboration has time to grow. Public administrations should develop collaborative governance mechanisms that are transparent and intelligible in order to facilitate stakeholder engagement.

The workings of an action learning program for building mental health promotion capacity - A realist evaluation

Action learning is a promising approach for building mental health promotion (MHP) capacity. The aim of this study is to explore how action learning processes can strengthen MHP capacity within and across organizations in a community setting. We applied an embedded case study design and a realist evaluation framework to explore key combinations of mechanisms and contextual factors that generated the emergent MHP capacity outcomes of an action learning program, i.e. context-mechanism-outcome-configurations (CMO-configurations). Data consisted of 18 semi-structured face-to-face interviews, 10 telephone interviews, two group interviews, observations, and documents. Interviewees (n = 21) were participants and management employees. Our analytical provision of CMO-configurations provides insights into how contextual factors, such as participant motivation, organizational support, and existing task descriptions, in combination with certain program mechanisms, such as legitimization of specific agendas, learning-by-doing, and collaborations across organizational boundaries, explain the identified outcomes. Outcomes ranged across implementation of MHP initiatives, personal development among participants, and relational and collaborative development. Taken together, our results strengthen the notion that mechanisms of action learning hold the potential to build MHP capacity on an individual, organizational, and community level. This study, also, illustrates that realist evaluation offers a relevant methodology for investigating the underlying workings of capacity building programs.

Realist inquiry: a new way to think about overcrowding interventions

OBJECTIVES

Physician-focused throughput initiatives are intended to mitigate the effects of emergency department (ED) overcrowding. Our tertiary care academic hospital recently piloted an emergency physician leader role intended to improve throughput. Although a separate experimental evaluation of this initiative was undertaken, it was expected that such an evaluation could not alone provide the necessary information to inform 'how' the emergency physician leader intervention worked. The objective of this study was to conduct a realist evaluation of the emergency physician leader. Realist inquiry utilizes Context Mechanism Outcome configurations to determine the impacts of interventions that are often missed by quantitative analysis. Using a realist perspective, this study aimed to evaluate the emergency physician leader initiative's effects on throughput with the goal of building transferable lessons to the implementation of future interventions.

METHODS

Semi-structured interviews with key stakeholders in the intervention were conducted and analyzed using thematic and template techniques specifically aimed at identifying Context Mechanism Outcomes.

RESULTS

13 interviews were conducted with physicians and nurses who identified contexts and mechanisms which promoted or impeded ED throughput. For example, in situations where there was a clear indication for imaging or management that could not be initiated within the scope of a nursing protocol, the emergency physician leader initiating these orders was felt to promote ED throughput. Conversely, in contexts where there was no nurse available to fulfill early orders, the emergency physician leader's initiation of orders was perceived to impede throughput.

CONCLUSION

This evaluation provides insights into the reasoning and behaviour of individuals involved in the emergency physician leader initiative and provides a systematic approach to unraveling its complex causal pathway. Knowledge of context-mechanism-outcome relationships may help implementers design and measure the impact of future physician-focused throughput interventions.

Contextual factors of external inspections and mechanisms for improvement in healthcare organizations: A realist evaluation

External inspections constitute a key element of healthcare regulation. Improved quality of care is one of the important goals of inspections but the mechanisms of how inspections might contribute to quality improvement are poorly understood. Drawing on interviews with healthcare professionals and managers and health record data from inspected organizations, we used a realist evaluation approach to explore how twelve inspections of healthcare providers in x= Norway influenced quality improvement. We found that for inspections to contribute to quality improvement, there must be contextual structures present supporting accountability and engaging staff in improvement work. When such structures are present, inspections can contribute to improvement by creating awareness of gaps between desired and current practices, which leads to readiness for change and stimulates intra-organizational reasoning around quality improvement. We discuss our findings using the theory of de- and recoupling, noting how regulators can identify decoupling between intended goals, management systems, practices, and patient outcomes. We further argue that regulators can contribute to a recoupling between these levels by having the capacity to track the providers' clinical performance over time. This will hold the organization accountable for implementing improvement measures and evaluate the effects of the measures on quality of care.

At the interface between the formal and informal, the actual and the real: a realist study protocol for governance and accountability practices in urban settings focusing on adolescent sexual and reproductive health and rights

BACKGROUND

This paper presents the protocol of a study that aims at exploring how different multi-level governance arrangements impact on sexual and reproductive health of adolescents living in informal settlements. The overall objective of this study is to contribute to a better understanding of the causal chains underlying accountability in sexual and reproductive health for adolescent girls and young women living in urban informal settlements in low-and middle-income countries.

METHODS

The overarching methodology is realist evaluation. The study adopts a case study design, through which governance and accountability practices in Mumbai, Delhi, Cotonou and Kampala will be examined. Different social science methods to develop and test a programme theory will be used. Heuristic tools for the analysis of the accountability ecosystem and mapping of governance arrangements, drawing from contextual political analysis and critical realism, will be developed in order to identify the intervention-context-actor-mechanism-outcome configurations.

DISCUSSION

The methodological approach is geared towards building robust case-based explanation with due attention to context and the roles of different actors. The combination of different social science methods will lead us to a better grasp of the inherently political nature of social accountability.

What helps older people persevere with yoga classes? A realist process evaluation of a COVID-19-affected yoga program for fall prevention

Background

Falls among older people are a major global health concern. This process evaluation investigates the experience of participants aged 60+ in a yoga program aimed at preventing falls which transitioned from studio-based classes to online classes in response to COVID-19 restrictions. We sought to understand how the Successful AGEing (SAGE) yoga program functioned in both settings and as a hybrid program, and to explain why it worked well for most participants.

Methods

Realist process evaluation was used to explore the factors that facilitated a successful transition for most participants, and to consider why it did not work for a minority. This approach develops program theories that describe which mechanisms an intervention is (or is not) activating, and how this is mediated by context to generate process outcomes. Data included interviews with participants (n = 21) and yoga instructors (n = 3), self-report feedback forms (n = 46), observation of classes and routine process measures.

Results

Factors that facilitated a successful transition for most participants included the quality of yoga instruction, the program format and inherent characteristics of yoga. Gains in transitioning online included continuity and greater convenience. Losses included perceived reduction in the effectiveness of yoga instruction. There were greater challenges for people struggling with pain and in disadvantageous home environments. We identified six program theories configured around 16 mechanisms: 1. It’s worth the effort and 2. In expert hands (these had the same mechanisms: value expectancy, therapeutic alliance and achievement/mastery), 3. A communal experience (these mechanisms were shared experience, social connection, social comparison and peer checking), 4. Putting yoga within reach (accessibility, convenience, gratitude), 5. Building yoga habits (purposeful structure, momentum, accountability and continuity), and 6. Yoga’s special properties (embodiment and mindfulness).

Conclusions

This study showed that online delivery of a yoga program for people aged 60+ retained much of the value of a face-to-face program for the majority of participants, and increased the value for some. The structured, communal nature of an organised group program delivered by a skilled instructor, together with yoga’s intrinsic focus on mindfulness, facilitated continued engagement and perceived health benefits, despite the change in delivery mode.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-12818-5.

Do the implementation processes of a school-based daily physical activity (DPA) program vary according to the socioeconomic context of the schools? a realist evaluation of the Active at school program

Background

Less than half of Canadian children meet the Canadian Physical Activity (PA) Guidelines, and the proportion is even lower among children living in underprivileged neighbourhoods. Regular PA supports physical, cognitive, and psychological/social health among school-aged children. Successful implementation of school-based daily physical activity (DPA) programs is therefore important for all children and crucial for children who attend schools in lower socioeconomic settings. The purpose of this study is to uncover what worked, for whom, how, and why during the three-year implementation period of a new “flexible” DPA program, while paying particular attention to the socioeconomic setting of the participating schools.

Methods

This study is a realist evaluation using mixed methods for data generation. Longitudinal data were collected in 415 schools once a year during the three-year implementation period of the program using questionnaires. Data analysis was completed in three steps and included qualitative thematic analysis using a mixed inductive and deductive method and chi-square tests to test and refine context-mechanism-outcome (CMO) configurations.

Results

Giving the school teams autonomy in the choice of strategies appropriate to their context have allowed schools to take ownership of program implementation by activating a community empowerment process, which resulted in a cultural shift towards a sustainable DPA provision in most settings. In rural underprivileged settings, the mobilization of local resources seems to have successfully created the conditions necessary for implementing and maintaining changes in practice. In disadvantaged urban settings, implementing local leadership structures (leader, committee, and meetings) provided pivotal assistance to members of the school teams in providing new DPA opportunities. However, without continued external funding, those schools seem unable to support local leadership structures on their own, jeopardizing the sustainability of the program for children living in disadvantaged urban areas.

Conclusion

By exploring CMO configurations, we have been able to better understand what worked, for whom, how and why during the three-year implementation period of the Active at School! program. When implementing DPA policies, decision makers should consider adjusting resource allocations to meet the actual needs of schools from different backgrounds to promote equal PA opportunities for all children.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-12797-7.

The effectiveness of extended-duration supervision training for nurses and allied health professionals: A realist evaluation

BACKGROUND

Training helps maintain high-quality supervision and its associated benefits (e.g. reduced burnout, improved care). While studies have previously evaluated extended-duration supervision training programmes, none have treated these as complex interventions so have not employed realist approaches.

OBJECTIVES

Building on a previous realist synthesis, this evaluation tests and develops programme theory for extended-duration supervision training to answer the question: to what extent does the supervision training programme work, for whom, under what circumstances and why?

DESIGN

We conducted a realist evaluation of a novel state-wide Victorian 3-month supervision training programme including one or two 3.5-h workshops followed by weekly reflexive longitudinal audio diaries (LADs) for up to 12 weeks.

METHODS

Realist evaluation data comprised 25 entrance interviews with nurses and allied health professionals, 176 LADs (and 29 written diaries), and 23 exit interviews. We employed team-based realist analysis to identify context-mechanism-outcome configurations (CMOCs) to test and develop programme theory.

RESULTS

We refined four recurring CMOCs from the realist synthesis programme theory, found insufficient evidence for two, and established five new recurring CMOCs. We identified multiple positive outcomes from our extended-duration supervision training programme (e.g. improved supervisor practices) through various mechanisms relating to pedagogy (e.g. weekly reflexivity), supervisors (e.g. engagement), and workplaces (e.g. enabling supervision cultures). Some negative outcomes were reported (e.g. decreased engagement) through various mechanisms (e.g. suboptimal training design). Such mechanisms were thought to come about by diverse contexts including supervisors (e.g. inexperienced/experienced), professions (nursing/allied health), and organisations (supervision-enabled/non-enabled cultures).

CONCLUSIONS

Our findings extend the realist synthesis programme theory by highlighting various contexts triggering outcome-generating mechanisms. Programme outcomes are maximised through ongoing supervisor reflexivity paying attention to facilitator-supervisor relationships, as well as protected time for supervisors to translate learning into practice.

How do supported employment programs work? Answers from a systematic literature review

Many studies have found that supported employment (SE) has effectively helped people with severe mental illness obtain and maintain competitive employment. However, most SE studies have asked "What works?" rather than discerning what works for whom, in what circumstances, in what respects and how. It is important to understand the outcomes of SE and identify the impact factors (contexts and mechanisms) that can trigger them. Four literature databases were searched for studies that analyzed counseling settings. Overall, 104 publications met the inclusion criteria. The review showed that most of the research on SE programs were one-dimensional, looking at either the effects of SE programs, the client or the professional, or the relationship between clients and professionals. The model reveals that impact factors are interconnected and can have a cumulative impact on the client, professionals, and the environment.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10775-022-09533-3.

Are realist randomised controlled trials possible? A reflection on the INCLUSIVE evaluation of a whole-school, bullying-prevention intervention

We previously proposed that realist randomised controlled trials could be used to evaluate how, for whom and under what conditions complex interventions can be used to activate mechanisms to improve health. While this idea was accepted by some, it was also met with resistance, particularly from some realist evaluators who believe that trials are inextricably positivist and dependent on constant conjunctions to understand causation, and that realist trials are unfeasible because participants and contexts will be insufficiently diverse to enable the testing of context-mechanism-outcome configurations. In this paper, we reflect on analyses of qualitative and quantitative data from the Initiating Change Locally in Bullying and Aggression through the School Environment (INCLSUIVE) trial, and whether these are useful and aligned with realism. We summarise the concerns expressed by realists and reflect on the philosophical and practical challenges that we encountered and whether or not they are related to the trial's design. Finally, we reflect on the trial's weaknesses and highlight areas that future researchers might consider when running realist trials. We conclude that realist randomised controlled trials are philosophically coherent, practically feasible, and can produce nuanced findings.

Role of trust in sustaining provision and uptake of maternal and child healthcare: Evidence from a national programme in Nigeria

Despite increasing attention to implementation research in global health, evidence from low- and middle-income countries (LMICs) using realist evaluations, in understanding how complex health programmes work remains limited. This paper contributes to bridging this knowledge gap by reporting how, why and in what circumstances, the implementation and subsequent termination of a maternal and child health programme affected the trust of service users and healthcare providers in Nigeria. Key documents were reviewed, and initial programme theories of how context triggers mechanisms to produce intended and unintended outcomes were developed. These were tested, consolidated and refined through iterative cycles of data collection and analysis. Testing and validation of the trust theory utilized eight in-depth interviews with health workers, four focus group discussions with service users and a household survey of 713 pregnant women and analysed retroductively. The conceptual framework adopted Hurley's perspective on 'decision to trust' and Straten et al.'s framework on public trust and social capital theory. Incentives offered by the programme triggered confidence and satisfaction among service users, contributing to their trust in healthcare providers, increased service uptake, motivated healthcare providers to have a positive attitude to work, and facilitated their trust in the health system. Termination of the programme led to most service users' dissatisfaction, and distrust reflected in the reduction in utilization of MCH services, increased staff workloads leading to their decreased performance although residual trust remained. Understanding the role of trust in a programme's short and long-term outcomes can help policymakers and other key actors in the planning and implementation of sustainable and effective health programmes. We call for more theory-driven approaches such as realist evaluation to advance understanding of the implementation of health programmes in LMICs.

Contextual factors and mechanisms that influence sustainability: a realist evaluation of two scaled, multi-component interventions

Background

In 2012, Alberta Health Services created Strategic Clinical Networks^TM (SCNs) to develop and implement evidence-informed, clinician-led and team-delivered health system improvement in Alberta, Canada. SCNs have had several provincial successes in improving health outcomes. Little research has been done on the sustainability of these evidence-based implementation efforts.

Methods

We conducted a qualitative realist evaluation using a case study approach to identify and explain the contextual factors and mechanisms perceived to influence the sustainability of two provincial SCN evidence-based interventions, a delirium intervention for Critical Care and an Appropriate Use of Antipsychotics (AUA) intervention for Senior’s Health. The context (C) + mechanism (M) = outcome (O) configurations (CMOcs) heuristic guided our research.

Results

We conducted thirty realist interviews in two cases and found four important strategies that facilitated sustainability: Learning collaboratives, audit & feedback, the informal leadership role, and patient stories. These strategies triggered certain mechanisms such as sense-making, understanding value and impact of the intervention, empowerment, and motivation that increased the likelihood of sustainability. For example, informal leaders were often hands-on and influential to front-line staff. Learning collaboratives broke down professional and organizational silos and encouraged collective sharing and learning, motivating participants to continue with the intervention. Continual audit-feedback interventions motivated participants to want to perform and improve on a long-term basis, increasing the likelihood of sustainability of the two multi-component interventions. Patient stories demonstrated the interventions’ impact on patient outcomes, motivating staff to want to continue doing the intervention, and increasing the likelihood of its sustainability.

Conclusions

This research contributes to the field of implementation science, providing evidence on key strategies for sustainability and the underlying causal mechanisms of these strategies that increases the likelihood of sustainability. Identifying causal mechanisms provides evidence on the processes by which implementation strategies operate and lead to sustainability. Future work is needed to evaluate the impact of informal leadership, learning collaboratives, audit-feedback, and patient stories as strategies for sustainability, to generate better guidance on planning sustainable improvements with long term impact.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-07214-5.

A realist evaluation of the continuum of HIV services for men who have sex with men

Background

Men who have Sex with Men (MSM) represent the risk group that are disproportionately most affected by the human immunodeficiency virus (HIV) and continue to drop-off from the steps of the continuum of HIV services that have been adopted to overcome poor engagement and retention in care. This realist evaluation aimed at: (1) describing the evaluation carried out in Haiti aiming to ascertain why, how and under which circumstances MSM are linked and retained along the continuum, (2) assessing the outcomes of this approach and (3) exploring the motivators and facilitators for the HIV continuum of services through mechanisms and pathways.

Methods

Guided by a realist approach, first, an initial program theory (IPT) was developed based on literature and frameworks review, participant observations and discussions with stakeholders. Then, the IPT was tested using a mixed method explanatory study: a quantitative phase to build the continuum from a cross-sectional analysis, and a qualitative phase to explore the motivators and facilitators related to proper linkages along the continuum. Finally, the IPT was refined by eliciting the mechanisms and pathways for outcomes improvement.

Results

The results showed that the current service delivery model is suboptimal in identifying, engaging, linking and retaining MSM, resulting in loss to follow-up at every step of the continuum and failure to fully realize the health and prevention benefits of antiretroviral. However, the mechanisms through which linkages across the continuum can be improved are: self-acceptance, sense of community support and sense of comprehensive and tailored HIV services. These mechanisms are based on 10 different pathways: self-esteem, awareness and pride, perception of HIV risk, pcceptance and HIV status, addressing community stigma, strengthening of MSM organizations and community networks, societal acceptation and tolerance, stigma reduction training for healthcare providers, engagement of peers as educators and navigators and, adapted services delivery through drug dispensing points and mobile technology and financial assistance.

Conclusions

The study findings show that engagement, adherence and retention to the continuum of HIV service for MSM are affected by a multi-layer of factors, thus highlighting the importance of taking a comprehensive approach to improve the program.

Understanding the mechanisms generating outcomes in a Danish peer support intervention for socially vulnerable people with type 2-diabetes: a realist evaluation

BACKGROUND

Despite an increasing use and positive effects of peer support interventions, little is known about how the outcomes are produced. Thus, it is essential not only to measure outcomes, but also to identify the mechanisms by which they are generated. Using a realist evaluation approach, we aimed to identify the mechanisms generating outcomes in a Danish peer support intervention for socially vulnerable people with type 2-diabetes (peers). By investigating the participating peers' interactions, we furthermore examined how their individual contextual factors either facilitated or hindered the mechanisms in operation.

METHODS

We used a multi-method case-study design (n = 9). Data included semi-structured interviews with four key groups of informants (peer, peer supporter, project manager, and a diabetes nurse) for each case (n = 25). Furthermore, we collected survey data from peers both before and after participation (n = 9). The interview data were analysed using a systematic text condensation, and the Intervention-context-actor-mechanism-outcome framework was used to structure the analysis.

RESULTS

We identified 2 groups of mechanisms that improved diabetes self-management and the use of healthcare services (outcomes): 'perceived needs and readiness' and 'encouragement and energy'. However, the mechanisms only generated the intended outcomes among peers with a stable occupation and financial situation, a relatively good health condition, and sufficient energy (all defined as contextual factors). Independent of these contextual factors, 'experience of social and emotional support' was identified as a mechanism within all peers that increased self-care awareness (defined as output). Dependent on whether the contextual factors facilitated or hindered the mechanisms to generate outcomes, we categorised the peers into those who achieved outcomes and those who did not.

CONCLUSIONS

We identified two groups of mechanisms that improved the peers' diabetes self-management and use of healthcare services. The mechanisms only generated the intended outcomes if peers' individual contextual factors facilitated an active interaction with the elements of the intervention. However, independent of these contextual factors, a third group of mechanisms increased self-care awareness among all peers. We highlight the importance of contextual awareness of the target groups in the design and evaluation of peer support interventions for socially vulnerable people with type 2-diabetes.

TRIAL REGISTRATION

ClinicalTrials.gov, Retrospective Registration (20 Jan 2021), registration number NCT04722289 .

Fundamental challenges in assessing the impact of research infrastructure

Clinical research infrastructure is one of the unsung heroes of the scientific response to the current COVID-19 pandemic. The extensive, long-term funding into research support structures, skilled people, and technology allowed the United Kingdom research response to move off the starting blocks at pace by utilizing pre-existing platforms. The increasing focus from funders on evaluating the outcomes and impact of research infrastructure investment requires both a reframing and progression of the current models in order to address the contribution of the underlying support infrastructure. The majority of current evaluation/outcome models focus on a "pipeline" approach using a methodology which follows the traditional research funding route with the addition of quantitative metrics. These models fail to embrace the complexity caused by the interplay of previous investment, the coalescing of project outputs from different funders, the underlying infrastructure investment, and the parallel development across different parts of the system. Research infrastructure is the underpinning foundation of a project-driven research system and requires long-term, sustained funding and capital investment to maintain scientific and technological expertise. Therefore, the short-term focus on quantitative metrics that are easy to collect and interpret and that can be assessed in a roughly 5-year funding cycle needs to be addressed. The significant level of investment in research infrastructure necessitates investment to develop bespoke methodologies that develop fit-for-purpose, longer-term/continual approach(es) to evaluation. Real-world research should reflect real-world evaluation and allow for the accrual of a narrative of value indicators that build a picture of the contribution of infrastructure to research outcomes. The linear approach is not fit for purpose, the research endeavour is a complex, twisted road, and the evaluation approach needs to embrace this complexity through the development of realist approaches and the rapidly evolving data ecosystem. This paper sets out methodological challenges and considers the need to develop bespoke methodological approaches to allow a richer assessment of impact, contribution, attribution, and evaluation of research infrastructure. This paper is the beginning of a conversation that invites the community to "take up the mantle" and tackle the complexity of real-world research translation and evaluation.

A realist evaluation of a "single point of contact" end-of-life care service

PURPOSE

Following the development of a service that consisted of a "single point of contact" to coordinate end-of-life care (EoLC), including EoLC facilitators and an urgent response team, we aimed to explore whether the provision of coordinated EoLC would support patients being cared or dying in their preferred place and avoid unwanted hospital admissions.

DESIGN/METHODOLOGY/APPROACH

Using a realist evaluation approach, the authors examined "what worked for whom, how, in what circumstances and why". Multiple data were collected, including activity/performance indicators, observations of management meetings, documents, satisfaction survey and 30 interviews with service providers and users.

FINDINGS

Advance care planning (ACP) increased through the first three years of the service (from 45% to 83%) and on average 74% of patients achieved preferred place of death. More than 70% of patients avoided an emergency or unplanned hospital admission in their last month of life. The mechanisms and context identified as driving forces of the service included: 7/7 single point of contact; coordinating services across providers; recruiting and developing the workforce; understanding and clarifying new roles; and managing expectations.

RESEARCH LIMITATIONS/IMPLICATIONS

This was a service evaluation and the outcomes are related to the specific context and mechanisms. However, findings can be transferable to similar settings.

PRACTICAL IMPLICATIONS

"Single point of contact" services that offer coordinated EoLC can contribute in supporting people to be cared and die in their preferred place.

ORIGINALITY/VALUE

This paper provides an evaluation of a novel approach to EoLC and creates a set of hypotheses that could be further tested in similar services in the future.

Tracing theories in realist evaluations of large-scale health programmes in low- and middle-income countries: experience from Nigeria

Realist evaluations (RE) are increasingly popular in assessing health programmes in low- and middle-income countries (LMICs). This article reflects on processes of gleaning, developing, testing, consolidating and refining two programme theories (PTs) from a longitudinal mixed-methods RE of a national maternal and child health programme in Nigeria. The two PTs, facility security and patient–provider trust, represent complex and diverse issues: trust is all encompassing although less tangible, while security is more visible. Neither PT was explicit in the original programme design but emerged from the data and was supported by substantive theories. For security, we used theories of fear of crime, which perceive security as progressing from structural, political and socio-economic factors. Some facilities with the support of communities erected fences, improved lighting and employed guards, which altogether contributed to reduced fear of crime from staff and patients and improved provision and uptake of health care. The social theories for the trust PT were progressively selected to disentangle trust-related micro, meso and macro factors from the deployment and training of staff and conditional cash transfers to women for service uptake. We used taxonomies of trust factors such as safety, benevolent concerns and capability. We used social capital theory to interpret the sustainability of ‘residual’ trust after the funding for the programme ceased. Our overarching lesson is that REs are important though time-consuming ways of generating context-specific implications for policy and practice within ever-changing contexts of health systems in LMICs. It is important to ensure that PTs are ‘pitched at the right level’ of abstraction. The resource-constrained context of LMICs with insufficient documentation poses challenges for the timely convergence of nuggets of evidence to inform PTs. A retroductive approach to REs requires iterative data collection and analysis against the literature, which require continuity, coherence and shared understanding of the analytical processes within collaborative REs.