Realist evaluations in low- and middle-income countries: reflections and recommendations from the experiences of a foreign researcher

Community case management to accelerate access to healthcare in Mali: a realist process evaluation nested within a cluster randomized trial

The Proactive Community Case Management (ProCCM) trial in Mali reinforced the health system across both arms with user fee removal, professional community health workers (CHWs) and upgraded primary health centres (PHCs)-and randomized village-clusters to receive proactive home visits by CHWs (intervention) or fixed site-based services by passive CHWs (control). Across both arms, sick children's 24-hour treatment and pregnant women's four or more antenatal visits doubled, and under-5 mortality halved, over 3 years compared with baseline. In the intervention arm, proactive CHW home visits had modest effects on children's curative and women's antenatal care utilization, but no effect on under-5 mortality, compared with the control arm. We aimed to explain these results by examining implementation, mechanisms and context in both arms We conducted a process evaluation with a mixed method convergent design that included 79 in-depth interviews with providers and participants over two time-points, surveys with 195 providers and secondary analyses of clinical data. We embedded realist approaches in novel ways to test, refine and consolidate theories about how ProCCM worked, generating three context-intervention-actor-mechanism-outcome nodes that unfolded in a cascade. First, removing user fees and deploying professional CHWs in every cluster enabled participants to seek health sector care promptly and created a context of facilitated access. Second, health systems support to all CHWs and PHCs enabled equitable, respectful, quality healthcare, which motivated increased, rapid utilization. Third, proactive CHW home visits facilitated CHWs and participants to deliver and seek care, and build relationships, trust and expectations, but these mechanisms were also activated in both arms. Addressing multiple structural barriers to care, user fee removal, professional CHWs and upgraded clinics interacted with providers' and patients' agency to achieve rapid care and child survival in both arms. Proactive home visits expedited or compounded mechanisms that were activated and changed the context across arms.

Understanding the Mechanisms of Change in the Supportive and Respectful Maternity Care Intervention in Sindh, Pakistan: Provider Perspectives

INTRODUCTION

We conducted this qualitative investigation to explore the mechanisms of change in providing respectful care resulting from the supportive and respectful maternity care intervention (S-RMC) in Sindh, Pakistan.

METHODS

We applied the principles of realist evaluation methodology with a descriptive explanatory research design. We conducted in-depth interviews with 36 maternity care providers at secondary-level public health facilities where S-RMC was implemented for 6 months. The S-RMC broad components included capacity-building of maternity teams and systemic changes for improvements in governance and accountability within public health facilities. Data were analyzed using a deductive content analysis approach.

RESULTS

We identified mechanisms of change, categorized by the S-RMC components: (1) S-RMC training: insight into women's feelings and rights, realization of the value that nonclinical staff can play, understanding of team coordination, orientation in psychosocial components of maternity care; (2) assessment of women's psychosocial vulnerabilities: identification of women's differential needs beyond routine care to provide woman-centered care; (3) psychosocial support: effective engagement with women and within maternity teams and the customization of woman- and companion-focused care; (4) care coordination: improved coordination among clinical and nonclinical staff to provide personalized care and psychosocial support and proper handover to ensure continuity of care; (5) assessment of quality of care: identification of service gaps from women's feedback; and (6) performance review and accountability: monthly performance review meetings to establish team member communication, systematic awareness of the maternity team's performance and challenges, and implementation of collective corrective actions.

CONCLUSION

Our findings pointed to S-RMC working along multiple pathways-and concertedly with various health system components-to enable positive processes and behavioral change in maternity teams.

BREATHLEssness in INDIA (BREATHE-INDIA)-Study protocol for the co-design of a community breathlessness intervention in India using realist methods and intervention mapping

BACKGROUND

Breathlessness that persists despite treatment of causal disease(s) is disabling, associated with high therapy-related costs and poor socioeconomic outcomes. Low resource countries bear a disproportionate burden of respiratory problems, often characterised by disabling breathlessness. Low-cost self-management breathlessness-targeted interventions are effective and deliverable in community settings but have been developed in high-income countries. We aim to understand how breathlessness self-management works in 'real life' populations and cultural contexts, to develop programme theory and co-design a prototype intervention to improve persistent breathlessness management in India.

METHODS AND ANALYSIS

Using a Realist approach, Intervention Mapping and the Medical Research Council Complex Intervention Framework we will undertake two phases of work supported by our Expert Group (of respiratory, primary, palliative care physicians) and key stakeholder groups (opinion leader clinicians, community health workers and people with lived experiences of breathlessness). 1) Realist review and evaluation to identify and refine evidence and theory for breathlessness self-management, producing intervention and implementation programme theory. We will identify literature through our Expert Group, scoping searches and systematic searches (Medline, Ebscohost, CINAHL, Scopus, Psychinfo). We will map intervention components to 'what works, for whom, and where.' 2) Intervention development using Intervention Mapping to map intervention and implementation programme theory to intervention components, develop materials to support intervention delivery, and co-design a prototype educational intervention ready for early acceptability and delivery-feasibility testing and evaluation planning in India. Use of stakeholder groups is to allow people with experience of breathlessness and/or its management to contribute their views on content developed by our team based upon review of secondary data sources. Experts and Stakeholders are therefore not research subjects but are included as extended members of the study team and will not follow informed consent procedures. Experts and stakeholders will be acknowledge in outputs arising from our project if they wish to be. Our review conduct will be consistent with RAMESES quality standards.

DISCUSSION

At the conclusion of our study, we will have co-designed a breathlessness intervention for use in the community setting in India ready for further evaluation of: effectiveness, socioeconomic outcomes, acceptability and transferability to other low resource settings.

Methods in realist evaluation: A mapping review

Realist evaluation is becoming increasingly popular as an evaluation methodology. Its main objective is to uncover the mechanisms that lead to observed outcomes following an intervention and the contextual conditions that enabled this. The focus is on explaining why, for whom and in what circumstances an intervention works. It is a theory-driven approach and is explicitly method neutral, meaning that both quantitative and qualitative data collection methods can be used to unearth the underlying mechanisms that cause the intervention outcomes. In this review, we aim to map the methods used in realist evaluation studies, to draw lessons from the findings and to reflect on ways forward. We found that qualitative methods and interviews specifically are most commonly used in realist evaluations; that theory is often absent behind the methods and sampling techniques used; and that more innovative methods remain underexplored. We conclude the review by proposing four ways forward: (1) developing realist surveys, (2) exploring the relevance of innovative methods, (3) increasing the attention paid to sampling procedures and (4) strengthening the theory-driven nature of method. We believe that these four action points can strengthen the practice of realist evaluation and its outcomes.

Realist evaluation in times of decolonising global health

Realist evaluation (RE) is a theory-driven evaluation approach inspired by scientific realism. It has become increasingly popular in the field of global health where it is often applied in low- and middle-income countries. This makes it timely to discuss RE's relationship to the emerging decolonisation of global health movement. In this short perspective, we argue that the principles and practices that underpin RE have great potential to contribute to the decolonisation endeavour. Both the focus on the inclusion of local stakeholders and the openness to the rival theories these stakeholders bring to the fore, are promising. However, in practice, we see that a lack of acknowledgement of power imbalances and different ontologies and an overreliance on Western-based theories thwart this potential. We therefore suggest that realist evaluations performed by external researchers, especially in the field of global health, should actively engage with issues of (power) inequities. This is not only the just thing to do, but will also contribute to a better understanding of the intervention and may facilitate the emancipation of the disenfranchised. One way of doing this is through the adoption of participatory (action) research methods, currently underused in realist evaluations. We finally give a short example of an evaluation that combines emancipatory and participatory practice development with a realist approach. The Afya-Tek project in Tanzania has an innovative bottom-up approach throughout the full evaluation cycle and shows the possible strength of the proposed combination to create better interventions, more empowered stakeholders, and more illuminating programme theories.

A realist review protocol on communications for community engagement in maternal and newborn health programmes in low- and middle-income countries

BACKGROUND

Community engagement (CE) has been increasingly implemented across health interventions, including for maternal and newborn health (MNH). This may take various forms, from participatory women's groups and community health committees to public advocacy days. While research suggests a positive influence of CE on MNH outcomes, such as mortality or care-seeking behaviour, there is a need for further evidence on the processes of CE in different settings in order to inform the future development and implementation of CE across programmes. Communication is an integral component of CE serving as a link between the programme and community. The aim of the realist review described in this protocol is to understand how, why, to what extent, and for whom CE contributes to intended and unintended outcomes in MNH programming, focusing on the communication components of CE.

METHODS

Realist review methodology will be used to provide a causal understanding of what communication for CE interventions in MNH programming work, for whom, to what extent, why, and how. This will be done by developing and refining programme theories on communications for CE in MNH through a systematic review of the literature and engaging key experts for input and feedback. By extrapolating context-mechanism-outcome configurations, this review seeks to understand how certain contexts trigger or inhibit specific mechanisms and what outcomes this interaction generates when communication in CE interventions is used in MNH programming.

DISCUSSION

A realist philosophy is well-suited to address the aims of this study because of the complex nature of CE. The review findings will be used to inform a realist evaluation case study of CE for an MNH programme in order to ascertain transferable findings that can inform and guide engagement activities in various settings. Findings will also be shared with stakeholders and experts involved in the consultative processes of the review (through workshops or policy briefs) in order to ensure the relevance of these findings to policy and practice.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42022293564.

Strengthening capacity in hospitals to reduce perinatal morbidity and mortality through a codesigned intervention package: protocol for a realist evaluation as part of a stepped-wedge trial of the Action Leveraging Evidence to Reduce perinatal morTality and morbidity (ALERT) in sub-Saharan Africa project

INTRODUCTION

Despite a strong evidence base for developing interventions to reduce child mortality and morbidity related to pregnancy and delivery, major knowledge-implementation gaps remain. The Action Leveraging Evidence to Reduce perinatal morTality and morbidity (ALERT) in sub-Saharan Africa project aims to overcome these gaps through strengthening the capacity of multidisciplinary teams that provide maternity care. The intervention includes competency-based midwife training, community engagement for study design, mentoring and quality improvement cycles. The realist process evaluation of ALERT aims at identifying and testing the causal pathway through which the intervention achieves its impact.

METHODS AND ANALYSIS

This realist process evaluation complements the effectiveness evaluation and the economic evaluation of the ALERT intervention. Following the realist evaluation cycle, we will first elicit the initial programme theory on the basis of the ALERT theory of change, a review of the evidence on adoption and diffusion of innovations and the perspectives of the stakeholders. Second, we will use a multiple embedded case study design to empirically test the initial programme theory in two hospitals in each of the four intervention countries. Qualitative and quantitative data will be collected, using in-depth interviews with hospital staff and mothers, observations, patient exit interviews and (hospital) document reviews. Analysis will be guided by the Intervention-Actors-Context-Mechanism-Outcome configuration heuristic. We will use thematic coding to analyse the qualitative data. The quantitative data will be analysed descriptively and integrated in the analysis using a retroductive approach. Each case study will end with a refined programme theory (in-case analysis). Third, we will carry out a cross-case comparison within and between the four countries. Comparison between study countries should enable identifying relevant context factors that influence effectiveness and implementation, leading to a mid-range theory that may inform the scaling up the intervention.

ETHICS AND DISSEMINATION

In developing this protocol, we paid specific attention to cultural sensitivity, the do no harm principle, confidentiality and non-attribution. We received ethical approval from the local and national institutional review boards in Tanzania, Uganda, Malawi, Benin, Sweden and Belgium. Written or verbal consent of respondents will be secured after explaining the purpose, potential benefits and potential harms of the study using an information sheet. The results will be disseminated through workshops with the hospital staff and national policymakers, and scientific publications and conferences.

TRIAL REGISTRATION NUMBER

PACTR202006793783148.

Realist Evaluation of the "Abiye" Safe Motherhood Initiative in Nigeria: Unveiling the Black-Box of Program Implementation and Health System Strengthening

Introduction

Realist evaluation studies have spanned different aspects of medicine, especially in the field of public health. However, very few of these studies explicitly detailed how program implementation triggered outcomes that could strengthen understanding of its effect on Health System Strengthening in specific settings. In low- and middle-income countries, like Nigeria, there is a paucity of realist evaluation studies, despite the implementation of multiple intervention programs and projects in these countries. This article is aimed at unveiling the black-box of program implementation and Health System Strengthening of the "Abiye" Safe Motherhood Program in Ondo State, Nigeria. Specifically, it identified the role of contextual factors in the "Abiye" program in Ondo State, determined the mechanisms that facilitated or constrained outcomes of the "Abiye" program, and developed a Context Mechanism Outcome (CMO) Configuration from which a Middle Range Theory (MRT) can be framed.

Methodology

This was qualitative research structured along with the realist domains (Context, Mechanism, and Outcome). The Initial Program Theory was validated by the qualitative study, after which a new MRT was developed. The study population comprised key stakeholders, secondary stakeholders, and primary stakeholders in the Abiye safe motherhood program. Data was collected through 10 key informant interviews, 28 in-depth interviews, and six focus group discussions sessions. Thematic analysis was used to analyze all the qualitative data collected, and seven themes with 19 subthemes emerged in the study.

Results

We identified 13 contextual factors under five principal areas, with most of the factors playing enabling roles, some playing inhibitory roles, while very few played both roles. We elicited eight mechanisms, and some of these facilitated the outcomes, while some constrained the outcomes of the program. Health system strengthening was a key feature of the outcome of the program. We developed a middle-range theory based on the 6 CMO configurations we elicited from the study.

Conclusion and Policy Implications

Realist evaluation is an iterative process that looks beyond the surface to generate evidence. By applying the realist approach, we generated pieces of evidence that can be adapted for policymaking in public health interventions in LMIC.

Role of trust in sustaining provision and uptake of maternal and child healthcare: Evidence from a national programme in Nigeria

Despite increasing attention to implementation research in global health, evidence from low- and middle-income countries (LMICs) using realist evaluations, in understanding how complex health programmes work remains limited. This paper contributes to bridging this knowledge gap by reporting how, why and in what circumstances, the implementation and subsequent termination of a maternal and child health programme affected the trust of service users and healthcare providers in Nigeria. Key documents were reviewed, and initial programme theories of how context triggers mechanisms to produce intended and unintended outcomes were developed. These were tested, consolidated and refined through iterative cycles of data collection and analysis. Testing and validation of the trust theory utilized eight in-depth interviews with health workers, four focus group discussions with service users and a household survey of 713 pregnant women and analysed retroductively. The conceptual framework adopted Hurley's perspective on 'decision to trust' and Straten et al.'s framework on public trust and social capital theory. Incentives offered by the programme triggered confidence and satisfaction among service users, contributing to their trust in healthcare providers, increased service uptake, motivated healthcare providers to have a positive attitude to work, and facilitated their trust in the health system. Termination of the programme led to most service users' dissatisfaction, and distrust reflected in the reduction in utilization of MCH services, increased staff workloads leading to their decreased performance although residual trust remained. Understanding the role of trust in a programme's short and long-term outcomes can help policymakers and other key actors in the planning and implementation of sustainable and effective health programmes. We call for more theory-driven approaches such as realist evaluation to advance understanding of the implementation of health programmes in LMICs.

Epistemic injustice in academic global health

This Viewpoint calls attention to the pervasive wrongs related to knowledge production, use, and circulation in global health, many of which are taken for granted. We argue that common practices in academic global health (eg, authorship practices, research partnerships, academic writing, editorial practices, sensemaking practices, and the choice of audience or research framing, questions, and methods) are peppered with epistemic wrongs that lead to or exacerbate epistemic injustice. We describe two forms of epistemic wrongs, credibility deficit and interpretive marginalisation, which stem from structural exclusion of marginalised producers and recipients of knowledge. We then illustrate these forms of epistemic wrongs using examples of common practices in academic global health, and show how these wrongs are linked to the pose (or positionality) and the gaze (or audience) of producers of knowledge. The epistemic injustice framework shown in this Viewpoint can help to surface, detect, communicate, make sense of, avoid, and potentially undo unfair knowledge practices in global health that are inflicted upon people in their capacity as knowers, and as producers and recipients of knowledge, owing to structural prejudices in the processes involved in knowledge production, use, and circulation in global health.

Tracing theories in realist evaluations of large-scale health programmes in low- and middle-income countries: experience from Nigeria

Realist evaluations (RE) are increasingly popular in assessing health programmes in low- and middle-income countries (LMICs). This article reflects on processes of gleaning, developing, testing, consolidating and refining two programme theories (PTs) from a longitudinal mixed-methods RE of a national maternal and child health programme in Nigeria. The two PTs, facility security and patient–provider trust, represent complex and diverse issues: trust is all encompassing although less tangible, while security is more visible. Neither PT was explicit in the original programme design but emerged from the data and was supported by substantive theories. For security, we used theories of fear of crime, which perceive security as progressing from structural, political and socio-economic factors. Some facilities with the support of communities erected fences, improved lighting and employed guards, which altogether contributed to reduced fear of crime from staff and patients and improved provision and uptake of health care. The social theories for the trust PT were progressively selected to disentangle trust-related micro, meso and macro factors from the deployment and training of staff and conditional cash transfers to women for service uptake. We used taxonomies of trust factors such as safety, benevolent concerns and capability. We used social capital theory to interpret the sustainability of ‘residual’ trust after the funding for the programme ceased. Our overarching lesson is that REs are important though time-consuming ways of generating context-specific implications for policy and practice within ever-changing contexts of health systems in LMICs. It is important to ensure that PTs are ‘pitched at the right level’ of abstraction. The resource-constrained context of LMICs with insufficient documentation poses challenges for the timely convergence of nuggets of evidence to inform PTs. A retroductive approach to REs requires iterative data collection and analysis against the literature, which require continuity, coherence and shared understanding of the analytical processes within collaborative REs.

Harnessing the health systems strengthening potential of quality improvement using realist evaluation: an example from southern Tanzania

Quality improvement (QI) is a problem-solving approach in which stakeholders identify context-specific problems and create and implement strategies to address these. It is an approach that is increasingly used to support health system strengthening, which is widely promoted in Sub-Saharan Africa. However, few QI initiatives are sustained and implementation is poorly understood. Here, we propose realist evaluation to fill this gap, sharing an example from southern Tanzania. We use realist evaluation to generate insights around the mechanisms driving QI implementation. These insights can be harnessed to maximize capacity strengthening in QI and to support its operationalization, thus contributing to health systems strengthening. Realist evaluation begins by establishing an initial programme theory, which is presented here. We generated this through an elicitation approach, in which multiple sources (theoretical literature, a document review and previous project reports) were collated and analysed retroductively to generate hypotheses about how the QI intervention is expected to produce specific outcomes linked to implementation. These were organized by health systems building blocks to show how each block may be strengthened through QI processes. Our initial programme theory draws from empowerment theory and emphasizes the self-reinforcing nature of QI: the more it is implemented, the more improvements result, further empowering people to use it. We identified that opportunities that support skill- and confidence-strengthening are essential to optimizing QI, and thus, to maximizing health systems strengthening through QI. Realist evaluation can be used to generate rich implementation data for QI, showcasing how it can be supported in ‘real-world’ conditions for health systems strengthening.