Study protocol: realist evaluation of effectiveness and sustainability of a community health workers programme in improving maternal and child health in Nigeria

Collaborative sustainability planning for an outcome measure in outpatient stroke rehabilitation: A qualitative description study

RATIONALE

Evidence suggests that sustainability planning and the use of a collaborative approach to planning result in better sustainability outcomes and more relevant knowledge. Yet, both approaches appear to be underutilized. A detailed description of collaborative sustainability planning may encourage the use of these two impactful strategies.

AIMS AND OBJECTIVES

To explore the collaborative sustainability planning process for a single outcome measure in three rehabilitation sites.

METHODOLOGY

Within the Mayo-Portland Adaptability Inventory-version 4 (MPAI-4) implementation project, we conducted a qualitative description study. We used data from 12 core sustainability planning meetings and 108 follow-up meetings that included a total of 31 clinical and research team participants. Sustainability planning was informed by an MPAI-4-specific implementation guide, and by the results from a realist review of the sustainability of rehabilitation practices and the Clinical Sustainability Assessment Tool. We analyzed qualitative data using thematic content analysis.

RESULTS

Three themes describe the collaborative sustainability planning process: (1) "collaboration as a driver for sustainability" which captures the active collaboration underpinning sustainability planning; (2) "co-creation of a sustainability plan to achieve shared objectives" which captures the identified barriers and facilitators, and selected sustainability strategies linked to one of six collaboratively identified shared objectives; and (3) "the iterative nature of sustainability planning" which captures the necessity of an agile and responsive sustainability planning process.

CONCLUSION

Identified strategies may be useful to support (collaborative) sustainment. Future research could investigate the effect of collaborative sustainability planning on sustainability objectives, and the relationship between these objectives.

Demand and supply analysis for maternal and child health services at the primary healthcare level in Nigeria

BACKGROUND

The low demand for maternal and child health services is a significant factor in Nigeria's high maternal death rate. This paper explores demand and supply-side determinants at the primary healthcare level, highlighting factors affecting provision and utilization.

METHODS

This qualitative study was undertaken in Anambra state, southeast Nigeria. Anambra state was purposively chosen because a maternal and child health programme had just been implemented in the state. The three-delay model was used to analyze supply and demand factors that affect MCH services and improve access to care for pregnant women/mothers and newborns/infants.

RESULT

The findings show that there were problems with both the demand and supply aspects of the programme and both were interlinked. For service users, their delays were connected to the constraints on the supply side. On the demand side, the delays include poor conditions of the facilities, the roads to the facilities are inaccessible, and equipment were lacking in the facilities. These delayed the utilisation of facilities. On the supply side, the delays include the absence of security (fence, security guard), poor citing of the facilities, inadequate accommodation, no emergency transport for referrals, and lack of trained staff to man equipment. These delayed the provision of services.

CONCLUSION

Our findings show that there were problems with both the demand and supply aspects of the programme, and both were interlinked. For service users, their delays were connected to the constraints on the supply side.

Understanding Whether and How a Digital Health Intervention Improves Transition Care for Emerging Adults Living With Type 1 Diabetes: Protocol for a Mixed Methods Realist Evaluation

BACKGROUND

Emerging adults living with type 1 diabetes (T1D) face a series of challenges with self-management and decreased health system engagement, leading to an increased risk of acute complications and hospital admissions. Effective and scalable strategies are needed to support this population to transfer seamlessly from pediatric to adult care with sufficient self-management capability. While digital health interventions for T1D self-management are a promising strategy, it remains unclear which elements work, how, and for which groups of individuals.

OBJECTIVE

This study aims to evaluate the design and implementation of a multicomponent SMS text message-based digital health intervention to support emerging adults living with T1D in real-world settings. The objectives are to identify the intervention components and associated mechanisms that support user engagement and T1D health care transition experiences and determine the individual characteristics that influence the implementation process.

METHODS

We used a realist evaluation embedded alongside a randomized controlled trial, which uses a sequential mixed methods design to analyze data from multiple sources, including intervention usage data, patient-reported outcomes, and realist interviews. In step 1, we conducted a document analysis to develop a program theory that outlines the hypothesized relationships among "individual-level contextual factors, intervention components and features, mechanisms, and outcomes," with special attention paid to user engagement. Among them, intervention components and features depict 10 core characteristics such as transition support information, problem-solving information, and real-time interactivity. The proximal outcomes of interest include user engagement, self-efficacy, and negative emotions, whereas the distal outcomes of interest include transition readiness, self-blood glucose monitoring behaviors, and blood glucose. In step 2, we plan to conduct semistructured realist interviews with the randomized controlled trial's intervention-arm participants to test the hypothesized "context-intervention-mechanism-outcome" configurations. In step 3, we plan to triangulate all sources of data using a coincidence analysis to identify the necessary combinations of factors that determine whether and how the desired outcomes are achieved and use these insights to consolidate the program theory.

RESULTS

For step 1 analysis, we have developed the initial program theory and the corresponding data collection plan. For step 2 analysis, participant enrollment for the randomized controlled trial started in January 2023. Participant enrollment for this realist evaluation was anticipated to start in July 2023 and continue until we reached thematic saturation or achieved informational power.

CONCLUSIONS

Beyond contributing to knowledge on the multiple pathways that lead to successful engagement with a digital health intervention as well as target outcomes in T1D care transitions, embedding the realist evaluation alongside the trial may inform real-time intervention refinement to improve user engagement and transition experiences. The knowledge gained from this study may inform the design, implementation, and evaluation of future digital health interventions that aim to improve transition experiences.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/46115.

Community health workers' involvement in mother-child care during the 1st year after birth, in Kaya health district, Burkina Faso: A contribution analysis

Introduction

Maternal and infant morbidity and mortality remain high in sub-Saharan Africa. However, actions to strengthen postpartum care are still weak and mainly limited to health facilities (HFs). In Kaya health district, Burkina Faso, community health workers (CHWs) were involved in mother and child care during the 1st year postpartum through home visits, outreach sessions and accompanying mothers to health facilities. The aim of this study was to assess the contribution of CHWs to postpartum women's attendance at the health facilities.

Methods

We conducted an effect assessment using Mayne and Lemire's contribution analysis framework. Qualitative and quantitative data were collected through project documents review and individual semi-structured interviews with key-informants.

Results

All the participants interviewed acknowledged that the number of women, who came to postpartum care, had increased since the implementation of the project activities. Postpartum consultation rates within the 1st week postpartum increased from 29% in 2011 to 80% in 2015 and from 19 to 50% within 6 weeks. Others interventions such as Performance based financing, Save The Children nutritional project and the health services component of Missed Opportunities in Mother and Infant Health (MOMI) were the alternative explanations.

Conclusions

CHWs involvement in women care contributed to improve their adherence to postpartum consultations in Kaya health district.

Sustainable Return to Work for Workers with Mental Health and Musculoskeletal Conditions

Common mental health and musculoskeletal disorders (CMDs and MSDs) are two of the most significant causes of non-participation in employment amongst working age adults.

BACKGROUND

This case study fills an important gap in the scientific literature on reintegration back to work after sickness absence due to CMDs and MSDs. It particularly examines the return to work (RTW) experiences of sick-listed employees to understand the facilitators and barriers of sustainable RTW.

METHODS

Using a realist evaluation approach within a qualitative inquiry, perceptions of employees were explored to provide in-depth understanding of what, how and under what circumstances sustainable RTW can be enabled for employees absent on a short- or long-term basis. Repeat face-to-face semi-structured interviews were conducted with 22 participants (15 women and 7 men, aged 30-50 years and sick-listed with MSDs and CMDs) who were recruited using purposive sampling. Data was thematically analysed.

RESULTS

A total of 2 main codes and 5 subcodes were developed and grouped into three theoretical abstractions. As a result of validating the context, mechanism, and outcome configurations with accounts of participants, all three initial theories explaining the most prominent mechanisms that either facilitates or impedes a sustainable RTW for people with CMDs and MSDs were justified.

CONCLUSIONS

Our findings reveal the active role of line managers on the RTW outcomes of returning employees. However, line-manager's competence and ability to effectively support and implement appropriate RTW strategies suited to employees' hinges on working in alignment with key stakeholders and returning employees.

Conducting focus groups in realist evaluation

Focus groups are valuable tools for evaluators to help stakeholders to clarify programme theories. In 1987, R.K. Merton, often attributed with the birth of focus groups, wrote about how these were 'being mercilessly misused'. In the 1940s, his team had conceived focus groups as tools for developing middle-range theory, but through their astonishing success focus groups have metamorphosed and are often an 'unchallenged' choice in many evaluation approaches, while their practice seems to provide a philosophically diverse picture. This article examines what knowledge focus group data generate, and how they support theory development. It starts with an overview of the history of focus groups, establishing a relationship between their emergence as a data collection method and the evaluation profession. Practical lessons for conducting groups in realist evaluation are suggested, while exploring how qualitative data can support programme and middle-range theory development using the example of realist evaluation.

Did an Intervention Programme Aimed at Strengthening the Maternal and Child Health Services in Nigeria Improve the Completeness of Routine Health Data Within the Health Management Information System?

BACKGROUND

During 2012-2015, the Federal Government of Nigeria launched the Subsidy Reinvestment and Empowerment Programme, a health system strengthening (HSS) programme with a Maternal and Child Health component (Subsidy Reinvestment and Empowerment Programme [SURE-P]/MCH), which was monitored using the Health Management Information Systems (HMIS) data reporting tools. Good quality data is essential for health policy and planning decisions yet, little is known on whether and how broad health systems strengthening programmes affect quality of data. This paper explores the effects of the SURE-P/MCH on completeness of MCH data in the National HMIS.

METHODS

This mixed-methods study was undertaken in Anambra state, southeast Nigeria. A standardized proforma was used to collect facility-level data from the facility registers on MCH services to assess the completeness of data from 2 interventions and one control clusters. The facility data was collected to cover before, during, and after the SURE-P intervention activities. Qualitative in-depth interviews were conducted with purposefully-identified health facility workers to identify their views and experiences of changes in data quality throughout the above 3 periods.

RESULTS

Quantitative analysis of the facility data showed that data completeness improved substantially, starting before SURE-P and continuing during SURE-P but across all clusters (ie, including the control). Also health workers felt data completeness were improved during the SURE-P, but declined with the cessation of the programme. We also found that challenges to data completeness are dependent on many variables including a high burden on providers for data collection, many variables to be filled in the data collection tools, and lack of health worker incentives.

CONCLUSION

Quantitative analysis showed improved data completeness and health workers believed the SURE-P/MCH had contributed to the improvement. The functioning of national HMIS are inevitably linked with other health systems components. While health systems strengthening programmes have a great potential for improved overall systems performance, a more granular understanding of their implications on the specific components such as the resultant quality of HMIS data, is needed.

Mecanismos de evaluaciones realistas en intervenciones de atención primaria en salud en poblaciones rurales y urbanas marginales

Objetivo.

Identificar los mecanismos reportados en intervenciones de atención primaria en salud (APS) en poblaciones rurales y urbanas marginales entre los años 1997 y 2019.

Métodos.

Se utilizó el método de revisión sistemática exploratoria para identificar la literatura. La búsqueda de literatura fue realizada en las bases de datos Medline (Pubmed y Ovid), Global health, Embase, Web of science, Scopus y Lilacs. La inclusión de literatura consideró estudios de intervención primaria, observacionales con metodologías cualitativas, cuantitativas o mixtas. Se incluyeron artículos en inglés, portugués y español publicados entre 1997 y 2019. Para la selección de documentos definitivos se utilizó Rayyan QCRI, se excluyeron fuentes de información de literatura gris o investigación en progreso.

Resultados.

A nivel individual se identificaron tres grupos de mecanismos: niveles de relaciones entre las intervenciones y los sujetos, transformaciones que pueden generar en los individuos y las relaciones recíprocas. A nivel institucional se relacionaron con la temporalidad, el dinero, relaciones de poder, relaciones de confianza, el sistema de salud, las expectativas y las condiciones administrativas.

Conclusiones.

Los mecanismos individuales son similares en los diferentes contextos, los mecanismos institucionales son susceptibles a la variación del contexto, se representan en la temporalidad, recursos económicos, relaciones de poder y de confianza, el sistema de salud, las expectativas y condiciones administrativas.

Village health worker motivation for better performance in a maternal and child health programme in Nigeria: A realist evaluation

Background

Community health workers play an important role in linking communities with formal health service providers, thereby improving access to and utilization of health care. A novel cadre of community health workers known as village health workers (VHWs) were recruited to create demand for maternal health services in the Nigerian Subsidy Reinvestment Programme (SURE-P/MCH). In this study, we investigated the role of contextual factors and underlying mechanisms motivating VHWs.

Methods

We used realist evaluation to understand the impact of a multi-intervention maternal and child health programme on VHW motivation using Anambra State as a case study. Initial working theories and logic maps were developed through literature review and stakeholder engagement; programme theories were developed and tested using focus group discussions and in-depth interviews with various stakeholder groups. Interview transcripts were analysed through an integrated approach of Context, Mechanism and Outcomes (CMO) categorisation and connecting, and matching of patterns of CMO configurations. Motivation theories were used to explain factors that influence VHW motivation. Explanatory configurations are reported in line with RAMESES reporting standards.

Results

The performance of VHWs in the SURE-P maternal and child health programme was linked to four main mechanisms of motivation: feelings of confidence, sense of identity or feeling of acceptance, feeling of happiness and hopefulness/expectation of valued outcome. These mechanisms were triggered by interactions of programme-specific contexts and resources such as training and supervision of VHWs by skilled health workers, provision of first aid kits and uniforms, and payments of a monthly stipend. The monthly payment was considered to be the most important motivational factor by VHWs. VHWs used a combination of innovative approaches to create demand for maternity services among pregnant women, and their performance was influenced by health system factors such as organisational capacity and culture, and societal factors such as relationship with the community and community support.

Conclusion

This paper highlights important contextual factors and mechanisms for VHW motivation that can be applied to other interventions that seek to strengthen community engagement and demand creation in primary health care. Future research on how to sustain VHW motivation is also required.

Implementation of child mental health service improvement plans in four low- and middle-income countries: stakeholders' perspectives

Children in low- and middle-income countries (LMIC) have high levels of unmet mental health needs, especially in disadvantaged communities. To address this gap, we developed a child mental health service improvement programme. This was co-facilitated using interprofessional principles and values in four countries, South Africa, Kenya, Turkey and Brazil. Eighteen stakeholders from different professions were interviewed after six months on their perspectives on enabling factors and challenges they faced in implementing service plans. Participants valued the holistic case management approach and scaled service model that underpinned the service plans. Emerging themes on participants' priorities related to service user participation, integrated care, and different levels of capacity-building. We propose that an integrated care model in LMIC contexts can maximize available resources, engage families and mobilize communities. Implementation requires concurrent actions at micro-, meso- and macro-level.

Implementing Electronic Health Records in Primary Care Using the Theory of Change: A Nigerian Case Study (Preprint)

Background

Objective

Methods

Results

Conclusions

Influence of Conditional Cash Transfers on the Uptake of Maternal and Child Health Services in Nigeria: Insights From a Mixed-Methods Study

Background: Increasing access to maternal and child health (MCH) services is crucial to achieving universal health coverage (UHC) among pregnant women and children under-five (CU5). The Nigerian government between 2012 and 2015 implemented an innovative MCH programme to reduce maternal and CU5 mortality by reducing financial barriers of access to essential health services. The study explores how the implementation of a financial incentive through conditional cash transfer (CCT) influenced the uptake of MCH services in the programme. Methods: The study used a descriptive exploratory approach in Anambra state, southeast Nigeria. Data was collected through qualitative [in-depth interviews (IDIs), focus group discussions (FGDs)] and quantitative (service utilization data pre- and post-programme) methods. Twenty-six IDIs were conducted with respondents who were purposively selected to include frontline health workers (n = 13), National and State policymakers and programme managers (n = 13). A total of sixteen FGDs were conducted with service users and their family members, village health workers, and ward development committee members from four rural communities. We drew majorly upon Skinner's reinforcement theory which focuses on human behavior in our interpretation of the influence of CCT in the uptake of MCH services. Manual content analysis was used in data analysis to pull together core themes running through the entire data set. Results: The CCTs contributed to increasing facility attendance and utilization of MCH services by reducing the financial barrier to accessing healthcare among pregnant women. However, there were unintended consequences of CCT which included a reduction in birth spacing intervals, and a reduction of trust in the health system when the CCT was suddenly withdrawn by the government. Conclusion: CCT improved the utilization of MCH, but the sudden withdrawal of the CCT led to the opposite effect because people were discouraged due to lack of trust in government to keep using the MCH services. Understanding the intended and unintended outcomes of CCT will help to build sustainable structures in policy designs to mitigate sudden programme withdrawal and its subsequent effects on target beneficiaries and the health system at large.

Tracing theories in realist evaluations of large-scale health programmes in low- and middle-income countries: experience from Nigeria

Realist evaluations (RE) are increasingly popular in assessing health programmes in low- and middle-income countries (LMICs). This article reflects on processes of gleaning, developing, testing, consolidating and refining two programme theories (PTs) from a longitudinal mixed-methods RE of a national maternal and child health programme in Nigeria. The two PTs, facility security and patient–provider trust, represent complex and diverse issues: trust is all encompassing although less tangible, while security is more visible. Neither PT was explicit in the original programme design but emerged from the data and was supported by substantive theories. For security, we used theories of fear of crime, which perceive security as progressing from structural, political and socio-economic factors. Some facilities with the support of communities erected fences, improved lighting and employed guards, which altogether contributed to reduced fear of crime from staff and patients and improved provision and uptake of health care. The social theories for the trust PT were progressively selected to disentangle trust-related micro, meso and macro factors from the deployment and training of staff and conditional cash transfers to women for service uptake. We used taxonomies of trust factors such as safety, benevolent concerns and capability. We used social capital theory to interpret the sustainability of ‘residual’ trust after the funding for the programme ceased. Our overarching lesson is that REs are important though time-consuming ways of generating context-specific implications for policy and practice within ever-changing contexts of health systems in LMICs. It is important to ensure that PTs are ‘pitched at the right level’ of abstraction. The resource-constrained context of LMICs with insufficient documentation poses challenges for the timely convergence of nuggets of evidence to inform PTs. A retroductive approach to REs requires iterative data collection and analysis against the literature, which require continuity, coherence and shared understanding of the analytical processes within collaborative REs.

Protocol for a realist synthesis of health systems responsiveness in low-income and middle-income countries

INTRODUCTION

Health systems responsiveness is a key objective of any health system, yet it is the least studied of all objectives particularly in low-income and middle-income countries. Research on health systems responsiveness highlights its multiple elements, for example, dignity and confidentiality. Little is known, however, about underlying theories of health systems responsiveness, and the mechanisms through which responsiveness works. This realist synthesis contributes to bridging these two knowledge gaps.

METHODS AND ANALYSIS

In this realist synthesis, we will use a four-step process, comprising: mapping of theoretical bases, formulation of programme theories, theory refinement and testing of programme theories using literature and empirical data from Ghana and Vietnam. We will include theoretical and conceptual pieces, reviews, empirical studies and grey literature, alongside the primary data. We will explore responsiveness as entailing external and internal interactions within health systems. The search strategy will be purposive and iterative, with continuous screening and refinement of theories. Data extraction will be combined with quality appraisal, using appropriate tools. Each fragment of evidence will be appraised as it is being extracted, for its relevance to the emerging programme theories and methodological rigour. The extracted data pertaining to contexts, mechanisms and outcomes will be synthesised to identify patterns and contradictions. Results will be reported using narrative explanations, following established guidance on realist syntheses.

ETHICS AND DISSEMINATION

Ethics approvals for the wider RESPONSE (Improving health systems responsiveness to neglected health needs of vulnerable groups in Ghana and Vietnam) study, of which this review is one part, were obtained from the ethics committees of the following institutions: London School of Hygiene and Tropical Medicine (ref: 22981), University of Leeds, School of Medicine (ref: MREC19-051), Ghana Health Service (ref: GHS-ERC 012/03/20) and Hanoi University of Public Health (ref: 020-149/DD-YTCC).We will disseminate results through academic papers, conference presentations and stakeholder workshops in Ghana and Vietnam.

PROSPERO REGISTRATION NUMBER

CRD42020200353. Full record: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020200353.

Exploring mechanisms that explain how coalition groups are formed and how they work to sustain political priority for maternal and child health in Nigeria using the advocacy coalition framework

Background

The unacceptably high rate of maternal and child mortality in Nigeria prompted the government to introduce a free maternal and child health (MCH) programme, which was stopped abruptly following a change in government. This triggered increased advocacy for sustaining MCH as a political priority in the country and led to the formation of advocacy coalitions. This study set out to explain the process involved in the formation of advocacy coalition groups and how they work to bring about sustained political prioritization for MCH in Nigeria. It will contribute to the understanding of the Nigerian MCH sector subsystem and will be beneficial to health policy advocates and public health researchers in Nigeria.

Methods

This study employed a qualitative case study approach. Data were collected using a pretested interview guide to conduct 22 in-depth interviews, while advocacy events were reviewed pro forma. The document review was analysed using the manual content analysis method, while qualitative data audiotapes were transcribed verbatim, anonymized, double-coded in MS Word using colour-coded highlights and analysed using manual thematic and framework analysis guided by the advocacy coalition framework (ACF). The ACF was used to identify the policy subsystem including the actors, their belief, coordination and resources, as well as the effects of advocacy groups on policy change. Ethics and consent approval were obtained for the study.

Results

The policy subsystem identified the actors and characterized the coalitions, and described their group formation processes and resources/strategies for engagement. The perceived deep core belief driving the MCH agenda is the right of an individual to health. The effects of advocacy groups on policy change were identified, along with the factors that enabled effectiveness, as well as constraints to coalition formation. External factors and triggers of coalition formation were identified to include high maternal mortality and withdrawal of the free MCH programme, while the contextual issues were the health system issues and the socioeconomic factors affecting the country.

Conclusion

Our findings add to an increasing body of evidence that the use of ACF is beneficial in exploring how advocacy coalitions are formed and in identifying the effects of advocacy groups on policy change.

Realist evaluation to improve health systems responsiveness to neglected health needs of vulnerable groups in Ghana and Vietnam: Study protocol

BACKGROUND

Socio-economic growth in many low and middle-income countries has resulted in more available, though not equitably accessible, healthcare. Such growth has also increased demands from citizens for their health systems to be more responsive to their needs. This paper shares a protocol for the RESPONSE study which aims to understand, co-produce, implement and evaluate context-sensitive interventions to improve health systems responsiveness to health needs of vulnerable groups in Ghana and Vietnam.

METHODS

We will use a realist mixed-methods theory-driven case study design, combining quantitative (household survey, secondary analysis of facility data) and qualitative (in-depth interviews, focus groups, observations and document and literature review) methods. Data will be analysed retroductively. The study will comprise three Phases. In Phase 1, we will understand actors' expectations of responsive health systems, identify key priorities for interventions, and using evidence from a realist synthesis we will develop an initial theory and generate a baseline data. In Phase 2, we will co-produce jointly with key actors, the context-sensitive interventions to improve health systems responsiveness. The interventions will seek to improve internal (i.e. intra-system) and external (i.e. people-systems) interactions through participatory workshops. In Phase 3, we will implement and evaluate the interventions by testing and refining our initial theory through comparing the intended design to the interventions' actual performance.

DISCUSSION

The study's key outcomes will be: (1) improved health systems responsiveness, contributing to improved health services and ultimately health outcomes in Ghana and Vietnam and (2) an empirically-grounded and theoretically-informed model of complex contexts-mechanisms-outcomes relations, together with transferable best practices for scalability and generalisability. Decision-makers across different levels will be engaged throughout. Capacity strengthening will be underpinned by in-depth understanding of capacity needs and assets of each partner team, and will aim to strengthen individual, organisational and system level capacities.

Implementing large-system, value-based healthcare initiatives: a realist study protocol for seven natural experiments

INTRODUCTION

Value-based healthcare delivery models have emerged to address the unprecedented pressure on long-term health system performance and sustainability and to respond to the changing needs and expectations of patients. Implementing and scaling the benefits from these care delivery models to achieve large-system transformation are challenging and require consideration of complexity and context. Realist studies enable researchers to explore factors beyond 'what works' towards more nuanced understanding of 'what tends to work for whom under which circumstances'. This research proposes a realist study of the implementation approach for seven large-system, value-based healthcare initiatives in New South Wales, Australia, to elucidate how different implementation strategies and processes stimulate the uptake, adoption, fidelity and adherence of initiatives to achieve sustainable impacts across a variety of contexts.

METHODS AND ANALYSIS

This exploratory, sequential, mixed methods realist study followed RAMESES II (Realist And Meta-narrative Evidence Syntheses: Evolving Standards) reporting standards for realist studies. Stage 1 will formulate initial programme theories from review of existing literature, analysis of programme documents and qualitative interviews with programme designers, implementation support staff and evaluators. Stage 2 envisages testing and refining these hypothesised programme theories through qualitative interviews with local hospital network staff running initiatives, and analyses of quantitative data from the programme evaluation, hospital administrative systems and an implementation outcome survey. Stage 3 proposes to produce generalisable middle-range theories by synthesising data from context-mechanism-outcome configurations across initiatives. Qualitative data will be analysed retroductively and quantitative data will be analysed to identify relationships between the implementation strategies and processes, and implementation and programme outcomes. Mixed methods triangulation will be performed.

ETHICS AND DISSEMINATION

Ethical approval has been granted by Macquarie University (Project ID 23816) and Hunter New England (Project ID 2020/ETH02186) Human Research Ethics Committees. The findings will be published in peer-reviewed journals. Results will be fed back to partner organisations and roundtable discussions with other health jurisdictions will be held, to share learnings.

Was the Maternal Health Cash Transfer Programme in Nigeria Sustainable and Cost-Effective?

Background: The Subsidy Reinvestment and Empowerment Programme (SURE-P), Maternal and Child Health (MCH) was introduced by the Nigerian government to increase the use of skilled maternal health services and reduce maternal mortality. The programme, funded out of a reduction in the fuel subsidy, was implemented between October 2012 and April 2015 and incorporated a conditional cash transfer to women to encourage use of facility based maternal services. We seek to assess the incremental cost effectiveness and long term impact of the conditional cash transfer element of the programme. Methods: An impact analysis and incremental cost-effectiveness analysis of conditional cash transfers (CCTs) is undertaken taking a health service perspective toward costs of the intervention. The study was undertaken in Anambra state, comparing areas that received only the investment in health services with areas that implemented the conditional cash transfer programme. An interrupted time series analysis of the programme outputs was undertaken. These were combined with a programme costing to determine the incremental cost per output. Findings: Maternal services provided to patients in conditional cash transfer areas accelerated rapidly from the middle of 2014 until after the programme in late 2015. The costs of providing services in each Primary Health Center facility was US $52,128 in the areas that only invested in health services compared to US $90,702 in facilities that also provided cash transfers. Much of the additional cost was in managing cash transfers. The incremental cost in the cash transfer areas was $572 for delivery care and $11 for antenatal care. If the programme was to be integrated as a regular service in the public health system, the cost of a delivery is estimated to fall to $389 and to $188 if 2015 levels of activity are assumed. Conclusion: Although the cost of CCTs as originally constituted as a vertical programme are relatively high compared to other similar programmes, these would fall substantially if integrated into the main health system. There is also evidence of sustained impact beyond the end of the funding suggesting that short term programmes can lead to a long-term change in patterns of health seeking behavior.

"If you are on duty, you may be afraid to come out to attend to a person": fear of crime and security challenges in maternal acute care in Nigeria from a realist perspective

BACKGROUND

Maternal and Child Health is a global priority. Access and utilization of facility-based health services remain a challenge in low and middle-income countries. Evidence on barriers to providing and accessing services omits information on the role of security within facilities. This paper explores the role of security in the provision and use of maternal health services in primary healthcare facilities in Nigeria.

METHODS

Study was carried out in Anambra state, Nigeria. Qualitative data were initially collected from 35 in-depth interviews and 24 focus groups with purposively identified key informants. Information gathered was used to build a programme theory that was tested with another round of interviews (17) and focus group (4) discussions. Data analysis and reporting were based on the Context-Mechanism-Outcome heuristic of Realist Evaluation methodology.

RESULTS

The presence of a male security guard in the facility was the most important security factor that facilitated provision and uptake of services. Others include perimeter fencing, lighting and staff accommodation. Lack of these components constrained provision and use of services, by impacting on behaviour of staff and patients. Security concerns of facility staff who did not feel safe to let in people into unguarded facilities, mirrored those of pregnant women who did not utilize health facilities because of fear of not being let in and attended to by facility staff.

CONCLUSION

Health facility security should be key consideration in programme planning, to avert staff and women's fear of crime which currently constrains provision and use of maternal healthcare at health facilities.

What makes advocacy work? Stakeholders' voices and insights from prioritisation of maternal and child health programme in Nigeria

BACKGROUND

The Nigerian government introduced and implemented a health programme to improve maternal and child health (MCH) called Subsidy Reinvestment and Empowerment programme for MCH (SURE-P/MCH). It ran from 2012 and ended abruptly in 2015 and was followed by increased advocacy for sustaining the MCH (antenatal, delivery, postnatal and immunization) services as a policy priority. Advocacy is important in allowing social voice, facilitating prioritization, and bringing different forces/actors together. Therefore, the study set out to understand how advocacy works - through understanding what effective advocacy implementation processes comprise and what mechanisms are triggered by which contexts to produce the intended outcomes.

METHODS

The study used a Realist Evaluation design through a mixed quantitative and qualitative methods case study approach. The programme theory (PT) was developed from three substantive social theories (power politics, media influence communication theory, and the three-streams theory of agenda-setting), data and programme design documentation, and subsequently tested. We report information from 22 key informant interviews including national and State policy and law makers, policy implementers, CSOs, Development partners, NGOs, health professional groups, and media practitioners and review of relevant documents on advocacy events post-SURE-P.

RESULTS

Key advocacy organizations and individuals including health professional groups, the media, civil society organizations, powerful individuals, and policymakers were involved in advocacy activities. The nature of their engagement included organizing workshops, symposiums, town hall meetings, individual meetings, press conferences, demonstrations, and engagements with media. Effective advocacy mechanism involved alliance brokering to increase influence, the media supporting and engaging in advocacy, and the use of champions, influencers, and spouses (Leadership and Elite Gendered Power Dynamics). The key contextual influences which determined the effectiveness of advocacy measures for MCH included the political cycle, availability of evidence on the issue, networking with powerful and interested champions, and alliance building in advocacy. All these enhanced the entrenchment of MCH on the political and financial agenda at the State and Federal levels.

CONCLUSIONS

Our result suggest that advocacy can be a useful tool to bring together different forces by allowing expression of voices and ensuring accountability of different actors including policymakers. In the context of poor health outcomes, interest from policymakers and politicians in MCH, combined with advocacy from key policy actors armed with evidence, can improve prioritization and sustained implementation of MCH services.

Which mechanisms explain motivation the of primary health workers? Insights from the realist evaluation of a maternal and child health programme in Nigeria

INTRODUCTION

Well-trained, adequately skilled and motivated primary healthcare (PHC) workers are essential for attaining universal health coverage (UHC). While there is abundant literature on the drivers of workforce motivation, published knowledge on the mechanisms of motivation within different contexts is limited, particularly in resource-limited countries. This paper contributes to health workforce literature by reporting on how motivation works among PHC workers in a maternal and child health (MCH) programme in Nigeria.

METHODS

We adopted a realist evaluation design combining document review with 56 in-depth interviews of PHC workers, facility managers and policy-makers to assess the impact of the MCH programme in Anambra State, Nigeria. A realist process of theory development, testing and consolidation was used to understand how and under what circumstances the MCH programme impacted on workers' motivation and which mechanisms explain how motivation works. We drew on Herzberg's two-factor and Adam's equity theories to unpack how context shapes worker motivation.

RESULTS

A complex and dynamic interaction between the MCH programme and organisational and wider contexts triggered five mechanisms which explain PHC worker motivation: (1) feeling supported, (2) feeling comfortable with work environment, (3) feeling valued, (4) morale and confidence to perform tasks and (5) companionship. Some mechanisms were mutually reinforcing while others operated in parallel. Other conditions that enabled worker motivation were organisational values of fairness, recognition of workers' contributions and culture of task-sharing and teamwork.

CONCLUSIONS

Policy designs and management strategies for improving workforce performance, particularly in resource-constrained settings should create working environments that foster feelings of being valued and supported while enabling workers to apply their knowledge and skills to improve healthcare delivery and promote UHC. Future research can test the explanatory framework generated by this study and explore differences in motivational mechanisms among different cadres of PHC workers to inform cadre-related motivational interventions.

A realist process evaluation of Enhanced Triple P for Baby and Mellow Bumps, within a Trial of Healthy Relationship Initiatives for the Very Early years (THRIVE): study protocol for a randomized controlled trial

Background

THRIVE is a three-arm randomised controlled trial (RCT) that aims to evaluate whether antenatal and early postnatal interventions, Enhanced Triple B for Baby (ETPB) plus care as usual (CAU) or Mellow Bumps (MB) plus CAU (versus CAU alone), can: 1) improve the mental health and well-being of pregnant women with complex health and social care needs; 2) improve mother-infant bonding and interaction; 3) reduce child maltreatment; and 4) improve child language acquisition. This paper focuses on THRIVE’s realist process evaluation, which is carefully monitoring what is happening in the RCT.

Methods

Realistic evaluation provides the theoretical rationale for the process evaluation. We question: 1) how faithfully are MB and ETPB implemented? 2) What are the mechanisms by which they work, if they do, and who do they work for and how? 3) What contextual factors are necessary for the programmes to function, or might prevent them functioning?

The mixed-methods design includes quantitative measures, which are pre- and post-training/intervention questionnaires for facilitators and mothers-to-be, and post-session evaluation forms. Qualitative data collection methods include participant observation of facilitator training and the delivery of a series of antenatal sessions in selected intervention groups (n = 3 for ETPB and n = 3 for MB), semi-structured interviews with facilitators, pregnant women, partners, and referring facilitators, and telephone interviews examining the content of the postnatal components of ETPB and MB.

Discussion

The findings of this process evaluation will help researchers and decision makers interpret the outcomes of THRIVE. It will provide a greater understanding of: how the interventions work (if they do); the extent and quality of their implementation; contextual factors facilitating and constraining intervention functioning; variations in response within and between subgroups of vulnerable parents; and benefits or unintended consequences of either intervention. Few studies to date have published detailed research protocols illustrating how realist process evaluation is designed and conducted as an integral part of a randomised controlled trial.

Trial registration

ISRCTN, ISRCTN21656568. Registered on 8 November 2013.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3395-3) contains supplementary material, which is available to authorized users.

Dealing with context in logic model development: Reflections from a realist evaluation of a community health worker programme in Nigeria

Logic models (LMs) have been used in programme evaluation for over four decades. Current debate questions the ability of logic modelling techniques to incorporate contextual factors into logic models. We share experience of developing a logic model within an ongoing realist evaluation which assesses the extent to which, and under what circumstances a community health workers (CHW) programme promotes access to maternity services in Nigeria. The article contributes to logic modelling debate by: i) reflecting on how other scholars captured context during LM development in theory-driven evaluations; and ii) explaining how we explored context during logic model development for realist evaluation of the CHW programme in Nigeria. Data collection methods that informed our logic model development included documents review, email discussions and teleconferences with programme stakeholders and a technical workshop with researchers to clarify programme goals and untangle relationships among programme elements. One of the most important findings is that, rather than being an end in itself, logic model development is an essential step for identifying initial hypotheses for tentative relevant contexts, mechanisms and outcomes (CMOs) and CMO configurations of how programmes produce change. The logic model also informed development of a methodology handbook that is guiding verification and consolidation of underlying programme theories.

Impact of using eHealth tools to extend health services to rural areas of Nigeria: protocol for a mixed-method, non-randomised cluster trial

INTRODUCTION

eHealth solutions that use internet and related technologies to deliver and enhance health services and information are emerging as novel approaches to support healthcare delivery in sub-Saharan Africa. Using digital technology in this way can support cost-effectiveness of care delivery and extend the reach of services to remote locations. Despite the burgeoning literature on eHealth approaches, little is known about the effectiveness of eHealth tools for improving the quality and efficiency of health systems functions or client outcomes in resource-limited countries. eHealth tools including satellite communications are currently being implemented at scale, to extend health services to rural areas of Nigeria, in Ondo and Kano States and the Federal Capital Territory. This paper shares the protocol for a 2-year project ('EXTEND') that aims to evaluate the impact of eHealth tools on health system functions and health outcomes.

METHODOLOGY AND ANALYSIS

This multisite, mixed-method evaluation includes a non-randomised, cluster trial design. The study comprises three phases-baseline, midline and endline evaluations-that involve: (1) process evaluation of video training and digitisation of health data interventions; (2) evaluation of contextual influences on the implementation of interventions; and (3) impact evaluation of results of the project. A convergent mixed-method model will be adopted to allow integration of quantitative and qualitative findings to achieve study objectives. Multiple quantitative and qualitative datasets will be repeatedly analysed and triangulated to facilitate better understanding of impact of eHealth tools on health worker knowledge, quality and efficiency of health systems and client outcomes.

ETHICS AND DISSEMINATION

Ethics approvals were obtained from the University of Leeds and three States' Ministries of Health in Nigeria. All data collected for this study will be anonymised and reports will not contain information that could identify respondents. Study findings will be presented to Ministries of Health at scientific conferences and published in peer-reviewed journals.

TRIAL REGISTRATION NUMBER

ISRCTN32105372; Pre-results.

Characterizing performance improvement in primary care systems in Mesoamerica: A realist evaluation protocol

Background. Evaluations of performance measurement and management interventions in public, primary care delivery systems of low- and middle-income countries are scarce. In such contexts, few studies to date have focused on characterizing how, why and under what contextual conditions do such complex, multifaceted arrangements lead to intended and unintended consequences for the healthcare workforce, the healthcare organizations involved, and the communities that are served. Methods. Case-study design with purposeful outlier sampling of high-performing primary care delivery systems in El Salvador and Honduras, as part of the Salud Mesoamerica Initiative. Case study design is suitable for characterizing individual, interpersonal and collective mechanisms of change in complex adaptive systems. The protocol design includes literature review, document review, non-participant observation, and qualitative analysis of in-depth interviews. Data analysis will use inductive and deductive approaches to identify causal patterns organized as ‘context-mechanism-outcome’ configurations. Findings will be triangulated with existing secondary data sources collected including country-specific performance measurement data, impact, and process evaluations conducted by the Salud Mesoamerica Initiative. Discussion. This realist evaluation protocol aims to characterize how, why and under what conditions do performance measurement and management arrangements contribute to the improvement of primary care system performance in two low-income countries.

Access to primary care for socio-economically disadvantaged older people in rural areas: exploring realist theory using structural equation modelling in a linked dataset

Background

Realist approaches seek to answer questions such as ‘how?’, ‘why?’, ‘for whom?’, ‘in what circumstances?’ and ‘to what extent?’ interventions ‘work’ using context-mechanism-outcome (CMO) configurations. Quantitative methods are not well-established in realist approaches, but structural equation modelling (SEM) may be useful to explore CMO configurations. Our aim was to assess the feasibility and appropriateness of SEM to explore CMO configurations and, if appropriate, make recommendations based on our access to primary care research. Our specific objectives were to map variables from two large population datasets to CMO configurations from our realist review looking at access to primary care, generate latent variables where needed, and use SEM to quantitatively test the CMO configurations.

Methods

A linked dataset was created by merging individual patient data from the English Longitudinal Study of Ageing and practice data from the GP Patient Survey. Patients registered in rural practices and who were in the highest deprivation tertile were included. Three latent variables were defined using confirmatory factor analysis. SEM was used to explore the nine full CMOs. All models were estimated using robust maximum likelihoods and accounted for clustering at practice level. Ordinal variables were treated as continuous to ensure convergence.

Results

We successfully explored our CMO configurations, but analysis was limited because of data availability. Two hundred seventy-six participants were included. We found a statistically significant direct (context to outcome) or indirect effect (context to outcome via mechanism) for two of nine CMOs. The strongest association was between ‘ease of getting through to the surgery’ and ‘being able to get an appointment’ with an indirect mediated effect through convenience (proportion of the indirect effect of the total was 21%). Healthcare experience was not directly associated with getting an appointment, but there was a statistically significant indirect effect through convenience (53% mediated effect). Model fit indices showed adequate fit.

Conclusions

SEM allowed quantification of CMO configurations and could complement other qualitative and quantitative techniques in realist evaluations to support inferences about strengths of relationships. Future research exploring CMO configurations with SEM should aim to collect, preferably continuous, primary data.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0514-x) contains supplementary material, which is available to authorized users.

Reporting the methods used in public health research and practice

The methods section of a scientific article often receives the most scrutiny from journal editors, peer reviewers, and skeptical readers because it allows them to judge the validity of the results. The methods section also facilitates critical interpretation of study activities, explains how the study avoided or corrected for bias, details how the data support the answer to the study question, justifies generalizing the findings to other populations, and facilitates comparison with past or future studies. In 2006, the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Programme began collecting and disseminating guidelines for reporting health research studies. In addition, guidelines for reporting public health investigations not classified as research have also been developed. However, regardless of the type of study or scientific report, the methods section should describe certain core elements: the study design; how participants were selected; the study setting; the period of interest; the variables and their definitions used for analysis; the procedures or instruments used to measure exposures, outcomes, and their association; and the analyses. Specific requirements for each study type should be consulted during the project planning phase and again when writing begins. We present requirements for reporting methods for public health activities, including outbreak investigations, public health surveillance programs, prevention and intervention program evaluations, research, surveys, systematic reviews, and meta-analyses.

Protocol for a mixed-methods realist evaluation of a health service user feedback system in Bangladesh

INTRODUCTION

Responsiveness to service users' views is a widely recognised objective of health systems. A key component of responsive health systems is effective interaction between users and service providers. Despite a growing literature on patient feedback from high-income settings, less is known about effectiveness of such systems in low-income and middle-income countries.

METHODOLOGY AND ANALYSIS

This paper disseminates the protocol for an 18-month 'RESPOND' project that aims to evaluate the system of collecting and responding to user feedback in Bangladesh. This mixed-method study uses a realist evaluation approach to examine user feedback systems at two Upazila health complexes in Comilla District of Bangladesh, and comprises three steps: (1) initial theory development; (2) theory validation; and (3) theory refinement and development of lessons learnt. The project also uses (1) process evaluation to understand causal mechanisms and contexts of implementation; (2) statistical analysis of patient feedback to clarify the nature of issues reported; (3) social science methods to illuminate feedback processes and user and provider experiences; and (4) health policy and systems research to clarify issues related to integration of feedback systems with quality assurance and human resource management. During data analysis, qualitative and quantitative findings will be integrated throughout to help achieve study objectives. Analysis of qualitative and quantitative data will be done using a convergent mixed-methods model, involving continuous triangulation of multiple data sets to facilitate greater understanding of the context of user feedback systems including the links with relevant policies, practices and programmes.

ETHICS AND DISSEMINATION

Ethics approvals were obtained from the University of Leeds and the Bangladesh Medical Research Council. All data collected for this study will be anonymised, and identifying characteristics of respondents will not appear in a final manuscript or reports. The study findings will be presented at scientific conferences and published in peer-reviewed journals.

RAMESES II reporting standards for realist evaluations

BACKGROUND

Realist evaluation is increasingly used in health services and other fields of research and evaluation. No previous standards exist for reporting realist evaluations. This standard was developed as part of the RAMESES II project. The project's aim is to produce initial reporting standards for realist evaluations.

METHODS

We purposively recruited a maximum variety sample of an international group of experts in realist evaluation to our online Delphi panel. Panel members came from a variety of disciplines, sectors and policy fields. We prepared the briefing materials for our Delphi panel by summarising the most recent literature on realist evaluations to identify how and why rigour had been demonstrated and where gaps in expertise and rigour were evident. We also drew on our collective experience as realist evaluators, in training and supporting realist evaluations, and on the RAMESES email list to help us develop the briefing materials. Through discussion within the project team, we developed a list of issues related to quality that needed to be addressed when carrying out realist evaluations. These were then shared with the panel members and their feedback was sought. Once the panel members had provided their feedback on our briefing materials, we constructed a set of items for potential inclusion in the reporting standards and circulated these online to panel members. Panel members were asked to rank each potential item twice on a 7-point Likert scale, once for relevance and once for validity. They were also encouraged to provide free text comments.

RESULTS

We recruited 35 panel members from 27 organisations across six countries from nine different disciplines. Within three rounds our Delphi panel was able to reach consensus on 20 items that should be included in the reporting standards for realist evaluations. The overall response rates for all items for rounds 1, 2 and 3 were 94 %, 76 % and 80 %, respectively.

CONCLUSION

These reporting standards for realist evaluations have been developed by drawing on a range of sources. We hope that these standards will lead to greater consistency and rigour of reporting and make realist evaluation reports more accessible, usable and helpful to different stakeholders.