How does capacity building of health managers work? A realist evaluation study protocol

How capacity building of district health managers has been designed, delivered and evaluated in sub-Saharan Africa: a scoping review and best fit framework analysis

OBJECTIVES

We aimed to understand how capacity building programmes (CBPs) of district health managers (DHMs) have been designed, delivered and evaluated in sub-Saharan Africa. We focused on identifying the underlying assumptions behind leadership and management CBPs at the district level.

DESIGN

Scoping review.

DATA SOURCES

We searched five electronic databases (MEDLINE, Health Systems Evidence, Wiley Online Library, Cochrane Library and Google Scholar) on 6 April 2021 and 13 October 2022. We also searched for grey literature and used citation tracking.

ELIGIBILITY CRITERIA

We included all primary studies (1) reporting leadership or management capacity building of DHMs, (2) in sub-Saharan Africa, (3) written in English or French and (4) published between 1 January 1987 and 13 October 2022.

DATA EXTRACTION AND SYNTHESIS

Three independent reviewers extracted data from included articles. We used the best fit framework synthesis approach to identify an a priori framework that guided data coding, analysis and synthesis. We also conducted an inductive analysis of data that could not be coded against the a priori framework.

RESULTS

We identified 2523 papers and ultimately included 44 papers after screening and assessment for eligibility. Key findings included (1) a scarcity of explicit theories underlying CBPs, (2) a diversity of learning approaches with increasing use of the action learning approach, (3) a diversity of content with a focus on management rather than leadership functions and (4) a diversity of evaluation methods with limited use of theory-driven designs to evaluate leadership and management capacity building interventions.

CONCLUSION

This review highlights the need for explicit and well-articulated programme theories for leadership and management development interventions and the need for strengthening their evaluation using theory-driven designs that fit the complexity of health systems.

Capacity building of district health management teams in the era of provincial health administration reform in the Democratic Republic of Congo: a realist evaluation protocol

INTRODUCTION

In 2006, the Congolese Ministry of Health developed a health system strengthening strategy focusing on health district development. This strategy called for reforming the provincial health administration in order to better support the health district development through leadership and management capacity building of district health management teams. The implementation is currently underway, yet, more evidence on how, for whom and under what conditions this capacity building works is needed. The proposed research aims to address this gap using a realist evaluation approach.

METHODS AND ANALYSIS

We will follow the cycle of the realist evaluation. First, we will elicit the initial programme theory through a scoping review (completed in December 2022, using MEDLINE, Health Systems Evidence, Wiley Online Library, Cochrane Library, Google Scholar and grey literature), a review of health policy documents (completed in March 2023), and interviews with key stakeholders (by June 2023). Second, we will empirically test the initial programme theory using a multiple-embedded case study design in two provincial health administrations and four health districts (by March 2024). Data will be collected through document reviews, in-depth interviews, non-participant observations, a questionnaire, routine data from the health information management system and a context mapping tool. We will analyse data using the Intervention-Context-Actor-Mechanism-Outcome configuration heuristic. Last, we will refine the initial programme theory based on the results of the empirical studies and develop recommendations for policymakers (by June 2024).

ETHICS AND DISSEMINATION

The Institutional Review Board of the Institute of Tropical Medicine and the Medical Ethics Committee of the University of Lubumbashi approved this study. We will also seek approvals from provincial-level and district-level health authorities before data collection in their jurisdictions. We will disseminate the study findings through the publication of articles in peer-reviewed academic journals, policy briefs for national policymakers and presentations at national and international conferences.

Enablers of sexual and reproductive health and rights interventions in low- and middle-income countries. Insights from capacity development projects implemented in 13 countries in Africa and Asia

Background

The global community has committed to achieving universal access to sexual and reproductive health and rights (SRHR) services, but how to do it remains a challenge in many low-income countries. Capacity development is listed as a means of implementation for Agenda 2030. Although it has been a major element in international development cooperation, including SRHR, its effectiveness and circumstances under which it succeeds or fails have limited evidence.

Objective

The study sought to examine whether improvement in team capacity of SRHR practitioners resulted in improved organisational effectiveness and/or improved SRHR outcomes in low-income countries.

Methods

The study involved 99 SRHR interventions implemented in 13 countries from Africa and Asia. Self-reported evaluation data from healthcare practitioners who participated in a capacity development international training programme in SRHR was used. The training was conducted by Lund University in Sweden between 2015 and 2019. Logistic regression models were used to examine the association between improved team capacity, improved organizational effectiveness and improved SRHR outcomes, for all the 99 interventions. Adoption of new SRHR approaches (guidelines and policies), media engagement, support from partner organisations and involvement of stakeholders were assessed as possible confounders.

Results

Improved team capacity, support from partner organisations and media engagement were positively associated with improved organisational effectiveness. Improved team capacity was the strongest predictor of organisational effectiveness even after controlling for other covariates at multivariate analysis. However, adopting new SRHR approaches significantly reduced organisational effectiveness. Furthermore, support from partner organisations was positively associated with increased awareness of and demand for SRHR services.

Conclusions

Successful implementation of capacity development interventions requires an enabling environment. In this study, an SRHR training programme aiming at improving team capacity resulted in an improvement in organisational effectiveness. Support from partner organisations and media engagement were key enablers of organisational effectiveness.

Conceptual framework for systemic capacity strengthening for health policy and systems research

Health policy and systems research (HPSR) is critical in developing health systems to better meet the health needs of their populations. The highly contextualised nature of health systems point to the value of local knowledge and the need for context-embedded HPSR. Despite such need, relatively few individuals, groups or organisations carry out HPSR, particularly in low-income and middle-income countries. Greater effort is required to strengthen capacity for, and build the field of, HPSR by capturing the multilevel and nuanced representation of HPSR across contexts. No comprehensive frameworks were found that inform systemic HPSR capacity strengthening. Existing literature on capacity strengthening for health research and development tends to focus on individual-level capacity with less attention to collective, organisational and network levels. This paper proposes a comprehensive framework for systemic capacity strengthening for HPSR, uniquely drawing attention to the blurred boundaries and amplification potential for synergistic capacity strengthening efforts across the individual, organisational and network levels. Further, it identifies guiding values and principles that consciously acknowledge and manage the power dynamics inherent to capacity strengthening work. The framework was developed drawing on available literature and was peer-reviewed by the Board and Thematic Working Groups of Health Systems Global. While the framework focuses on HPSR, it may provide a useful heuristic for systemic approaches to capacity strengthening more generally; facilitate its mainstreaming within organisations and networks and help maintain a focused approach to, and structure repositories of resources on, capacity strengthening.

Tracing theories in realist evaluations of large-scale health programmes in low- and middle-income countries: experience from Nigeria

Realist evaluations (RE) are increasingly popular in assessing health programmes in low- and middle-income countries (LMICs). This article reflects on processes of gleaning, developing, testing, consolidating and refining two programme theories (PTs) from a longitudinal mixed-methods RE of a national maternal and child health programme in Nigeria. The two PTs, facility security and patient–provider trust, represent complex and diverse issues: trust is all encompassing although less tangible, while security is more visible. Neither PT was explicit in the original programme design but emerged from the data and was supported by substantive theories. For security, we used theories of fear of crime, which perceive security as progressing from structural, political and socio-economic factors. Some facilities with the support of communities erected fences, improved lighting and employed guards, which altogether contributed to reduced fear of crime from staff and patients and improved provision and uptake of health care. The social theories for the trust PT were progressively selected to disentangle trust-related micro, meso and macro factors from the deployment and training of staff and conditional cash transfers to women for service uptake. We used taxonomies of trust factors such as safety, benevolent concerns and capability. We used social capital theory to interpret the sustainability of ‘residual’ trust after the funding for the programme ceased. Our overarching lesson is that REs are important though time-consuming ways of generating context-specific implications for policy and practice within ever-changing contexts of health systems in LMICs. It is important to ensure that PTs are ‘pitched at the right level’ of abstraction. The resource-constrained context of LMICs with insufficient documentation poses challenges for the timely convergence of nuggets of evidence to inform PTs. A retroductive approach to REs requires iterative data collection and analysis against the literature, which require continuity, coherence and shared understanding of the analytical processes within collaborative REs.

Bottom-up innovation for health management capacity development: a qualitative case study in a South African health district

BACKGROUND

As part of health system strengthening in South Africa (2012-2017) a new district health manager, taking a bottom-up approach, developed a suite of innovations to improve the processes of monthly district management team meetings, and the practices of managers and NGO partners attending them. Understanding capacity as a property of the health system rather than only of individuals, the research explored the mechanisms triggered in context to produce outputs, including the initial sensemaking by the district manager, the subsequent sensegiving and sensemaking in the team and how these homegrown innovations interacted with existing social processes and norms within the system.

METHODS

We conducted a realist evaluation, adopting the case study design, over a two-year period (2013-2015) in the district of focus. The initial programme theory was developed from 10 senior manager interviews and a literature review. To understand the processes and mechanisms triggered in the local context and identify outputs, we conducted 15 interviews with managers in the management team and seven with non-state actors. These were supplemented by researcher notes based on time spent in the district. Thematic analysis was conducted using the Context-Mechanism-Outcome configuration alongside theoretical constructs.

RESULTS

The new district manager drew on systems thinking, tacit and experiential knowledge to design bottom-up innovations. Capacity was triggered through micro-practices of sensemaking and sensegiving which included using sticks (positional authority, enforcement of policies, over-coding), intentionally providing justifications for change and setting the scene (a new agenda, distributed leadership). These micro-practices in themselves, and by managers engaging with them, triggered a generative process of buy-in and motivation which influenced managers and partners to participate in new practices within a routine meeting.

CONCLUSION

District managers are well placed to design local capacity development innovations and must draw on systems thinking, tacit and experiential knowledge to enable relevant 'bottom-up' capacity development in district health systems. By drawing on soft skills and the policy resources (hardware) of the system they can influence motivation and buy-in to improve management practices. From a systems perspective, we argue that capacity development can be conceived of as part of the daily activity of managing within routine spaces.

What mechanisms drive uptake of family planning when integrated with childhood immunisation in Ethiopia? A realist evaluation

Background

Maternal and child health are key priorities among the Sustainable Development Goals, which include a particular focus on reducing morbidity and mortality among women of reproductive age, newborns, and children under the age of five. Two components of maternal and child health are family planning (FP) and immunisation. Providing these services through an integrated delivery system could increase the uptake of vaccines and modern contraceptive methods (MCMs) particularly during the post-partum period.

Methods

A realist evaluation was conducted in two woredas in Ethiopia to determine the key mechanisms and their triggers that drive successful implementation and service uptake of an intervention of integrated delivery of immunisations and FP. The methodological approach included the development of an initial programme theory and the selection of relevant, published implementation related theoretical frameworks to aid organisation and cumulation of findings. Data from 23 semi-structured interviews were then analysed to determine key empirical mechanisms and drivers and to test the initial programme theory. These mechanisms were mapped against published theoretical frameworks and a revised programme theory comprised of context-mechanism-outcome configurations was developed. A critique of theoretical frameworks for abstracting empirical mechanisms was also conducted.

Results

Key contextual factors identified were: the use of trained Health Extension Workers (HEWs) to deliver FP services; a strong belief in values that challenged FP among religious leaders and community members; and a lack of support for FP from male partners based on religious values. Within these contexts, empirical mechanisms of acceptability, access, and adoption of innovations that drove decision making and intervention outcomes among health workers, religious leaders, and community members were identified to describe intervention implementation.

Conclusions

Linking context and intervention components to the mechanisms they triggered helped explain the intervention outcomes, and more broadly how and for whom the intervention worked. Linking empirical mechanisms to constructs of implementation related theoretical frameworks provided a level of abstraction through which findings could be cumulated across time, space, and conditions by theorising middle-range mechanisms.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-020-10114-8.

Understanding the influence of the MomConnect programme on antenatal and postnatal care service utilisation in two South African provinces: a realist evaluation protocol

INTRODUCTION

Timely antenatal care (ANC) and postnatal care (PNC) attendance decrease maternal and child mortality by improving maternal and child health (MCH) outcomes. Mobile health or mHealth has been identified as an effective way of improving the uptake of MCH services. The MomConnect programme is an mHealth initiative launched by the National Department of Health of South Africa in August 2014 to support MCH. Although widely used, there is a limited understanding of how, why, for whom and under which health system conditions, the implementation of MomConnect improves the health-seeking behaviour of pregnant women and mothers of infants in ANC and PNC facilities. This paper describes the protocol for a realist evaluation of the MomConnect programme, to provide a theory-based understanding of how, why and under what healthcare conditions the MomConnect programme works or not.

METHOD AND ANALYSIS

We will use the realist evaluation approach through its research cycle conducted in three phases. In phase I, a multimethod elicitation study design will be used, including a document review, key informant interviews and a scoping review to formulate an initial programme theory of the MomConnect intervention. Content and thematic analytic approaches will be used to analyse the data that will be fitted into a realist framework to formulate the initial programme theory. In phase II, a multi-case study design will be applied using a multimethod approach in two South African provinces. In each case, a theory-testing approach underpinned by the hypothetico-deduction analytic model will be used to test the initial programme theory. Surveys, interviews and focus group discussions will be conducted with various programme actors and analysed using appropriate methods. Phase III will entail refining the tested/modified programme theory through cross-case analysis.

EXPECTED OUTCOMES

An improved understanding of how and why the MomConnect intervention improves the health-seeking behaviour of pregnant women and mothers of infants, and the health system conditions that influence its implementation.

ETHICS AND DISSEMINATION

Ethics approval was granted by the Stellenbosch University Ethics Committee (S18/09/189). The protocol has been designed and the study will be conducted in line with the principles of the Declaration of Helsinki (1964).

Elements for harnessing participatory action research to strengthen health managers' capacity: a critical interpretative synthesis

BACKGROUND

Health managers play a key role in ensuring that health services are responsive to the needs of the population. Participatory action research (PAR) is one of the approaches that have been used to strengthen managers' capacity. However, collated knowledge on elements for harnessing PAR to strengthen managers' capacity is missing. This paper bridges this gap by reviewing existing literature on the subject matter.

METHODS

A critical interpretive synthesis method was used to interrogate eight selected articles. These articles reported the use of PAR to strengthen health managers' capacity. The critical interpretive synthesis method's approach to analysis guided the synthesis. Here, the authors interpretively made connections and linkages between different elements identified in the literature. Finally, the Atun et al. (Heal Pol Plann, 25:104-111, 2010) framework on integration was used to model the elements synthesised in the literature into five main domains.

RESULTS

Five elements with intricate bi-directional interactions were identified in the literature reviewed. These included a shared purpose, skilled facilitation and psychological safety, activity integration into organisational procedures, organisational support, and external supportive monitoring. A shared purpose of the managers' capacity strengthening initiative created commitment and motivation to learn. This purpose was built upon a set of facilitation skills that included promoting participation, self-efficacy and reflection, thereby creating a safe psychological space within which the managers interacted and learnt from each other and their actions. Additionally, an integrated intervention strengthened local capacity and harnessed organisational support for learning. Finally, supportive monitoring from external partners, such as researchers, ensured quality, building of local capacity and professional safety networks essential for continued learning.

CONCLUSIONS

The five elements identified in this synthesis provide a basis upon which the use of PAR can be harnessed, not only to strengthen health managers' capacity, but also to foster other health systems strengthening initiatives involving implementation research. In addition, the findings demonstrated the intricate and complex relations between the elements, which further affirms the need for a systems thinking approach to tackling health systems challenges.

Characterizing performance improvement in primary care systems in Mesoamerica: A realist evaluation protocol

Background. Improving performance of primary care systems in low- and middle-income countries (LMICs) may be a necessary condition for achievement of universal health coverage in the age of Sustainable Development Goals. The Salud Mesoamerica Initiative (SMI), a large-scale, multi-country program that uses supply-side financial incentives directed at the central-level of governments, and continuous, external evaluation of public, health sector performance to induce improvements in primary care performance in eight LMICs. This study protocol seeks to explain whether and how these interventions generate program effects in El Salvador and Honduras. Methods. This study presents the protocol for a study that uses a realist evaluation approach to develop a preliminary program theory that hypothesizes the interactions between context, interventions and the mechanisms that trigger outcomes. The program theory was completed through a scoping review of relevant empirical, peer-reviewed and grey literature; a sense-making workshop with program stakeholders; and content analysis of key SMI documents. The study will use a multiple case-study design with embedded units with contrasting cases. We define as a case the two primary care systems of Honduras and El Salvador, each with different context characteristics. Data will be collected through in-depth interviews with program actors and stakeholders, documentary review, and non-participatory observation. Data analysis will use inductive and deductive approaches to identify causal patterns organized as 'context, mechanism, outcome' configurations. The findings will be triangulated with existing secondary, qualitative and quantitative data sources, and contrasted against relevant theoretical literature. The study will end with a refined program theory. Findings will be published following the guidelines generated by the Realist and Meta-narrative Evidence Syntheses study (RAMESES II). This study will be performed contemporaneously with SMI's mid-term stage of implementation. Of the methods described, the preliminary program theory has been completed. Data collection, analysis and synthesis remain to be completed.

The challenge of complexity in evaluating health policies and programs: the case of women's participatory groups to improve antenatal outcomes

Background

During the last years, randomized designs have been promoted as the cornerstone of evidence-based policymaking. Also in the field of community participation, Random Control Trials (RCTs) have been the dominant design, used for instance to examine the contribution of community participation to health improvement. We aim at clarifying why RCTs and related (quasi-) experimental designs may not be the most appropriate approach to evaluate such complex programmes.

Results

We argue that the current methodological debate could be more fruitful if it would start from the position that the choice of designs should fit the nature of the program and research questions rather than be driven by methodological preferences. We present how realist evaluation, a theory-driven approach to research and evaluation, is a relevant methodology that could be used to assess whether and how community participation works. Using the realist evaluation approach to examine the relationship between participation and action of women groups and antenatal outcomes would enable evaluators to examine in detail the underlying mechanisms which influence actual practices and outcomes, as well as the context conditions required to make it work.

Conclusions

Realist research in fact allows opening the black boxes of “community” and “participation” in order to examine the role they play in ensuring cost-effective, sustainable interventions. This approach yields important information for policy makers and programme managers considering how such programs could be implemented in their own setting.

Protocol for a mixed-methods realist evaluation of a health service user feedback system in Bangladesh

INTRODUCTION

Responsiveness to service users' views is a widely recognised objective of health systems. A key component of responsive health systems is effective interaction between users and service providers. Despite a growing literature on patient feedback from high-income settings, less is known about effectiveness of such systems in low-income and middle-income countries.

METHODOLOGY AND ANALYSIS

This paper disseminates the protocol for an 18-month 'RESPOND' project that aims to evaluate the system of collecting and responding to user feedback in Bangladesh. This mixed-method study uses a realist evaluation approach to examine user feedback systems at two Upazila health complexes in Comilla District of Bangladesh, and comprises three steps: (1) initial theory development; (2) theory validation; and (3) theory refinement and development of lessons learnt. The project also uses (1) process evaluation to understand causal mechanisms and contexts of implementation; (2) statistical analysis of patient feedback to clarify the nature of issues reported; (3) social science methods to illuminate feedback processes and user and provider experiences; and (4) health policy and systems research to clarify issues related to integration of feedback systems with quality assurance and human resource management. During data analysis, qualitative and quantitative findings will be integrated throughout to help achieve study objectives. Analysis of qualitative and quantitative data will be done using a convergent mixed-methods model, involving continuous triangulation of multiple data sets to facilitate greater understanding of the context of user feedback systems including the links with relevant policies, practices and programmes.

ETHICS AND DISSEMINATION

Ethics approvals were obtained from the University of Leeds and the Bangladesh Medical Research Council. All data collected for this study will be anonymised, and identifying characteristics of respondents will not appear in a final manuscript or reports. The study findings will be presented at scientific conferences and published in peer-reviewed journals.

Building a competent health manager at district level: a grounded theory study from Eastern Uganda

BACKGROUND

Health systems in low-income countries are often characterized by poor health outcomes. While many reasons have been advanced to explain the persistently poor outcomes, management of the system has been found to play a key role. According to a WHO framework, the management of health systems is central to its ability to deliver needed health services. In this study, we examined how district managers in a rural setting in Uganda perceived existing approaches to strengthening management so as to provide a pragmatic and synergistic model for improving management capacity building.

METHODS

Twenty-two interviews were conducted with district level administrative and political managers, district level health managers and health facility managers to understand their perceptions and definitions of management and capacity building. Kathy Charmaz's constructive approach to grounded theory informed the data analysis process.

RESULTS

An interative, dynamic and complex model with three sub-process of building a competent health manager was developed. A competent manager was understood as one who knew his/her roles, was well informed and was empowered to execute management functions. Professionalizing health managers which was viewed as the foundation, the use of engaging learning approaches as the inside contents and having a supportive work environment the frame of the model were the sub-processes involved in the model. The sub-processes were interconnected although the respondents agreed that having a supportive work environment was more time and effort intensive relative to the other two sub-processes.

CONCLUSIONS

The model developed in our study makes four central contributions to enhance the WHO framework and the existing literature. First, it emphasizes management capacity building as an iterative, dynamic and complex process rather than a set of characteristics of competent managers. Second, our model suggests the need for professionalization of health managers at different levels of the health system. Third, our model underscores the benefits that could be accrued from the use of engaging learning approaches through prolonged and sustained processes that act in synergy. Lastly, our model postulates that different resource investments and a varied range of stakeholders could be required at each of the sub-processes.

How do community health committees contribute to capacity building for maternal and child health? A realist evaluation protocol

INTRODUCTION

The proposed research is part of ongoing operations research within World Vision's Access: Infant and Maternal Health Programme. This study aims to identify key context features and underlying mechanisms through which community health committees build community capacity within the field of maternal and child health. This may help to improve programme implementation by providing contextually informed and explanatory findings for how community health committees work, what works best and for whom do they work for best for. Though frequently used within health programmes, little research is carried out on such committees' contribution to capacity building-a frequent goal or proposed outcome of these groups.

METHODS AND ANALYSIS

The scarce information that does exist often fails to explain 'how, why, and for whom' these committees work best. Since such groups typically operate within or as components of complex health interventions, they require a systems thinking approach and design, and thus so too does their evaluation. Using a mixed methods realist evaluation with intraprogramme case studies, this protocol details a proposed study on community health committees in rural Tanzania and Uganda to better understand underlying mechanisms through which these groups work (or do not) to build community capacity for maternal and child health. This research protocol follows the realist evaluation methodology of eliciting initial programme theories, to inform the field study design, which are detailed within. Thus far, the methodology of a realist evaluation has been well suited to the study of community health committees within these contexts. Implications for its use within these contexts are discussed within.

ETHICS AND DISSEMINATION

Institutional Review Boards and the appropriate research clearance bodies within Ireland, Uganda and Tanzania have approved this study. Planned dissemination activities include via academic and programme channels, as well as feedback to the communities in which this work occurs.

Improving the performance of community health workers in humanitarian emergencies: a realist evaluation protocol for the PIECES programme

INTRODUCTION

Understanding what enhances the motivation and performance of community health workers (CHWs) in humanitarian emergencies represents a key research gap within the field of human resources for health. This paper presents the research protocol for the Performance ImprovEment of CHWs in Emergency Settings (PIECES) research programme. Enhancing Learning and Research in Humanitarian Action (ELRHA) funded the development of this protocol as part of their Health in Humanitarian Crises (R2HC) call (No.19839). PIECES aims to understand what factors improve the performance of CHWs in level III humanitarian emergencies.

METHODS AND ANALYSIS

The suggested protocol uses a realist evaluation with multiple cases across the 3 country sites: Turkey, Iraq and Lebanon. Working with International Medical Corps (IMC), an initial programme theory was elicited through literature and document reviews, semistructured interviews and focus groups with IMC programme managers and CHWs. Based on this initial theory, this protocol proposes a combination of semistructured interviews, life histories and critical incident narratives, surveys and latent variable modelling of key constructs to explain how contextual factors work to trigger mechanisms for specific outcomes relating to IMC's 300+ CHWs' performance. Participants will also include programme staff, CHWs and programme beneficiaries. Realist approaches will be used to better understand 'what works, for whom and under what conditions' for improving CHW performance within humanitarian contexts.

ETHICS AND DISSEMINATION

Trinity College Dublin's Health Policy and Management/Centre for Global Health Research Ethics Committee gave ethical approval for the protocol development phase. For the full research project, additional ethical approval will be sought from: Université St. Joseph (Lebanon), the Ethics Committee of the Ministry of Health in Baghdad (Iraq) and the Middle East Technical University (Turkey). Dissemination activities will involve a mixture of research feedback, policy briefs, guidelines and recommendations, as well as open source academic articles.

Study protocol: realist evaluation of effectiveness and sustainability of a community health workers programme in improving maternal and child health in Nigeria

BACKGROUND

Achievement of improved maternal and child health (MCH) outcomes continues to be an issue of international priority, particularly for sub-Saharan African countries such as Nigeria. Evidence suggests that the use of Community Health Workers (CHWs) can be effective in broadening access to, and coverage of, health services and improving MCH outcomes in such countries.

METHODS/DESIGN

In this paper, we report the methodology for a 5-year study which aims to evaluate the context, processes, outcomes and longer-term sustainability of a Nigerian CHW scheme. Evaluation of complex interventions requires a comprehensive understanding of intervention context, mechanisms and outcomes. The multidisciplinary and mixed-method realist approach will facilitate such evaluation. A favourable policy environment within which the study is conducted will ensure the successful uptake of results into policy and practice. A realist evaluation provides an overall methodological framework for this multidisciplinary and mixed methods research, which will be undertaken in Anambra state. The study will draw upon health economics, social sciences and statistics. The study comprises three steps: (1) initial theory development; (2) theory validation and (3) theory refinement and development of lessons learned. Specific methods for data collection will include in-depth interviews and focus group discussions with purposefully identified key stakeholders (managers, service providers and service users), document reviews, analyses of quantitative data from the CHW programme and health information system, and a small-scale survey. The impact of the programme on key output and outcome indicators will be assessed through an interrupted time-series analysis (ITS) of monthly quantitative data from health information system and programme reports. Ethics approvals for this study were obtained from the University of Leeds and the University of Nigeria.

DISCUSSION

This study will provide a timely and important contribution to health systems strengthening specifically within Anambra state in southeast Nigeria but also more widely across Nigeria. This paper should be of interest to researchers who are interested in adapting and applying robust methodologies for assessing complex health system interventions. The paper will also be useful to policymakers and practitioners who are interested in commissioning and engaging in such complex evaluations to inform policies and practices.

Midwifery capacity building in Papua New Guinea: Key achievements and ways forward

BACKGROUND

Papua New Guinea has some of the poorest health outcomes in the Asia-Pacific region. Maternal mortality is unacceptably high and there is a severe midwifery shortage requiring a quadrupling of the workforce.

AIM

This paper outlines the findings of an evaluation of the Maternal Child Health Initiative (MCHI) (2012-2013) to determine key factors contributing to maternal health workforce strengthening.

METHOD

A descriptive mixed methods study was undertaken. Data were gathered through interviews, focus group discussions and surveys with clinicians, midwifery students and staff from nursing and midwifery schools and National Department of Health staff. Documentation from stakeholder meetings and regular site reports were reviewed. Each data set was analysed separately and meta-inferences were drawn across all data.

FINDINGS

Learning opportunities were found to have increased for midwifery educators and improvements were described in midwifery educators teaching capacity and student clinical education experience. There was an increase in the number of midwifery graduates and improvements were noted in the working environment and skills of clinical staff. Education challenges were described including the lack of clinical preceptoring and limited continuing education for clinical educators. Participants recommended increasing clinical education hours and extending the length of the midwifery program. Ongoing efforts to accredit the midwifery curricula and regulate midwifery graduates were noted.

CONCLUSION

The MCHI has contributed to strengthening the midwifery workforce nationally. However, scaling-up and sustaining these achievements requires leadership and funding commitments from the midwifery schools and government alongside the accreditation of midwifery curricula and regulation of new graduates.

Advancing the application of systems thinking in health: a realist evaluation of a capacity building programme for district managers in Tumkur, India

Background

Health systems interventions, such as capacity-building of health workers, are implemented across districts in order to improve performance of healthcare organisations. However, such interventions often work in some settings and not in others. Local health systems could be visualised as complex adaptive systems that respond variously to inputs of capacity building interventions, depending on their local conditions and several individual, institutional, and environmental factors. We aim at demonstrating how the realist evaluation approach advances complex systems thinking in healthcare evaluation by applying the approach to understand organisational change within local health systems in the Tumkur district of southern India.

Methods

We collected data on several input, process, and outcome measures of performance of the talukas (administrative sub-units of the district) and explore the interplay between the individual, institutional, and contextual factors in contributing to the outcomes using qualitative data (interview transcripts and observation notes) and quantitative measures of commitment, self-efficacy, and supervision style.

Results

The talukas of Tumkur district responded differently to the intervention. Their responses can be explained by the interactions between several individual, institutional, and environmental factors. In a taluka with committed staff and a positive intention to make changes, the intervention worked through aligning with existing opportunities from the decentralisation process to improve performance. However, commitment towards the organisation was neither crucial nor sufficient. Committed staff in two other talukas were unable to actualise their intentions to improve organisational performance. In yet another taluka, the leadership was able to compensate for the lack of commitment.

Conclusions

Capacity building of local health systems could work through aligning or countering existing relationships between internal (individual and organisational) and external (policy and socio-political environment) attributes of the organisation. At the design and implementation stage, intervention planners need to identify opportunities for such triggering alignments. Local health systems may differ in their internal configuration and hence capacity building programmes need to accommodate possibilities for change through different pathways. By a process of formulating and testing hypotheses, making critical comparisons, discovering empirical patterns, and monitoring their scope and extent, a realist evaluation enables a comprehensive assessment of system-wide change in health systems.

Electronic supplementary material

The online version of this article (doi:10.1186/1478-4505-12-42) contains supplementary material, which is available to authorized users.

Evaluation of capacity-building program of district health managers in India: a contextualized theoretical framework

Performance of local health services managers at district level is crucial to ensure that health services are of good quality and cater to the health needs of the population in the area. In many low- and middle-income countries, health services managers are poorly equipped with public health management capacities needed for planning and managing their local health system. In the south Indian Tumkur district, a consortium of five non-governmental organizations partnered with the state government to organize a capacity-building program for health managers. The program consisted of a mix of periodic contact classes, mentoring and assignments and was spread over 30 months. In this paper, we develop a theoretical framework in the form of a refined program theory to understand how such a capacity-building program could bring about organizational change. A well-formulated program theory enables an understanding of how interventions could bring about improvements and an evaluation of the intervention. In the refined program theory of the intervention, we identified various factors at individual, institutional, and environmental levels that could interact with the hypothesized mechanisms of organizational change, such as staff’s perceived self-efficacy and commitment to their organizations. Based on this program theory, we formulated context–mechanism–outcome configurations that can be used to evaluate the intervention and, more specifically, to understand what worked, for whom and under what conditions. We discuss the application of program theory development in conducting a realist evaluation. Realist evaluation embraces principles of systems thinking by providing a method for understanding how elements of the system interact with one another in producing a given outcome.

Advancing the application of systems thinking in health: realist evaluation of the Leadership Development Programme for district manager decision-making in Ghana

Background

Although there is widespread agreement that strong district manager decision-making improves health systems, understanding about how the design and implementation of capacity-strengthening interventions work is limited. The Ghana Health Service has adopted the Leadership Development Programme (LDP) as one intervention to support the development of management and leadership within district teams. This paper seeks to address how and why the LDP ‘works’ when it is introduced into a district health system in Ghana, and whether or not it supports systems thinking in district teams.

Methods

We undertook a realist evaluation to investigate the outcomes, contexts, and mechanisms of the intervention. Building on two working hypotheses developed from our earlier work, we developed an explanatory case study of one rural district in the Greater Accra Region of Ghana. Data collection included participant observation, document review, and semi-structured interviews with district managers prior to, during, and after the intervention. Working backwards from an in-depth analysis of the context and observed short- and medium-term outcomes, we drew a causal loop diagram to explain interactions between contexts, outcomes, and mechanisms.

Results

The LDP was a valuable experience for district managers and teams were able to attain short-term outcomes because the novel approach supported teamwork, initiative-building, and improved prioritisation. However, the LDP was not institutionalised in district teams and did not lead to increased systems thinking. This was related to the context of high uncertainty within the district, and hierarchical authority of the system, which triggered the LDP’s underlying goal of organisational control.

Conclusions

Consideration of organisational context is important when trying to sustain complex interventions, as it seems to influence the gap between short- and medium-term outcomes. More explicit focus on systems thinking principles that enable district managers to better cope with their contexts may strengthen the institutionalisation of the LDP in the future.

How do primary health care teams learn to integrate intimate partner violence (IPV) management? A realist evaluation protocol

BACKGROUND

Despite the existence of ample literature dealing, on the one hand, with the integration of innovations within health systems and team learning, and, on the other hand, with different aspects of the detection and management of intimate partner violence (IPV) within healthcare facilities, research that explores how health innovations that go beyond biomedical issues-such as IPV management-get integrated into health systems, and that focuses on healthcare teams' learning processes is, to the best of our knowledge, very scarce if not absent. This realist evaluation protocol aims to ascertain: why, how, and under what circumstances primary healthcare teams engage (if at all) in a learning process to integrate IPV management in their practices; and why, how, and under what circumstances team learning processes lead to the development of organizational culture and values regarding IPV management, and the delivery of IPV management services.

METHODS

This study will be conducted in Spain using a multiple-case study design. Data will be collected from selected cases (primary healthcare teams) through different methods: individual and group interviews, routinely collected statistical data, documentary review, and observation. Cases will be purposively selected in order to enable testing the initial middle-range theory (MRT). After in-depth exploration of a limited number of cases, additional cases will be chosen for their ability to contribute to refining the emerging MRT to explain how primary healthcare learn to integrate intimate partner violence management.

DISCUSSION

Evaluations of health sector responses to IPV are scarce, and even fewer focus on why, how, and when the healthcare services integrate IPV management. There is a consensus that healthcare professionals and healthcare teams play a key role in this integration, and that training is important in order to realize changes. However, little is known about team learning of IPV management, both in terms of how to trigger such learning and how team learning is connected with changes in organizational culture and values, and in service delivery. This realist evaluation protocol aims to contribute to this knowledge by conducting this project in a country, Spain, where great endeavours have been made towards the integration of IPV management within the health system.

Mechanisms for achieving adolescent-friendly services in Ecuador: a realist evaluation approach

BACKGROUND

Despite evidence showing that adolescent-friendly health services (AFSs) increase young people's access to these services, health systems across the world are failing to integrate this approach. In Latin America, policies aimed at strengthening AFS abound. However, such services are offered only in a limited number of sites, and providers' attitudes and respect for confidentiality have not been addressed to a sufficient extent.

METHODS

The aim of this study was to explore the mechanisms that triggered the transformation of an 'ordinary' health care facility into an AFS in Ecuador. For this purpose, a realist evaluation approach was used in order to analyse three well-functioning AFSs. Information was gathered at the national level and from each of the settings including: (i) statistical information and unpublished reports; (ii) in-depth interviews and focus group discussions with policy makers, health care providers, users and adolescents participating in youth organisations and (iii) observations at the health care facilities. Thematic analysis was carried out, driven by the realist evaluation approach, namely exploring the connections between mechanisms, contexts and outcomes.

RESULTS

The results highlighted that the development of the AFSs was mediated by four mechanisms: grounded self-confidence in trying new things, legitimacy, a transformative process and an integral approach to adolescents. Along this process, contextual factors at the national and institutional levels were further explored.

CONCLUSION

The Ministry of Health of Ecuador, based on the New Guidelines for Comprehensive Care of Adolescent Health, has started the scaling up of AFSs. Our research points towards the need to recognise and incorporate these mechanisms as part of the implementation strategy from the very beginning of the process. Although contextually limited to Ecuador, many mechanisms and good practices in these AFS may be relevant to the Latin American setting and elsewhere.