Advancing the application of systems thinking in health: a realist evaluation of a capacity building programme for district managers in Tumkur, India

Five lessons from a mid-level health manager intervention to increase uptake of tuberculosis prevention therapy in Uganda: 'it is a completely different thing to implement what you know.'

BACKGROUND

Leadership skills are essential for middle-level healthcare manager efficacy. Capacity-building efforts may attempt behavioural change by filling 'knowledge gaps' while neglecting a sustainable application of that knowledge. Sustainable application of that knowledge, or implementation know-how, must resonate with local cultural patterns. When it is neglected, root issues like unclear decision-making space and local authority to interpret policy during implementation remain unaddressed. Particularly in decentralized healthcare systems, the impact can appear in implementation challenges, subjective decision-making, poor teamwork, and an absence of disseminating best practices.

OBJECTIVES

The SEARCH-IPT trial led a series of mini-collaborative meetings, which provided business leadership and management training for an intervention group of mid-level healthcare system managers in rural Eastern, East-Central, and Southwestern Uganda to see whether this would increase uptake of isoniazid-prevention therapy (IPT) for people living with HIV (PLHIV) in intervention districts. IPT is known to reduce active tuberculosis (TB), a leading cause of death among PLHIV, by 40-60%.

METHODS

We performed a thematic analysis of six focus-group discussions from this intervention (held in May 2019, January 2020, September 2021) and 23 key informant interviews with control group participants (between February and August 2019 and September and December 2020).

RESULTS

Analysis revealed five implementation skill sets District Health Officers (DHOs) and District Tuberculosis and Leprosy Supervisors (DTLSs) deployed to achieve sustainable implementation and realize their decision-making space. The five practices were as follows: data-based decision-making, root-cause analysis, quality assurance, evidence-based empowerment, and sharing best practices with colleagues.

CONCLUSION

These practices reached beyond outcome measures to address root problems around the DHO's range of authority and elicit buy-in from district health workers. For successful capacity building at the mid-manager level, focusing on core practices as part of competency is objectively implementable and measurable at the system level and does not rely on DHO self-assessments.

A learning health systems approach to scaling up an evidence-based intervention for integrated primary mental healthcare case finding and referral in South Africa

Despite progress in the development and evaluation of evidence-based primary mental health interventions in low-income and middle-income countries, implementation and scale-up efforts have had mixed results. Considerable gaps remain in the effective translation of research knowledge into routine health system practices, largely due to real-world contextual constraints on implementation and scale-up efforts. The Southern African Research Consortium for Mental Health Integration (S-MhINT) programme used implementation research to strengthen the implementation of an evidence-based integrated collaborative depression care model for primary healthcare (PHC) services in South Africa. To facilitate the scale-up of this model from a testing site to the whole province of KwaZulu-Natal, a capacity building programme was embedded within the Alliance for Health Policy and Systems Research (AHPSR) learning health systems (LHS) approach. The paper discusses efforts to scale up and embed case finding and referral elements of the S-MhINT package within routine PHC. Data from semistructured interviews, a focus group discussion, proceedings from participatory workshops and outputs from the application of continuous quality improvement (CQI) cycles were thematically analysed using the AHPSR LHS framework. Learning particularly occurred through information sharing at routine participatory workshops, which also offered mutual deliberation following periods of applying CQI tools to emergent problems. Individual-level, single-loop learning seemed to be particularly observable elements of the AHPSR LHS framework. Ultimately, our experience suggests that successful scale-up requires strong and sustained relationships between researchers, policy-makers and implementers, investments into learning platforms and organisational participation across all levels to ensure ownership and acceptance of learning processes.

Workplace-based learning in district health leadership and management strengthening: a framework synthesis

Effective leadership and management has been identified as critical in enabling health systems to respond adequately to their population needs. The changing nature of low- and middle-income countries' health systems, given resource scarcity, a high disease burden and other contextual challenges, has also led to learning-including workplace-based learning (WPBL)-being recognized as a key process supporting health system reform and transformation. This review used a framework synthesis approach in addressing the question: 'What forms of WPBL, support leadership and management development; and how does such learning impact district health leadership and management strengthening?'. A search for English language empirical qualitative, mixed-methods and quantitative studies and grey literature published from January 1990 to May 2024 was conducted using four electronic databases (PubMed, EBSCOhost, Scopus and Web of Science). Twenty-five articles were included in the synthesis. The findings reveal that over the last decade, WPBL has received consideration as an approach for leadership and management development. While WPBL interventions differed in type and nature, as well as length of delivery, there was no conclusive evidence about which approach had a greater influence than others on strengthening district health leadership and management. However, the synthesis demonstrates the need for a focus on the sustainability and institutionalization of interventions, including the need to integrate WPBL interventions in health systems. To support sustainability and institutionalization, there should be flexibility in the design and delivery of such interventions and they are best supported through national or regional institutions.

The role of provincial health administration in supporting district health management teams in the Democratic Republic of Congo: eliciting an initial programme theory of a realist evaluation

BACKGROUND

In 2006, the Ministry of Health in the Democratic Republic of Congo designed a strategy to strengthen the health system by developing health districts. This strategy included a reform of the provincial health administration to provide effective technical support to district health management teams in terms of leadership and management. The provincial health teams were set up in 2014, but few studies have been done on how, for whom, and under what circumstances their support to the districts works. We report on the development of an initial programme theory that is the first step of a realist evaluation seeking to address this knowledge gap.

METHODS

To inform the initial programme theory, we collected data through a scoping review of primary studies on leadership or management capacity building of district health managers in sub-Saharan Africa, a review of policy documents and interviews with the programme designers. We then conducted a two-step data analysis: first, identification of intervention features, context, actors, mechanisms and outcomes through thematic content analysis, and second, formulation of intervention-context-actor-mechanism-outcome (ICAMO) configurations using a retroductive approach.

RESULTS

We identified six ICAMO configurations explaining how effective technical support (i.e. personalised, problem-solving centred and reflection-stimulating) may improve the competencies of the members of district health management teams by activating a series of mechanisms (including positive perceived relevance of the support, positive perceived credibility of provincial health administration staff, trust in provincial health administration staff, psychological safety, reflexivity, self-efficacy and perceived autonomy) under specific contextual conditions (including enabling learning environment, integration of vertical programmes, competent public health administration staff, optimal decision space, supportive work conditions, availability of resources and absence of negative political influences).

CONCLUSIONS

We identified initial ICAMO configurations that explain how provincial health administration technical support for district health management teams is expected to work, for whom and under what conditions. These ICAMO configurations will be tested in subsequent empirical studies.

How do health system factors (funding and performance) impact on access to healthcare for populations experiencing homelessness: a realist evaluation

BACKGROUND

People experiencing long-term homelessness face significant difficulties accessing appropriate healthcare at the right time and place. This study explores how and why healthcare performance management and funding arrangements contribute to healthcare accessibility or the lack thereof using long-term homeless adults as an example of a population experiencing social exclusion.

METHODS

A realist evaluation was undertaken. Thirteen realist interviews were conducted after which data were transcribed, coded, and analysed.

RESULTS

Fourteen CMOCs were created based on analysis of the data collected. These were then consolidated into four higher-level CMOCs. They show that health systems characterised by fragmentation are designed to meet their own needs above the needs of patients, and they rely on practitioners with a special interest and specialised services to fill the gaps in the system. Key contexts identified in the study include: health system fragmentation; health service fragmentation; bio-medical, one problem at a time model; responsive specialised services; unresponsive mainstream services; national strategy; short health system funding cycles; and short-term goals.

CONCLUSION

When health services are fragmented and complex, the needs of socially excluded populations such as those experiencing homelessness are not met. Health systems focus on their own metrics and rely on separate actors such as independent NGOs to fill gaps when certain people are not accommodated in the mainstream health system. As a result, health systems lack a comprehensive understanding of the needs of all population groups and fail to plan adequately, which maintains fragmentation. Policy makers must set policy and plan health services based on a full understanding of needs of all population groups.

How capacity building of district health managers has been designed, delivered and evaluated in sub-Saharan Africa: a scoping review and best fit framework analysis

OBJECTIVES

We aimed to understand how capacity building programmes (CBPs) of district health managers (DHMs) have been designed, delivered and evaluated in sub-Saharan Africa. We focused on identifying the underlying assumptions behind leadership and management CBPs at the district level.

DESIGN

Scoping review.

DATA SOURCES

We searched five electronic databases (MEDLINE, Health Systems Evidence, Wiley Online Library, Cochrane Library and Google Scholar) on 6 April 2021 and 13 October 2022. We also searched for grey literature and used citation tracking.

ELIGIBILITY CRITERIA

We included all primary studies (1) reporting leadership or management capacity building of DHMs, (2) in sub-Saharan Africa, (3) written in English or French and (4) published between 1 January 1987 and 13 October 2022.

DATA EXTRACTION AND SYNTHESIS

Three independent reviewers extracted data from included articles. We used the best fit framework synthesis approach to identify an a priori framework that guided data coding, analysis and synthesis. We also conducted an inductive analysis of data that could not be coded against the a priori framework.

RESULTS

We identified 2523 papers and ultimately included 44 papers after screening and assessment for eligibility. Key findings included (1) a scarcity of explicit theories underlying CBPs, (2) a diversity of learning approaches with increasing use of the action learning approach, (3) a diversity of content with a focus on management rather than leadership functions and (4) a diversity of evaluation methods with limited use of theory-driven designs to evaluate leadership and management capacity building interventions.

CONCLUSION

This review highlights the need for explicit and well-articulated programme theories for leadership and management development interventions and the need for strengthening their evaluation using theory-driven designs that fit the complexity of health systems.

Capacity building of district health management teams in the era of provincial health administration reform in the Democratic Republic of Congo: a realist evaluation protocol

INTRODUCTION

In 2006, the Congolese Ministry of Health developed a health system strengthening strategy focusing on health district development. This strategy called for reforming the provincial health administration in order to better support the health district development through leadership and management capacity building of district health management teams. The implementation is currently underway, yet, more evidence on how, for whom and under what conditions this capacity building works is needed. The proposed research aims to address this gap using a realist evaluation approach.

METHODS AND ANALYSIS

We will follow the cycle of the realist evaluation. First, we will elicit the initial programme theory through a scoping review (completed in December 2022, using MEDLINE, Health Systems Evidence, Wiley Online Library, Cochrane Library, Google Scholar and grey literature), a review of health policy documents (completed in March 2023), and interviews with key stakeholders (by June 2023). Second, we will empirically test the initial programme theory using a multiple-embedded case study design in two provincial health administrations and four health districts (by March 2024). Data will be collected through document reviews, in-depth interviews, non-participant observations, a questionnaire, routine data from the health information management system and a context mapping tool. We will analyse data using the Intervention-Context-Actor-Mechanism-Outcome configuration heuristic. Last, we will refine the initial programme theory based on the results of the empirical studies and develop recommendations for policymakers (by June 2024).

ETHICS AND DISSEMINATION

The Institutional Review Board of the Institute of Tropical Medicine and the Medical Ethics Committee of the University of Lubumbashi approved this study. We will also seek approvals from provincial-level and district-level health authorities before data collection in their jurisdictions. We will disseminate the study findings through the publication of articles in peer-reviewed academic journals, policy briefs for national policymakers and presentations at national and international conferences.

Pivoting from systems “thinking” to systems “doing” in health systems—Documenting stakeholder perspectives from Southeast Asia

Applications of systems thinking in the context of Health Policy and Systems Research have been scarce, particularly in Low- and Middle-Income Countries (LMICs). Given the urgent need for addressing implementation challenges, the WHO Alliance for Health Policy and Systems Research, in collaboration with partners across five global regions, recently initiated a global community of practice for applied systems thinking in policy and practice contexts within LMICs. Individual one on one calls were conducted with 56 researchers, practitioners & decision-makers across 9 countries in Southeast Asia to elucidate key barriers and opportunities for applying systems thinking in individual country settings. Consultations presented the potential for collaboration and co-production of knowledge across diverse stakeholders to strengthen opportunities by applying systems thinking tools in practice. While regional nuances warrant further exploration, there is a clear indication that policy documentation relevant to health systems will be instrumental in advancing a shared vision and interest in strengthening capacities for applied systems thinking in health systems across Southeast Asia.

Designing for a Healthier Indore, India: Participatory Systems Mapping

In Indore, India, BHC engaged 247 multi-sector stakeholders through a systems mapping approach to gather qualitative data across three workshops and four citizen town halls from 2018 to 2020. These data were synthesized with results from BHC's 18 other city activities to build a systems map and identify high-impact areas for engagement. Contextual findings showed a tension at the heart of Indore's growth-Indore's great success as a city has spurred rapid population growth. This growth creates pressure on municipal systems as population outpaces service delivery capacity. This is central to the systems map that BHC developed and is expanded upon through additional patterns that fall within four main domains: (1) leadership, governance, and financing; (2) essential service delivery and workforce; (3) information systems; and (4) community infrastructure and education. Stakeholders found three key leverage opportunities within this context that, if included in every action, could help overcome barriers. These opportunities are: (1) improving data quality, use, and integration; (2) supporting accountability to, and enforcement of, policies and regulations; and (3) increasing community engagement. Brought together through a better understanding of the key patterns driving system behavior from the context map and leverage opportunities, BHC was able to co-create, with stakeholders, seven "coherent actions" to move Indore to a healthier, more equitable state. When COVID-19 regulations ease, BHC and city officials will reconvene to finalize an implementation plan for these actions.

Study on the Sustainability Potential of Thyme, Oregano, and Coriander Essential Oils Used as Vapours for Antifungal Protection of Wheat and Wheat Products

This study aims to highlight the antifungal, antimicotoxigenic potential and phytotoxic effect of three essential oils (EOs) of Origanum vulgare (OEO), Thymus vulgaris (TEO), and Coriandrum sativum (CEO) on wheat storage, but also the impact of EOs treatment on the sensory properties of bakery products obtained from the wheat seeds. The chemical composition of EOs was determined using GC-MS analysis; the fungal load was evaluated using the direct plating technique, while mycotoxin analyses were conducted using enzyme-linked immunosorbent assay (ELISA). A selective antifungal effect has been highlighted in terms of the action of EOs vapours. OEO and TEO are inhibited Alternaria, Fusarium and Drechslera, while Saccharomyces and Cladosporium have proven to be the most tolerant fungi. Drechslera is the most sensitive, the effect of all EOs being a fungicidal one. However, the fungicidal effect proved present in all EOs applied as vapours with values ranging between 0.2–0.4%. Regarding the phytotoxic effect of EOs vapours on the germination of the seeds, TEO and OEO had an inhibitory effect, especially at 0.4%. The effect is cumulative over time. The EOs inhibited deoxynivalenol (DON) occurrence; the maximum percentage of inhibition was obtained after 21 days of vapours exposure, being more effective in the case of 0.2%. EOs vapours treatment does not affect the quality of bread obtained from treated wheat seeds from a sensory point of view.

Improving access to medicines for non-communicable diseases in rural primary care: results from a quasi-randomized cluster trial in a district in South India

BACKGROUND

A large proportion of non-communicable diseases (NCDs) are treatable within primary health care (PHC) settings in a cost-effective manner. However, the utilization of PHCs for NCD care is comparatively low in India. The Access-to-Medicines (ATM) study examined whether (and how) interventions aimed at health service optimization alone or combined with community platform strengthening improve access to medicines at the primary health care level within the context of a local health system.

METHOD

A quasi-randomized cluster trial was used to assess the effectiveness of the intervention (18 months) implemented across 39 rural PHCs (clusters) of three sub-districts of Tumkur in southern India. The intervention was allocated randomly in a 1:1:1 sequence across PHCs and consisted of three arms: Arm A with a package of interventions aimed at health service delivery optimization; B for strengthening community platforms in addition to A; and the control arm. Group allocation was not blinded to providers and those who assessed outcomes. A household survey was used to understand health-seeking behaviour, access and out-of-pocket expenditure (OOP) on key anti-diabetic and anti-hypertension medicines among patients; facility surveys were used to assess the availability of medicines at PHCs. Primary outcomes of the study are the mean number of days of availability of antidiabetic and antihypertensive medicines at PHCs, the mean number of patients obtaining medicines from PHC and OOP expenses.

RESULT

The difference-in-difference estimate shows a statistically insignificant increase of 31.5 and 11.9 in mean days for diabetes and hypertension medicines availability respectively in the study arm A PHCs beyond the increase in the control arm. We further found that there was a statistically insignificant increase of 2.2 and 3.8 percentage points in the mean proportion of patients obtaining medicines from PHC in arm A and arm B respectively, beyond the increase in the control arm.

CONCLUSION

There were improvements in NCD medicine availability across PHCs, the number of patients accessing PHCs and reduction in OOP expenditure among patients, across the study arms as compared to the control arm; however, these differences were not statistically significant.

TRIAL REGISTRATION

Trial registration number CTRI/2015/03/005640 . This trial was registered on 17/03/2015 in the Clinical Trial Registry of India (CTRI) after PHCs were enrolled in the study (retrospectively registered). The CTRI is the nodal agency of the Indian Council of Medical Research for registration of all clinical, experimental, field intervention and observation studies.

Ensuring the ordinary: Politics and public service in municipal primary care in India

This paper examines the political embeddedness of public-sector primary care in urban India. The low quality of urban healthcare in many low- and middle-income countries is well documented. But there is relatively little analysis showing how the politics of urban healthcare delivery contribute to quality shortfalls. This study integrates urban and political theory and draws on ethnographic fieldwork in municipal government-run primary care clinics in Pune, India. I conceptualize Pune's municipal doctors as street-level bureaucrats: frontline state agents charged with delivering public services, who regularly confront conflicts between their mandate and its realization in practice. I observe how the municipal doctors experience and respond to these conflicts; delineate the historical design of the municipal institutions in which they operate; and interview doctors, nurses, nonclinical staff, administrators, and elected officials, who collectively shape primary care delivery in municipal clinics. My findings show how the doctors' work is characterized by routine departures from public service ideals. The departures stem from local electoral politics (politicians' patronage and clientelistic relations with municipal employees and patients) and weak administrative capacity (misuse and incompetent planning of public resources). The doctors are compelled to follow extra-policy directives, meaning instructions that have little to do with healthcare goals and that emphasize the political utility rather than medical purpose of their work. In response, the doctors circumscribe their clinical practice. They aim, as one doctor put it, only to "ensure the ordinary," or to sustain a deficient status quo. In these conditions, improving quality of care requires not just behavioral interventions targeted at doctors. It requires normative, social, and organizational shifts in public service planning and delivery so that doctors are positioned - materially and affectively - to meet urban healthcare challenges in low-resource contexts.

Bottom-up innovation for health management capacity development: a qualitative case study in a South African health district

BACKGROUND

As part of health system strengthening in South Africa (2012-2017) a new district health manager, taking a bottom-up approach, developed a suite of innovations to improve the processes of monthly district management team meetings, and the practices of managers and NGO partners attending them. Understanding capacity as a property of the health system rather than only of individuals, the research explored the mechanisms triggered in context to produce outputs, including the initial sensemaking by the district manager, the subsequent sensegiving and sensemaking in the team and how these homegrown innovations interacted with existing social processes and norms within the system.

METHODS

We conducted a realist evaluation, adopting the case study design, over a two-year period (2013-2015) in the district of focus. The initial programme theory was developed from 10 senior manager interviews and a literature review. To understand the processes and mechanisms triggered in the local context and identify outputs, we conducted 15 interviews with managers in the management team and seven with non-state actors. These were supplemented by researcher notes based on time spent in the district. Thematic analysis was conducted using the Context-Mechanism-Outcome configuration alongside theoretical constructs.

RESULTS

The new district manager drew on systems thinking, tacit and experiential knowledge to design bottom-up innovations. Capacity was triggered through micro-practices of sensemaking and sensegiving which included using sticks (positional authority, enforcement of policies, over-coding), intentionally providing justifications for change and setting the scene (a new agenda, distributed leadership). These micro-practices in themselves, and by managers engaging with them, triggered a generative process of buy-in and motivation which influenced managers and partners to participate in new practices within a routine meeting.

CONCLUSION

District managers are well placed to design local capacity development innovations and must draw on systems thinking, tacit and experiential knowledge to enable relevant 'bottom-up' capacity development in district health systems. By drawing on soft skills and the policy resources (hardware) of the system they can influence motivation and buy-in to improve management practices. From a systems perspective, we argue that capacity development can be conceived of as part of the daily activity of managing within routine spaces.

How Does District Health Management Emerge Within a Complex Health System? Insights for Capacity Strengthening in Ghana

Introduction: District health managers (DHMs) play a pivotal role in the operation of district health systems in low-and middle income countries, including Ghana. Their capacity is determined by their competencies, but also by the organization and system in which they are embedded. The objective of this paper is to explore how district health management emerges from contextual, organizational, and individual factors in order to demonstrate that capacity strengthening efforts at district level need to transcend individual competencies to take on more systemic approaches. Methods: Semi-structured interviews (n = 21) were conducted to gain insight into aspects that affect district health management in the Eastern Region of Ghana. Interviews were conducted with DHMs (n = 15) from six different districts, as well as with their superiors at the regional level (n = 4) and peers from non-governmental organizations (n = 2). A thematic analysis was conducted by using an analytical approach based on systems theory. Results: Contextual aspects, such as priorities among elected officials, poor infrastructure and working conditions, centralized decision-making, delayed disbursement of funds and staff shortages, affect organizational processes and the way DHMs carry out their role. Enabling organizational aspects that provide DHMs with direction and a clear perception of their role, include positive team dynamics, good relations with supervisors, job descriptions, appraisals, information systems, policies and guidelines. Meanwhile, hierarchical organizational structures, agendas driven by vertical programs and limited opportunities for professional development provide DHMs with limited authority to make decisions and dampens their motivation. The DHMs ability to carry out their role effectively depends on their perception of their role and the effort they expend, in addition to their competencies. In regards to competencies, a need for more general management and leadership skills were called for by DHMs as well as by their superiors and peers. Conclusion: Systemic approaches are called for in order to strengthen district health management capacity. This study can provide national policy-makers, donors and researchers with a deeper understanding of factors that should be taken into consideration when developing, planning, implementing, and assessing capacity-building strategies targeted at strengthening district health management.

Using an implementation science approach to implement and evaluate patient-reported outcome measures (PROM) initiatives in routine care settings

Purpose

Patient-reported outcome and experience measures (PROMs/PREMs) are well established in research for many health conditions, but barriers persist for implementing them in routine care. Implementation science (IS) offers a potential way forward, but its application has been limited for PROMs/PREMs.

Methods

We compare similarities and differences for widely used IS frameworks and their applicability for implementing PROMs/PREMs through case studies. Three case studies implemented PROMs: (1) pain clinics in Canada; (2) oncology clinics in Australia; and (3) pediatric/adult clinics for chronic conditions in the Netherlands. The fourth case study is planning PREMs implementation in Canadian primary care clinics. We compare case studies on barriers, enablers, implementation strategies, and evaluation.

Results

Case studies used IS frameworks to systematize barriers, to develop implementation strategies for clinics, and to evaluate implementation effectiveness. Across case studies, consistent PROM/PREM implementation barriers were technology, uncertainty about how or why to use PROMs/PREMs, and competing demands from established clinical workflows. Enabling factors in clinics were context specific. Implementation support strategies changed during pre-implementation, implementation, and post-implementation stages. Evaluation approaches were inconsistent across case studies, and thus, we present example evaluation metrics specific to PROMs/PREMs.

Conclusion

Multilevel IS frameworks are necessary for PROM/PREM implementation given the complexity. In cross-study comparisons, barriers to PROM/PREM implementation were consistent across patient populations and care settings, but enablers were context specific, suggesting the need for tailored implementation strategies based on clinic resources. Theoretically guided studies are needed to clarify how, why, and in what circumstances IS principles lead to successful PROM/PREM integration and sustainability.

Electronic supplementary material

The online version of this article (10.1007/s11136-020-02564-9) contains supplementary material, which is available to users.

The impact of a community-based music intervention on the health and well-being of young people: A realist evaluation

In recent years, music-based interventions have been utilised as a tool for improving public health, reducing inequalities and promoting well-being of young people. Although some researchers have begun to draw links between music-related interventions and positive health outcomes, there is little understanding as to how such effects are produced. Realist evaluations-understanding what works, for whom and under what circumstances-are a particularly apt means by which we can open this 'black box'. In this paper, we use a realist evaluation to assess a community-based music initiative designed and implemented to support the well-being of disadvantaged young people in Scotland. In order to gain perspectives on the range of contextual characteristics, mechanisms and outcomes, we collected quantitative and qualitative data in the form of pre- and post-questionnaires, as well as conducting interviews with beneficiaries and stakeholders. Our findings show that the intervention achieved a positive impact on the self-confidence, well-being and engagement of disadvantaged young people. This impact was achieved via an approach personally tailored to the individual needs of the young people; and an organisational environment characterised by trust, whereby young people felt safe to express themselves.

Realist evaluations in low- and middle-income countries: reflections and recommendations from the experiences of a foreign researcher

Realist evaluation, a methodology for exploring generative causation within complex health interventions to understand 'how, why and for whom' programmes work, is experiencing a surge of interest. Trends indicate that the proliferation in the use of this methodology also applies to research in low- and middle-income countries (LMICs). The value of using realist evaluation for project evaluation is also being noticed by non-governmental organisations (NGOs) and other programme implementers within such contexts. Yet, there is limited exploration of the use of realist evaluations in LMICs, especially their use by foreign researchers. This paper draws on the author's experience of conducting two realist evaluations across three different sub-Saharan African settings: Mundemu, Tanzania; Kabale, Uganda and Marsabit, Kenya. The realist evaluations were used as an operations research methodology to study two NGO community health programmes. This paper highlights four main challenges experienced by the author throughout the methodological process: (1) power imbalances prevalent during realist interviews, (2) working through translation and what this means for identfying Context-Mechanism-Outcome Configurations, (3) limited contextual familiarity and being an 'engaged researcher' and (4) the use or dependence on 'WEIRD' theories (i.e. theories based on the study of Western, Educated, Industrialized, Rich, Democratic people) within testing and refinement. Realist evaluation's enticing and straightforward slogan of finding 'what works, for whom and why' is in contrast to the complexity of the methodology used to generate these results (and often to the results themselves). Striking a balance between theory and pragmatism, while adhering to realist ontological underpinnings of generative causation and retroduction, is no easy task. This paper concludes by providing concrete recommendations for those who want to undertake a realist evaluation, with particular attention to cross-cultural settings, in light of the aforementioned challenges. In doing so, it aims to foster improved methodological rigour and help those engaging in this research methodology to work towards more appropriate and contextually relevant findings.

Exploring the Human Factors Affecting Health Service Managers: A Qualitative Study

BACKGROUND

In recent decades, managing health-service systems has faced multiple challenges. Identifying and resolving these challenges promote the efficiency and effectiveness of hospital activities.

AIM

The present study aimed to explore the human factors affecting health service managers.

MATERIAL AND METHODS

In this qualitative study, in-depth unstructured interviews were conducted with 29 employees who were in close contact with the health service managers. All the interviews were transcribed verbatim. Data were collected using purposeful sampling and were analysed using conventional content analysis via MAXQDA software.

RESULTS

A group of 29 participants were interviewed (male 65.5%, female 34.4%). "Managing managers" has been identified as the primary theme with four supporting secondary themes including the inappropriate appointment of managers, the impact of human and social needs of managers, influential employees, and disrupting organisational communications. These are the challenges faced by managers in managing human resources in health-service organisations.

CONCLUSION

Results showed that employees manage their managers within the organisation so that they can prevent managers from concentrating on their management affairs and tasks resulting in the distortion of management practices. The results of this study can help the key policy makers and planners in health-service organisations to guide the organisation to pursue its goals through appointing appropriate managers and identifying influential employees.

Global Health Research, Anthropology and Realist Enquiry

In this article, I set out to capture the dynamics of two streams within the field of global health research: realist research and medical anthropology. I critically discuss the development of methodology and practice in realist health research in low- and middle-income countries against the background of anthropological practice in global health to make claims on why realist enquiry has taken a high flight. I argue that in order to provide a contribution to today’s complex global issues, we need to adopt a pragmatic stance and move past disciplinary silos: both methodologies have the potential to be well-suited to an analysis of deep layers of context and of key social mechanisms.

Complex Leadership in Healthcare: A Scoping Review

BACKGROUND

Nowadays, health systems are generally acknowledged to be complex social systems. Consequently, scholars, academics, practitioners, and policy-makers are exploring how to adopt a complexity perspective in health policy and system research. While leadership and complexity has been studied extensively outside health, the implications of complexity theories for the study of leadership in healthcare have received limited attention. We carried out a scoping review of complex leadership (CL) in healthcare to investigate how CL in healthcare has been defined, theorised and conceptualised and to explore how 'CL' has been applied in healthcare settings.

METHODS

We followed the methodological steps proposed by (Arksey and O'Malley, 2005): (1) specifying the research question, (2) identifying relevant studies, (3) study selection, (4) charting the data, (5) collating and summarizing the findings, and (6) reporting the results. We searched using Medline, Psychinfo, Wiley online library, and Google Scholar. Our inclusion criteria were: publication type (peer reviewed articles, theses, and book chapters); phenomenon of interest: complex leadership; context: healthcare and period of publication: between 2000 and 2016.

RESULTS

Our search and selection resulted in 37 papers (16 conceptual papers, 14 empirical studies and 7 advocacy papers). We note that empirical studies on CL are few and almost all research reported by these papers was carried out in the North (mainly in USA and UK). We found that there is some variation in definitions of CL. Furthermore, the research papers adopt mostly an explorative or explanatory approach and do not focus on assessing effectiveness of CL approaches. Finally, we found that the majority of researchers seem to adhere to the mathematical complexity perspective.

CONCLUSION

Complexity concepts derived from natural sciences may not automatically fit management of health services. Further research into how social complexity theories may offer researchers useful grounds to empirically test CL theories in health settings is warranted. Specific attention should be paid to the multi-layered nature of leadership.

Characterizing performance improvement in primary care systems in Mesoamerica: A realist evaluation protocol

Background. Evaluations of performance measurement and management interventions in public, primary care delivery systems of low- and middle-income countries are scarce. In such contexts, few studies to date have focused on characterizing how, why and under what contextual conditions do such complex, multifaceted arrangements lead to intended and unintended consequences for the healthcare workforce, the healthcare organizations involved, and the communities that are served. Methods. Case-study design with purposeful outlier sampling of high-performing primary care delivery systems in El Salvador and Honduras, as part of the Salud Mesoamerica Initiative. Case study design is suitable for characterizing individual, interpersonal and collective mechanisms of change in complex adaptive systems. The protocol design includes literature review, document review, non-participant observation, and qualitative analysis of in-depth interviews. Data analysis will use inductive and deductive approaches to identify causal patterns organized as ‘context-mechanism-outcome’ configurations. Findings will be triangulated with existing secondary data sources collected including country-specific performance measurement data, impact, and process evaluations conducted by the Salud Mesoamerica Initiative. Discussion. This realist evaluation protocol aims to characterize how, why and under what conditions do performance measurement and management arrangements contribute to the improvement of primary care system performance in two low-income countries.

Advancing the science behind human resources for health: highlights from the Health Policy and Systems Research Reader on Human Resources for Health

Health workers are central to people-centred health systems, resilient economies and sustainable development. Given the rising importance of the health workforce, changing human resource for health (HRH) policy and practice and recent health policy and systems research (HPSR) advances, it is critical to reassess and reinvigorate the science behind HRH as part of health systems strengthening and social development more broadly. Building on the recently published Health Policy and Systems Research Reader on Human Resources for Health (the Reader), this commentary reflects on the added value of HPSR underpinning HRH. HPSR does so by strengthening the multi-disciplinary base and rigour of HRH research by (1) valuing diverse research inferences and (2) deepening research enquiry and quality. It also anchors the relevance of HRH research for HRH policy and practice by (3) broadening conceptual boundaries and (4) strengthening policy engagement. Most importantly, HPSR enables us to transform HRH from being faceless numbers or units of health producers to the heart and soul of health systems and vital change agents in our communities and societies. Health workers' identities and motivation, daily routines and negotiations, and training and working environments are at the centre of successes and failures of health interventions, health system functioning and broader social development. Further, in an increasingly complex globalised economy, the expansion of the health sector as an arena for employment and the liberalisation of labour markets has contributed to the unprecedented movement of health workers, many or most of whom are women, not only between public and private health sectors, but also across borders. Yet, these political, human development and labour market realities are often set aside or elided altogether. Health workers' lives and livelihoods, their contributions and commitments, and their individual and collective agency are ignored. The science of HRH, offering new discoveries and deeper understanding of how universal health coverage and the Sustainable Development Goals are dependent on millions of health workers globally, has the potential to overcome this outdated and ineffective orthodoxy.

Advancing the science behind human resources for health: highlights from the Health Policy and Systems Research Reader on Human Resources for Health

Health workers are central to people-centred health systems, resilient economies and sustainable development. Given the rising importance of the health workforce, changing human resource for health (HRH) policy and practice and recent health policy and systems research (HPSR) advances, it is critical to reassess and reinvigorate the science behind HRH as part of health systems strengthening and social development more broadly. Building on the recently published Health Policy and Systems Research Reader on Human Resources for Health (the Reader), this commentary reflects on the added value of HPSR underpinning HRH. HPSR does so by strengthening the multi-disciplinary base and rigour of HRH research by (1) valuing diverse research inferences and (2) deepening research enquiry and quality. It also anchors the relevance of HRH research for HRH policy and practice by (3) broadening conceptual boundaries and (4) strengthening policy engagement. Most importantly, HPSR enables us to transform HRH from being faceless numbers or units of health producers to the heart and soul of health systems and vital change agents in our communities and societies. Health workers' identities and motivation, daily routines and negotiations, and training and working environments are at the centre of successes and failures of health interventions, health system functioning and broader social development. Further, in an increasingly complex globalised economy, the expansion of the health sector as an arena for employment and the liberalisation of labour markets has contributed to the unprecedented movement of health workers, many or most of whom are women, not only between public and private health sectors, but also across borders. Yet, these political, human development and labour market realities are often set aside or elided altogether. Health workers' lives and livelihoods, their contributions and commitments, and their individual and collective agency are ignored. The science of HRH, offering new discoveries and deeper understanding of how universal health coverage and the Sustainable Development Goals are dependent on millions of health workers globally, has the potential to overcome this outdated and ineffective orthodoxy.

Strengthening health system leadership for better governance: what does it take?

This editorial provides an overview of the six papers included in this special supplement on health leadership in Africa. Together the papers provide evidence of leadership in public hospital settings and of initiatives to strengthen leadership development. On the one hand, they demonstrate both that current leadership practices often impact negatively on staff motivation and patient care, and that contextual factors underpin poor leadership. On the other hand, they provide some evidence of the positive potential of new forms of participatory leadership, together with ideas about what forms of leadership development intervention can nurture new forms of leadership. Finally, the papers prompt reflection on the research needed to support the implementation of such interventions.

From bouncing back, to nurturing emergence: reframing the concept of resilience in health systems strengthening

Recent health system shocks such as the Ebola disease outbreak have focused global health attention on the notion of resilient health systems. In this commentary, we reflect on the current framing of the concept of resilience in health systems discourse and propose a reframing. Specifically, we propose that: (1) in addition to sudden shocks, health systems face the ongoing strain of multiple factors. Health systems need the capacity to continue to deliver services of good quality and respond effectively to wider health challenges. We call this capacity everyday resilience; (2) health system resilience entails more than bouncing back from shock. In complex adaptive systems (CAS), resilience emerges from a combination of absorptive, adaptive and transformative strategies; (3) nurturing the resilience of health systems requires understanding health systems as comprising not only hardware elements (such as finances and infrastructure), but also software elements (such as leadership capacity, power relations, values and appropriate organizational culture). We also reflect on current criticisms of the concept of resilient health systems, such as that it assumes that systems are apolitical, ignoring actor agency, promoting inaction, and requiring that we accept and embrace vulnerability, rather than strive for stronger and more responsive systems. We observe that these criticisms are warranted to the extent that they refer to notions of resilience that are mismatched with the reality of health systems as CAS. We argue that the observed weaknesses of resilience thinking can be addressed by reframing and applying a resilience lens that is better suited to the attributes of health systems as CAS.

Elements for harnessing participatory action research to strengthen health managers' capacity: a critical interpretative synthesis

BACKGROUND

Health managers play a key role in ensuring that health services are responsive to the needs of the population. Participatory action research (PAR) is one of the approaches that have been used to strengthen managers' capacity. However, collated knowledge on elements for harnessing PAR to strengthen managers' capacity is missing. This paper bridges this gap by reviewing existing literature on the subject matter.

METHODS

A critical interpretive synthesis method was used to interrogate eight selected articles. These articles reported the use of PAR to strengthen health managers' capacity. The critical interpretive synthesis method's approach to analysis guided the synthesis. Here, the authors interpretively made connections and linkages between different elements identified in the literature. Finally, the Atun et al. (Heal Pol Plann, 25:104-111, 2010) framework on integration was used to model the elements synthesised in the literature into five main domains.

RESULTS

Five elements with intricate bi-directional interactions were identified in the literature reviewed. These included a shared purpose, skilled facilitation and psychological safety, activity integration into organisational procedures, organisational support, and external supportive monitoring. A shared purpose of the managers' capacity strengthening initiative created commitment and motivation to learn. This purpose was built upon a set of facilitation skills that included promoting participation, self-efficacy and reflection, thereby creating a safe psychological space within which the managers interacted and learnt from each other and their actions. Additionally, an integrated intervention strengthened local capacity and harnessed organisational support for learning. Finally, supportive monitoring from external partners, such as researchers, ensured quality, building of local capacity and professional safety networks essential for continued learning.

CONCLUSIONS

The five elements identified in this synthesis provide a basis upon which the use of PAR can be harnessed, not only to strengthen health managers' capacity, but also to foster other health systems strengthening initiatives involving implementation research. In addition, the findings demonstrated the intricate and complex relations between the elements, which further affirms the need for a systems thinking approach to tackling health systems challenges.

What can we learn from interventions that aim to increase policy-makers' capacity to use research? A realist scoping review

Background

Health policy-making can benefit from more effective use of research. In many policy settings there is scope to increase capacity for using research individually and organisationally, but little is known about what strategies work best in which circumstances. This review addresses the question: What causal mechanisms can best explain the observed outcomes of interventions that aim to increase policy-makers’ capacity to use research in their work?

Methods

Articles were identified from three available reviews and two databases (PAIS and WoS; 1999–2016). Using a realist approach, articles were reviewed for information about contexts, outcomes (including process effects) and possible causal mechanisms. Strategy + Context + Mechanism = Outcomes (SCMO) configurations were developed, drawing on theory and findings from other studies to develop tentative hypotheses that might be applicable across a range of intervention sites.

Results

We found 22 studies that spanned 18 countries. There were two dominant design strategies (needs-based tailoring and multi-component design) and 18 intervention strategies targeting four domains of capacity, namely access to research, skills improvement, systems improvement and interaction. Many potential mechanisms were identified as well as some enduring contextual characteristics that all interventions should consider. The evidence was variable, but the SCMO analysis suggested that tailored interactive workshops supported by goal-focused mentoring, and genuine collaboration, seem particularly promising. Systems supports and platforms for cross-sector collaboration are likely to play crucial roles. Gaps in the literature are discussed.

Conclusion

This exploratory review tentatively posits causal mechanisms that might explain how intervention strategies work in different contexts to build capacity for using research in policy-making.

Electronic supplementary material

The online version of this article (10.1186/s12961-018-0277-1) contains supplementary material, which is available to authorized users.

Evaluation of the district health management fellowship training programme: a case study in Iran

OBJECTIVE

To evaluate the district health management fellowship training programme in the north-west of Iran.

DATA SOURCES/STUDY SETTING

The programme was introduced to build the managerial capacity of district health managers in Iran. Eighty-nine heads of units in the province's health centre, district health managers and the health deputies of the district health centres in the north-west provinces of Iran had registered for the district health management fellowship training programme in Tabriz in 2015-2016.

STUDY DESIGN

This was an educational evaluation study to evaluate training courses to measure participants' reactions and learning and, to a lesser extent, application of training to their job and the organisational impact.

DATA COLLECTION/EXTRACTION METHODS

Valid and reliable questionnaires were used to assess learning techniques and views towards the fellowship, and self-assessment of health managers' knowledge and skills. Also, pretest and post-test examinations were conducted in each course and a portfolio was provided to the trainees to be completed in their work settings.

PRINCIPAL FINDINGS

About 63% of the participants were medical doctors and 42.3% of them had over 20 years of experience. Learning by practice (scored 18.37 out of 20) and access to publications (17.27) were the most useful methods of training in health planning and management from the participants' perspective. Moreover, meeting peers from other districts and the academic credibility of teachers were the most important features of the current programme. Based on the managers' self-assessment, they were most skilful in quality improvement, managing, planning and evaluation of the district. The results of the post-test analysis on data collected from district health managers showed the highest scores in managing the district (77 out of 100) and planning and evaluation (69) of the courses.

CONCLUSION

The results of this study indicated that training courses, methods and improvement in managers' knowledge about the health system and the skills necessary to manage their organisation were acceptable.

Preparedness for delivering non-communicable disease services in primary care: access to medicines for diabetes and hypertension in a district in south India

Introduction

Non-communicable diseases (NCDs) have become a major public health challenge worldwide; they account for 28 million deaths per year in low-and-middle-income countries (LMICs). Like many other LMICs, India is struggling to organise quality care for a large NCD-affected population especially at the primary healthcare level. The aim of this study was to assess local health system preparedness in a south Indian primary healthcare setting for addressing diabetes and hypertension.

Methods

This paper draws on a mixed-methods research study on access to medicines conducted in Tumkur, Karnataka, India. We used quantitative data from household and health facility surveys, and qualitative data from focus group discussions and in-depth interviews with health workers and patients. We identified systemic drivers that influence utilisation of services at government primary health centres (PHCs) using thematic analysis of qualitative data and a systems framework on access to medicines to assess supply and demand side factors.

Results

Majority of households depend on private facilities for diabetes and hypertension care because of the lack of laboratory facilities and frequent medicine stockouts at PHCs. Financial and managerial resource allocation for NCDs and prioritisation of care and processes related to NCDs was suboptimal compared to the prominence of this agenda at global and national levels. Primary healthcare has a limited role even in the activities under the national programme that addresses diabetes and hypertension.

Discussion

The study finds critical gaps in the preparedness of PHCs and district health systems in organising and managing care for diabetes and hypertension. Due to the lack of continuous care organised through PHCs, patients depend on expensive and often episodic care in the private sector. There is a need to improve managerial and financial resource allocation towards diabetes and hypertension (and other NCDs) at the district level.

Trial registration number

CTRI/2015/03/005640; Pre-results.

A participatory action research approach to strengthening health managers' capacity at district level in Eastern Uganda

Background

Many approaches to improving health managers’ capacity in poor countries, particularly those pursued by external agencies, employ non-participatory approaches and often seek to circumvent (rather than strengthen) weak public management structures. This limits opportunities for strengthening local health managers’ capacity, improving resource utilisation and enhancing service delivery. This study explored the contribution of a participatory action research approach to strengthening health managers’ capacity in Eastern Uganda.

Methods

This was a qualitative study that used open-ended key informant interviews, combined with review of meeting minutes and observations to collect data. Both inductive and deductive thematic analysis was undertaken. The Competing Values Framework of organisational management functions guided the deductive process of analysis and the interpretation of the findings. The framework builds on four earlier models of management and regards them as complementary rather than conflicting, and identifies four managers’ capacities (collaborate, create, compete and control) by categorising them along two axes, one contrasting flexibility versus control and the other internal versus external organisational focus.

Results

The findings indicate that the participatory action research approach enhanced health managers’ capacity to collaborate with others, be creative, attain goals and review progress. The enablers included expanded interaction spaces, encouragement of flexibility, empowerment of local managers, and the promotion of reflection and accountability. Tension and conflict across different management functions was apparent; for example, while there was a need to collaborate, maintaining control over processes was also needed. These tensions meant that managers needed to learn to simultaneously draw upon and use different capacities as reflected by the Competing Values Framework in order to maximise their effectiveness.

Conclusions

Improved health manager capacity is essential if sustained improvements in health outcomes in low-income countries are to be attained. The expansion of interaction spaces, encouragement of flexibility, empowerment of local managers, and the promotion of reflection and accountability were the key means by which participatory action research strengthened health managers’ capacity. The participatory approach to implementation therefore created opportunities to strengthen health managers’ capacity.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-017-0273-x) contains supplementary material, which is available to authorized users.

Quality and reporting standards, resources, training materials and information for realist evaluation: the RAMESES II project

Background

Many of the problems confronting policy- and decision-makers, evaluators and researchers today are complex, as are the interventions designed to tackle them. Their success depends both on individuals’ responses and on the wider context of people’s lives. Realist evaluation tries to make sense of these complex interventions. It is a form of theory-driven evaluation, based on realist philosophy, that aims to understand why these complex interventions work, how, for whom, in what context and to what extent.

Objectives

Our objectives were to develop (a) quality standards, (b) reporting standards, (c) resources and training materials, (d) information and resources for patients and other lay participants and (e) to build research capacity among those interested in realist evaluation.

Methods

To develop the quality and reporting standards, we undertook a thematic review of the literature, supplemented by our content expertise and feedback from presentations and workshops. We synthesised findings into briefing materials for realist evaluations for the Delphi panel (a structured method using experts to develop consensus). To develop our resources and training materials, we drew on our experience in developing and delivering education materials, feedback from the Delphi panel, the RAMESES JISCMail e-mail list, training workshops and feedback from training sessions. To develop information and resources for patients and other lay participants in realist evaluation, we convened a group consisting of patients and the public. We built research capacity by running workshops and training sessions.

Results

Our literature review identified 152 realist evaluations, and when 37 of these had been analysed we were able to develop our briefing materials for the Delphi panel. The Delphi panel comprised 35 members from 27 organisations across six countries and five disciplines. Within three rounds, the panels had reached a consensus on 20 key reporting standards. The quality standards consist of eight criteria for realist evaluations. We developed resources and training materials for 15 theoretical and methodological topics. All resources are available online (www.ramesesproject.org). We provided methodological support to 17 projects and presentations or workshops to help build research capacity in realist evaluations to 29 organisations. Finally, we produced a generic patient information leaflet for lay participants in realist evaluations.

Limitations

Our project had ambitious goals that created a substantial workload, leading to the need to prioritise objectives. For example, we truncated the literature review and focused on standards and training material development.

Conclusions

Although realist evaluation holds much promise, misunderstandings and misapplications of it are common. We hope that our project’s outputs and activities will help to address these problems. Our resources are the start of an iterative journey of refinement and development of better resources for realist evaluations. The RAMESES II project seeks not to produce the last word on these issues, but to capture current expertise and establish an agreed state of the science. Much methodological development is needed in realist evaluation but this can take place only if there is a sufficient pool of highly skilled realist evaluators. Capacity building is the next key step in realist evaluation.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

The challenge of complexity in evaluating health policies and programs: the case of women's participatory groups to improve antenatal outcomes

Background

During the last years, randomized designs have been promoted as the cornerstone of evidence-based policymaking. Also in the field of community participation, Random Control Trials (RCTs) have been the dominant design, used for instance to examine the contribution of community participation to health improvement. We aim at clarifying why RCTs and related (quasi-) experimental designs may not be the most appropriate approach to evaluate such complex programmes.

Results

We argue that the current methodological debate could be more fruitful if it would start from the position that the choice of designs should fit the nature of the program and research questions rather than be driven by methodological preferences. We present how realist evaluation, a theory-driven approach to research and evaluation, is a relevant methodology that could be used to assess whether and how community participation works. Using the realist evaluation approach to examine the relationship between participation and action of women groups and antenatal outcomes would enable evaluators to examine in detail the underlying mechanisms which influence actual practices and outcomes, as well as the context conditions required to make it work.

Conclusions

Realist research in fact allows opening the black boxes of “community” and “participation” in order to examine the role they play in ensuring cost-effective, sustainable interventions. This approach yields important information for policy makers and programme managers considering how such programs could be implemented in their own setting.

Hospitals as complex adaptive systems: A case study of factors influencing priority setting practices at the hospital level in Kenya

There is a dearth of literature on priority setting and resource allocation (PSRA) practices in hospitals, particularly in low and middle income countries (LMICs). Using a case study approach, we examined PSRA practices in 2 public hospitals in coastal Kenya. We collected data through a combination of in-depth interviews of national level policy makers, hospital managers, and frontline practitioners in the case study hospitals (n = 72), review of documents such as hospital plans and budgets, minutes of meetings and accounting records, and non-participant observations of PSRA practices in case study hospitals over a period of 7 months. In this paper, we apply complex adaptive system (CAS) theory to examine the factors that influence PSRA practices. We found that PSRA practices in the case hospitals were influenced by, 1) inadequate financing level and poorly designed financing arrangements, 2) limited hospital autonomy and decision space, and 3) inadequate management and leadership capacity in the hospital. The case study hospitals exhibited properties of complex adaptive systems (CASs) that exist in a dynamic state with multiple interacting agents. Weaknesses in system ‘hardware’ (resource scarcity) and ‘software’ (including PSRA guidelines that reduced hospitals decision space, and poor leadership skills) led to the emergence of undesired properties. The capacity of hospitals to set priorities should be improved across these interacting aspects of the hospital organizational system. Interventions should however recognize that hospitals are CAS. Rather than rectifying isolated aspects of the system, they should endeavor to create conditions for productive emergence.

•

Priority setting practices reveal the complex adaptive systems (CAS) nature of hospitals.

•

Hospital priority setting is influenced by deficiencies in both system hardware and software.

•

Intervening in CAS requires fostering productive emergence, not simple, mechanistic solutions.

Building a competent health manager at district level: a grounded theory study from Eastern Uganda

BACKGROUND

Health systems in low-income countries are often characterized by poor health outcomes. While many reasons have been advanced to explain the persistently poor outcomes, management of the system has been found to play a key role. According to a WHO framework, the management of health systems is central to its ability to deliver needed health services. In this study, we examined how district managers in a rural setting in Uganda perceived existing approaches to strengthening management so as to provide a pragmatic and synergistic model for improving management capacity building.

METHODS

Twenty-two interviews were conducted with district level administrative and political managers, district level health managers and health facility managers to understand their perceptions and definitions of management and capacity building. Kathy Charmaz's constructive approach to grounded theory informed the data analysis process.

RESULTS

An interative, dynamic and complex model with three sub-process of building a competent health manager was developed. A competent manager was understood as one who knew his/her roles, was well informed and was empowered to execute management functions. Professionalizing health managers which was viewed as the foundation, the use of engaging learning approaches as the inside contents and having a supportive work environment the frame of the model were the sub-processes involved in the model. The sub-processes were interconnected although the respondents agreed that having a supportive work environment was more time and effort intensive relative to the other two sub-processes.

CONCLUSIONS

The model developed in our study makes four central contributions to enhance the WHO framework and the existing literature. First, it emphasizes management capacity building as an iterative, dynamic and complex process rather than a set of characteristics of competent managers. Second, our model suggests the need for professionalization of health managers at different levels of the health system. Third, our model underscores the benefits that could be accrued from the use of engaging learning approaches through prolonged and sustained processes that act in synergy. Lastly, our model postulates that different resource investments and a varied range of stakeholders could be required at each of the sub-processes.

How do community health committees contribute to capacity building for maternal and child health? A realist evaluation protocol

INTRODUCTION

The proposed research is part of ongoing operations research within World Vision's Access: Infant and Maternal Health Programme. This study aims to identify key context features and underlying mechanisms through which community health committees build community capacity within the field of maternal and child health. This may help to improve programme implementation by providing contextually informed and explanatory findings for how community health committees work, what works best and for whom do they work for best for. Though frequently used within health programmes, little research is carried out on such committees' contribution to capacity building-a frequent goal or proposed outcome of these groups.

METHODS AND ANALYSIS

The scarce information that does exist often fails to explain 'how, why, and for whom' these committees work best. Since such groups typically operate within or as components of complex health interventions, they require a systems thinking approach and design, and thus so too does their evaluation. Using a mixed methods realist evaluation with intraprogramme case studies, this protocol details a proposed study on community health committees in rural Tanzania and Uganda to better understand underlying mechanisms through which these groups work (or do not) to build community capacity for maternal and child health. This research protocol follows the realist evaluation methodology of eliciting initial programme theories, to inform the field study design, which are detailed within. Thus far, the methodology of a realist evaluation has been well suited to the study of community health committees within these contexts. Implications for its use within these contexts are discussed within.

ETHICS AND DISSEMINATION

Institutional Review Boards and the appropriate research clearance bodies within Ireland, Uganda and Tanzania have approved this study. Planned dissemination activities include via academic and programme channels, as well as feedback to the communities in which this work occurs.

Improving access to medicines for non-communicable diseases in rural India: a mixed methods study protocol using quasi-experimental design

Background

India has the distinction of financing its healthcare mainly through out-of-pocket expenses by individual families contributing to catastrophic health expenditure and impoverishment. Nearly 70 % of the expenditure is on medicines purchased at private pharmacies. Patients with chronic ailments are especially affected, as they often need lifelong medicines. Over the past years in India, there have been several efforts to improve drug availability at government primary health centres. In this study, we aim to understand health system factors that affect utilisation and access to generic medicines for people with non-communicable diseases.

Methods

This study aims to understand if (and how) a package of interventions targeting primary health centres and community participation platforms affect utilisation and access to generic medicines for people with non-communicable diseases in the current district context in India. This study will employ a quasi-experimental design and a qualitative theory-driven approach. PHCs will be randomly assigned to one of three arms of the intervention. In one arm, PHCs will receive inputs to optimise service delivery for non-communicable diseases, while the second arm will receive an additional package of interventions to strengthen community participation platforms for improving non-communicable disease care. The third arm will be the control. We will conduct household and facility surveys, before and after the intervention and will estimate the effect of the intervention by difference-in-difference analysis. Sample size for measuring effects was calculated based on obtaining at least 30 households for each primary health centre spread across three distance-based clusters. Primary outcomes include availability and utilisation of medicines at primary health centres and out-of-pocket expenditure for medicines by non-communicable disease households. Focus group discussions with patients and in-depth interviews with health workers will also be conducted. Qualitative and process documentation data will be used to explain how the intervention could have worked.

Discussion

By taking into consideration several health system building blocks and trying to understand how they interact, our study aims to generate evidence for health planners on how to optimise health services to improve access to medicines.

Trial registration

Protocol registered on Clinical Trials Registry of India with registration identifier number CTRI/2015/03/005640 on 17^th March 2015.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-016-1680-3) contains supplementary material, which is available to authorized users.

RAMESES II reporting standards for realist evaluations

BACKGROUND

Realist evaluation is increasingly used in health services and other fields of research and evaluation. No previous standards exist for reporting realist evaluations. This standard was developed as part of the RAMESES II project. The project's aim is to produce initial reporting standards for realist evaluations.

METHODS

We purposively recruited a maximum variety sample of an international group of experts in realist evaluation to our online Delphi panel. Panel members came from a variety of disciplines, sectors and policy fields. We prepared the briefing materials for our Delphi panel by summarising the most recent literature on realist evaluations to identify how and why rigour had been demonstrated and where gaps in expertise and rigour were evident. We also drew on our collective experience as realist evaluators, in training and supporting realist evaluations, and on the RAMESES email list to help us develop the briefing materials. Through discussion within the project team, we developed a list of issues related to quality that needed to be addressed when carrying out realist evaluations. These were then shared with the panel members and their feedback was sought. Once the panel members had provided their feedback on our briefing materials, we constructed a set of items for potential inclusion in the reporting standards and circulated these online to panel members. Panel members were asked to rank each potential item twice on a 7-point Likert scale, once for relevance and once for validity. They were also encouraged to provide free text comments.

RESULTS

We recruited 35 panel members from 27 organisations across six countries from nine different disciplines. Within three rounds our Delphi panel was able to reach consensus on 20 items that should be included in the reporting standards for realist evaluations. The overall response rates for all items for rounds 1, 2 and 3 were 94 %, 76 % and 80 %, respectively.

CONCLUSION

These reporting standards for realist evaluations have been developed by drawing on a range of sources. We hope that these standards will lead to greater consistency and rigour of reporting and make realist evaluation reports more accessible, usable and helpful to different stakeholders.

Realist evaluation of the antiretroviral treatment adherence club programme in selected primary healthcare facilities in the metropolitan area of Western Cape Province, South Africa: a study protocol

INTRODUCTION

Suboptimal retention in care and poor treatment adherence are key challenges to antiretroviral therapy (ART) in sub-Saharan Africa. Community-based approaches to HIV service delivery are recommended to improve patient retention in care and ART adherence. The implementation of the adherence clubs in the Western Cape province of South Africa was with variable success in terms of implementation and outcomes. The need for operational guidelines for its implementation has been identified. Therefore, understanding the contexts and mechanisms for successful implementation of the adherence clubs is crucial to inform the roll-out to the rest of South Africa. The protocol outlines an evaluation of adherence club intervention in selected primary healthcare facilities in the metropolitan area of the Western Cape Province, using the realist approach.

METHODS AND ANALYSIS

In the first phase, an exploratory study design will be used. Document review and key informant interviews will be used to elicit the programme theory. In phase two, a multiple case study design will be used to describe the adherence clubs in five contrastive sites. Semistructured interviews will be conducted with purposively selected programme implementers and members of the clubs to assess the context and mechanisms of the adherence clubs. For the programme's primary outcomes, a longitudinal retrospective cohort analysis will be conducted using routine patient data. Data analysis will involve classifying emerging themes using the context-mechanism-outcome (CMO) configuration, and refining the primary CMO configurations to conjectured CMO configurations. Finally, we will compare the conjectured CMO configurations from the cases with the initial programme theory. The final CMOs obtained will be translated into middle range theories.

ETHICS AND DISSEMINATION

The study will be conducted according to the principles of the declaration of Helsinki (1964). Ethics clearance was obtained from the University of the Western Cape. Dissemination will be done through publications and curation.

Systems science and systems thinking for public health: a systematic review of the field

OBJECTIVES

This paper reports on findings from a systematic review designed to investigate the state of systems science research in public health. The objectives were to: (1) explore how systems methodologies are being applied within public health and (2) identify fruitful areas of activity.

DESIGN

A systematic review was conducted from existing literature that draws on or uses systems science (in its various forms) and relates to key public health areas of action and concern, including tobacco, alcohol, obesity and the social determinants of health.

DATA ANALYSIS

117 articles were included in the review. An inductive qualitative content analysis was used for data extraction. The following were systematically extracted from the articles: approach, methodology, transparency, strengths and weaknesses. These were then organised according to theme (ie, commonalities between studies within each category), in order to provide an overview of the state of the field as a whole. The assessment of data quality was intrinsic to the goals of the review itself, and therefore, was carried out as part of the analysis.

RESULTS

4 categories of research were identified from the review, ranging from editorial and commentary pieces to complex system dynamic modelling. Our analysis of each of these categories of research highlighted areas of potential for systems science to strengthen public health efforts, while also revealing a number of limitations in the dynamic systems modelling being carried out in public health.

CONCLUSIONS

There is a great deal of interest in how the application of systems concepts and approach might aid public health. Our analysis suggests that soft systems modelling techniques are likely to be the most useful addition to public health, and align well with current debate around knowledge transfer and policy. However, the full range of systems methodologies is yet to be engaged with by public health researchers.