A realist evaluation of the management of a well-performing regional hospital in Ghana

Alternative payment models in Dutch hospital care: what works, how, why and under what circumstances? Protocol for a realist evaluation study

INTRODUCTION

The predominant provider payment models in healthcare, particularly fee-for-service, hinder the delivery of high-value care and can encourage healthcare providers to prioritise the volume of care over the value of care. To address these issues, healthcare providers, payers and policymakers are increasingly experimenting with alternative payment models (APMs), such as shared savings (SS) and bundled payment (BP). Despite a growing body of literature on APMs, there is still limited insight into what works in developing and implementing successful APMs, as well as how, why and under what circumstances. This paper presents the protocol for a study that aims to (1) identify these circumstances and reveal the underlying mechanisms through which outcomes are achieved and (2) identify transferrable lessons for successful APMs in practice.

METHODS AND ANALYSIS

Drawing on realist evaluation principles, this study will employ an iterative three-step approach to elicit a programme theory that describes the relationship between context, mechanisms and outcomes of APMs. The first step involves a literature review to identify the initial programme theory. The second step entails empirical testing of this theory via a multiple case study design including seven SS and BP initiatives in Dutch hospital care. We will use various qualitative and quantitative methods, including interviews with involved stakeholders, document analysis and difference-in-differences analyses. In the final step, these data and the applicable formal theories will be combined to test and refine the (I)PT and address the research objectives.

ETHICS AND DISSEMINATION

Ethical approval has been granted by the Research Ethics Review Committee of Erasmus School of Health Policy and Management (Project ID ETH2122-0170). Where necessary, informed consent will be obtained from study participants. Among other means, study results will be disseminated through a publicly available manual for stakeholders (eg, healthcare providers and payers), publications in peer-reviewed scientific journals and (inter)national conference presentations.

Sustainability of an outcome measure in outpatient stroke rehabilitation: A realist evaluation

RATIONALE

Only half of newly implemented evidence-based practices are sustained. Though poor sustainment can lead to negative consequences for clinical teams, organizations and patients, the causal explanations of sustainment are largely unknown.

AIMS AND OBJECTIVES

We aimed to ascertain how (mechanisms) and in what circumstances (context) a newly implemented outcome measure in rehabilitation was sustained or not (outcome).

METHODOLOGY

Informed by an integrated knowledge translation approach, we conducted a realist evaluation using a mixed method, embedded single case study design with data collection up to 18 months following the implementation of the Mayo-Portland Adaptability Inventory - version 4 (MPAI-4), a rehabilitation outcome measure. Quantitative data (survey and patient charts) was analysed using descriptive statistics, then integrated with qualitative data (interviews with 10 key informants) and analysed using inductive and deductive retroduction. We integrated the data to develop a case description and ultimately, to refine the programme theory to better understand the sustainability of the MPAI-4.

RESULTS

We linked context, mechanisms and outcomes, and also emphasized sustainability strategies in 18 explanations of how sustainability works. These explanations provide evidence for four overarching patterns: (1) implementation and sustainability phases are interconnected, (2) outcomes build on each other recursively, with patient benefits as the keystone outcome, (3) sustainment is achieved to varying levels across different sustainability outcomes [e.g., high level (e.g., rate of MPAI-4 scoring: 77.7%) to low level (e.g., rate of MPAI-4 application to clinical decision-making: 3.7%)] and (4) the work of sustaining the MPAI-4 is shared amongst different stakeholders.

CONCLUSION

Implementation teams can draw from this programme theory to improve the sustainment of outcomes measures while researchers could continue to refine the theory. Continued investigation of sustainability, including diverse and continuous sustainability outcomes, is needed to understand how to maintain improvements in quality of care and patient outcomes.

Realistic Evaluation of the Integrated Electronic Diagnosis Approach (IeDA) for the Management of Childhood Illnesses at Primary Health Facilities in Burkina Faso

BACKGROUND

In 2014, Terre des Hommes (Tdh) together with the Ministry of Health (MoH) launched the Integrated electronic Diagnosis Approach (IeDA) intervention in two regions of Burkina Faso consisting of supplying every health centre with a digital algorithm. A realistic evaluation was conducted to understand the implementation process, the mechanisms by which the IeDA intervention lead to change.

METHODS

Data collection took place between January 2016 and October 2017. Direct observation in health centres were conducted. In-depth interviews were conducted with 154 individuals including 92 healthcare workers (HCW) from health centres, 16 officers from district health authorities, 6 members of health centre management committees. In addition, 5 focus groups were organised with carers. The initial coding was based on a preliminary list of codes inspired by the middle-range theory (MRT).

RESULTS

Our results showed that the adoption of the electronic protocol depended on a multiplicity of management practices including role distribution, team work, problem solving approach, task monitoring, training, supervision, support and recognition. Such changes lead to reorganising the health team and redistributing roles before and during consultation, and positive atmosphere that included recognition of each team member, organisational commitment and sense of belonging. Conditions for such management changes to be effective included open dialog at all levels of the system, a minimum of resources to cover the support services and supervision and regular discussions focusing on solving problems faced by health centre teams.

CONCLUSION

This project reinforces the point that in a successful diffusion of IeDA, it is necessary to combine the introduction of technology with support and management mechanisms. It also important to highlight that managers' attitude plays a great place in the success of the intervention: open dialog and respect are crucial dimensions. This is aligned with the findings from other studies.

Power Dynamics Among Health Professionals in Nigeria: A Case Study of the Global Fund Policy Process

BACKGROUND

Health workers are central to health policy-making. Given health systems' complex, dynamic and political nature, various forms of 'hidden power' are at play as health workers navigate health systems. This study aims to explore the dynamics of power and its sources, and how this shapes policy-making and implementation within the Nigerian health systems context.

METHODS

The case study was the Global Fund grant in Nigeria, and results are based on an in-depth qualitative study involving 34 semi-structured key informant interviews (KIIs), board-meeting observations, and documentary analysis conducted in 2014 and 2016. Participants held mid to senior-level positions (eg, Director, Programme Manager) within organisations involved with Global Fund activities, particularly proposal development and implementation. Data were analysed using thematic analysis in order to gain insight into the power dynamics of health professionals in policy processes.

RESULTS

Medical professionals maintained dominance and professional monopoly, thereby controlling policy spaces. The structural and productive power of the biomedical discourse in policy-making encourages global actors and the local government's preference for rapid biomedical models that focus on medications, test kits, and the supply of health services, while neglecting aspects that would help us better understand the poor uptake of these services by those in need. The voices of the repressed groups (eg, non-clinical experts, patients and community based organisations) that better understand barriers to uptake of services are relegated.

CONCLUSION

Professional monopoly theories help illustrate how medical professionals occupy and maintain an elite position in the health system of Nigeria. Structural and agential factors specific to the contexts are key in maintaining this professional monopoly while limiting the opportunities for other health occupations' rise up the social status ladder.

Best and worst performing health facilities: A positive deviance analysis of perceived drivers of primary care performance in Nepal

Primary care services are on average of low quality in Nepal. However, there is marked variation in performance of basic clinical and managerial functions between primary health care centers. The determinants of variation in primary care performance in low- and middle-income countries have been understudied relative to the prominence of primary care in national health plans. We used the positive deviance approach to identify best and worst performing primary health care centers in Nepal and investigated perceived drivers of best performance. We selected eight primary health care centers in Province 1, Nepal, using an index of basic clinical and operational activities to identify four best and four worst performing primary health care centers. We conducted semi-structured, in-depth interviews with managers and clinical staff from each of the eight primary health care centers for a total of 32 interviews. We identified the following factors that distinguished best from worst performers: 1) Managing the facility effectively, 2) engaging local leadership, 3) building active community accountability, 4) assessing and responding to facility performance, 5) developing sources of funding, 6) compensating staff fairly, 7) managing clinical staff performance, and 8) promoting uninterrupted availability of supplies and equipment. These findings can be used to inform quality improvement efforts and health system reforms in Nepal and other similarly under-resourced health systems.

•

Local leaders and health workers felt good management was key to best performance.

•

Best performers reported strong leadership at both the facility and local levels.

•

Community accountability was also seen as a critical enabler of top performance.

•

Leaders of worst performers were less responsive to facility and community needs.

Factors Associated With Persistent Urinary Incontinence Among Women Undergoing Female Genital Fistula Surgery in the Democratic Republic of Congo From 2017 to 2019

Background

Despite high closure rates, residual urinary incontinence remains a common problem after successful closure of a vesico-vaginal fistula. The objective of this study was to identify factors associated with residual urinary incontinence in women with successful fistula closure in sites supported by the Fistula Care Plus project in the Democratic Republic of Congo (DRC).

Material and Methods

This was a retrospective cohort study using routine data extracted from the medical records of women undergoing fistula surgery in three hospitals supported by the Fistula Care Plus project in DRC between 2017 and 2019. We analyzed factors associated with residual urinary incontinence among a subsample of women with closed fistula at discharge. We collected data on sociodemographic, clinical, gynecological-obstetrical characteristics, and case management. Univariate and multivariate analyses were performed to determine the factors associated with residual urinary incontinence.

Results

Overall, 31 of 718 women discharged with closed fistula after repair (4.3%; 95% CI: 3.1-6.1) had residual incontinence. The leading causes identified in these women with residual incontinence were urethral voiding (6 women), short urethra (6 women), severe fibrosis (3 women) and micro-bladder (2 women). The prevalence of residual incontinence was higher among women who received repair at the Heal Africa (6.6%) and St Joseph's (3.7%) sites compared with the Panzi site (1.7%). Factors associated with increased odds of persistent urinary incontinence were the Heal Africa repair site (aOR: 54.18; 95% CI: 5.33-550.89), any previous surgeries (aOR: 3.17; 95% CI: 1.10-9.14) and vaginal surgical route (aOR: 6.78; 95% CI: 1.02-45.21).

Conclusion

Prior surgery and repair sites were the main predictors of residual incontinence after fistula closure. Early detection and management of urinary incontinence and further research to understand site contribution to persistent incontinence are needed.

An Investigation of the Relationship Between the Performance and Management Practices of Health Facilities Under a Performance-Based Financing Scheme in Nigeria

Whereas the effect of performance-based financing (PBF) on improving the quantity and quality of health services has been established, little is known about what matters for health facilities to improve performance under a PBF scheme. This study examined the associations between management practices and the performance of primary health care centers (PHCCs) under a PBF scheme in Nigeria. This study utilized longitudinal data on monthly institutional deliveries and outpatient visits collected between December 2011 and March 2016 from 111 randomly selected PHCCs in Adamawa, Ondo, and Nasarawa states of Nigeria. A management practices scorecard, based on a health facility survey conducted in April/May 2016, was used to derive management practices scores for the 111 PHCCs. The management practices examined included activities to recruit and retain clients, staff's attention to performance targets, listening and responding to client feedback, teamwork building, and addressing low-performing staff. A multilevel, multilinear regression model was used to investigate the associations between health facility performance (monthly number of institutional deliveries and outpatient visits) and management practices at the PHCCs, adjusting for key control variables (number of skilled health workers, the size of PHCC catchment population, PHCC quality score, seasonality, and states). Following PBF introduction, PHCCs with medium management score had 0.42 (95% CI 0.18-0.65; p<0.001) and 9.93 (95% CI 6.15-13.71; p<0.001) higher monthly improvement rates for institutional delivery and outpatient visits respectively compared to the PHCCs with low management score. Also, the PHCCs with high management scores had 0.49 (95%CI 0.28-0.70; p<0.001) and 5.10 (95%CI 1.76-8.44; p<0.003) higher monthly improvement rates for institutional delivery and outpatient visits compared to the PHCCs with low management scores. These findings suggest the importance of management practices in facilitating the effect of PBF on health facility performance, and the need to strengthen PHCC management practices in low- and-middle-income countries.

Realist Evaluation of the "Abiye" Safe Motherhood Initiative in Nigeria: Unveiling the Black-Box of Program Implementation and Health System Strengthening

Introduction

Realist evaluation studies have spanned different aspects of medicine, especially in the field of public health. However, very few of these studies explicitly detailed how program implementation triggered outcomes that could strengthen understanding of its effect on Health System Strengthening in specific settings. In low- and middle-income countries, like Nigeria, there is a paucity of realist evaluation studies, despite the implementation of multiple intervention programs and projects in these countries. This article is aimed at unveiling the black-box of program implementation and Health System Strengthening of the "Abiye" Safe Motherhood Program in Ondo State, Nigeria. Specifically, it identified the role of contextual factors in the "Abiye" program in Ondo State, determined the mechanisms that facilitated or constrained outcomes of the "Abiye" program, and developed a Context Mechanism Outcome (CMO) Configuration from which a Middle Range Theory (MRT) can be framed.

Methodology

This was qualitative research structured along with the realist domains (Context, Mechanism, and Outcome). The Initial Program Theory was validated by the qualitative study, after which a new MRT was developed. The study population comprised key stakeholders, secondary stakeholders, and primary stakeholders in the Abiye safe motherhood program. Data was collected through 10 key informant interviews, 28 in-depth interviews, and six focus group discussions sessions. Thematic analysis was used to analyze all the qualitative data collected, and seven themes with 19 subthemes emerged in the study.

Results

We identified 13 contextual factors under five principal areas, with most of the factors playing enabling roles, some playing inhibitory roles, while very few played both roles. We elicited eight mechanisms, and some of these facilitated the outcomes, while some constrained the outcomes of the program. Health system strengthening was a key feature of the outcome of the program. We developed a middle-range theory based on the 6 CMO configurations we elicited from the study.

Conclusion and Policy Implications

Realist evaluation is an iterative process that looks beyond the surface to generate evidence. By applying the realist approach, we generated pieces of evidence that can be adapted for policymaking in public health interventions in LMIC.

A guide to systems-level, participatory, theory-informed implementation research in global health

Implementation research is a multidisciplinary field that addresses the complex phenomenon of how context influences our ability to deliver evidence-informed healthcare. There is increasing realisation of the importance of applying robust implementation research to scale-up life-saving interventions that meet health-related sustainable development goals. However, the lack of high-quality implementation research is impeding our ability to meet these targets, globally. Within implementation research, theory refers to the proposed hypothesis and/or explanation of how an intervention is expected to interact with the local context and actors to bring about change. Although there is increasing interest in applying theory to understand how and why implementation programmes work in real-world settings, global health actors still tend to favour impact evaluations conducted in controlled environments. This may, in part, be due to the relative novelty as well as methodological complexity of implementation research and the need to draw on divergent disciplines, including epidemiology, implementation science and social sciences. Because of this, implementation research is faced with a particular set of challenges about how to reconcile different ways of thinking and constructing knowledge about healthcare interventions. To help translate some of the ambiguity surrounding how divergent theoretical approaches and methods contribute to implementation research, we draw on our multidisciplinary expertise in the field, particularly in global health. We offer an overview of the different theoretical approaches and describe how they are applied to continuously select, monitor and evaluate implementation strategies throughout the different phases of implementation research. In doing so, we offer a relatively brief, user-focused guide to help global health actors implement and report on evaluation of evidence-based and scalable interventions, programmes and practices.

A Realist Evaluation of Team Interventions in Acute Hospital Contexts-Use of Two Case Studies to Test Initial Programme Theories

BACKGROUND

Designing and implementing team interventions to improve quality and safety of care in acute hospital contexts is challenging. There is little emphasis in the literature on how contextual conditions impact interventions or how specific active ingredients of interventions impact on team members' reasoning and enact change. This realist evaluation helps to deepen the understanding of the enablers and barriers for effective team interventions in these contexts.

METHODS

Five previously developed initial programme theories were tested using case studies from two diverse hospital contexts. Data were collected from theory driven interviews (n = 19) in an Irish context and from previously conducted evaluative interviews (n = 16) in a US context. Data were explored to unpack the underlying social and psychological drivers that drove both intended and unintended outcomes. Patterns of regularity were identified and synthesised to develop middle-range theories (MRTs).

RESULTS

Eleven MRTs demonstrate how and why intervention resources introduced in specific contextual conditions enact reasoning mechanisms and generate intended and unintended outcomes for patients, team members, the team and organisational leaders. The triggered mechanisms relate to shared mental models; openness, inclusivity and connectedness; leadership and engagement; social identity and intrinsic motivational factors.

CONCLUSIONS

The findings provide valuable information for architects and facilitators of team interventions in acute hospital contexts, as well as help identify avenues for future research. Dataset: The data presented in this study are available on request from the corresponding author. The data are not publicly available due to their sensitive nature and potential identification of participants.

Performance management in complex adaptive systems: a conceptual framework for health systems

Existing performance management approaches in health systems in low-income and middle-income countries are generally ineffective at driving organisational-level and population-level outcomes. They are largely directive: they try to control behaviour using targets, performance monitoring, incentives and answerability to hierarchies. In contrast, enabling approaches aim to leverage intrinsic motivation, foster collective responsibility, and empower teams to self-organise and use data for shared sensemaking and decision-making.The current evidence base is too limited to guide reforms to strengthen performance management in a particular context. Further, existing conceptual frameworks are undertheorised and do not consider the complexity of dynamic, multilevel health systems. As a result, they are not able to guide reforms, particularly on the contextually appropriate balance between directive and enabling approaches. This paper presents a framework that attempts to situate performance management within complex adaptive systems. Building on theoretical and empirical literature across disciplines, it identifies interdependencies between organisational performance management, organisational culture and software, system-level performance management, and the system-derived enabling environment. It uses these interdependencies to identify when more directive or enabling approaches may be more appropriate. The framework is intended to help those working to strengthen performance management to achieve greater effectiveness in organisational and system performance. The paper provides insights from the literature and examples of pitfalls and successes to aid this thinking. The complexity of the framework and the interdependencies it describes reinforce that there is no one-size-fits-all blueprint for performance management, and interventions must be carefully calibrated to the health system context.

Bottom-up innovation for health management capacity development: a qualitative case study in a South African health district

BACKGROUND

As part of health system strengthening in South Africa (2012-2017) a new district health manager, taking a bottom-up approach, developed a suite of innovations to improve the processes of monthly district management team meetings, and the practices of managers and NGO partners attending them. Understanding capacity as a property of the health system rather than only of individuals, the research explored the mechanisms triggered in context to produce outputs, including the initial sensemaking by the district manager, the subsequent sensegiving and sensemaking in the team and how these homegrown innovations interacted with existing social processes and norms within the system.

METHODS

We conducted a realist evaluation, adopting the case study design, over a two-year period (2013-2015) in the district of focus. The initial programme theory was developed from 10 senior manager interviews and a literature review. To understand the processes and mechanisms triggered in the local context and identify outputs, we conducted 15 interviews with managers in the management team and seven with non-state actors. These were supplemented by researcher notes based on time spent in the district. Thematic analysis was conducted using the Context-Mechanism-Outcome configuration alongside theoretical constructs.

RESULTS

The new district manager drew on systems thinking, tacit and experiential knowledge to design bottom-up innovations. Capacity was triggered through micro-practices of sensemaking and sensegiving which included using sticks (positional authority, enforcement of policies, over-coding), intentionally providing justifications for change and setting the scene (a new agenda, distributed leadership). These micro-practices in themselves, and by managers engaging with them, triggered a generative process of buy-in and motivation which influenced managers and partners to participate in new practices within a routine meeting.

CONCLUSION

District managers are well placed to design local capacity development innovations and must draw on systems thinking, tacit and experiential knowledge to enable relevant 'bottom-up' capacity development in district health systems. By drawing on soft skills and the policy resources (hardware) of the system they can influence motivation and buy-in to improve management practices. From a systems perspective, we argue that capacity development can be conceived of as part of the daily activity of managing within routine spaces.

Measuring management practices in primary health care facilities - development and validation of management practices scorecard in Nigeria

Background

In low- and middle-income countries, there is scarcity of validated and reliable measurement tools for health facility management, and many interventions to improve primary health care (PHC) facilities are designed without adequate evidence base on what management practices are critical.

Objective

This article developed and validated a scorecard to measure management practices at primary health care facilities under the performance-based financing (PBF) scheme in Nigeria.

Methods

Relevant management practice domains and indicators for PHC facilities were determined based on literature review and a prior qualitative study conducted in Nigeria. The domains and indicators were tested for face validity via experts review and organized into an interviewer-administered scorecard. A stratified random sampling of PHC facilities in three States in Nigeria was conducted to assess the reliability and construct validity of the scorecard. Inter-rater reliability using inter-class correlation (ICC) (1, k) was assessed with one-way ANOVA. Exploratory factor analysis (EFA) was conducted to assess the construct validity, and an updated factor structure were developed.

Results

32 indicators and 6 management practice domains were initially described. Ordinal responses were derived for each indicator. Data on the scorecard were obtained from 111 PHC facilities. The ICC of mean ratings for each team of judges was 0.94. The EFA identified 6 domains (Stakeholder engagement and communication; Community-level activities; Update of plan and target; Performance management; Staff attention to planning, target, and performance; and Drugs and financial management) and reduced the number of indicators to 17. The average communality of selected items was 0.45, and item per factor ratio was 17:6.

Conclusions

Despite a few areas for further refinement, this paper presents a reliable and valid scorecard for measuring management practices in PHC facilities. The scorecard can be applied for routine supervisory visits to PHC facilities, and can help accumulate knowledge on facility management, how it affects performance, and how it may be strengthened.

Context-acceptability theories: example of family planning interventions in five African countries

BACKGROUND

Family planning (FP) can lengthen birth intervals and potentially reduce the risk of foetal death, low birthweight, prematurity, and being small for gestational age. Effective FP is most easily achieved through access to and acceptability of modern contraceptive methods (MCMs). This study aimed to identify mechanisms of acceptability and the contexts in which they are triggered and to generate theories to improve the selection and implementation of effective interventions by studying an intervention integrating FP with childhood immunisation services.

METHODS

Qualitative interpretative synthesis of findings from realist evaluations of FP interventions in five African countries was guided by an analytical framework. Empirical mechanisms of acceptability were identified from semi-structured interviews and focus group discussions with key stakeholders (N = 253). The context in which these mechanisms were triggered was also defined. Empirical mechanisms of acceptability were matched to constructs of a theoretical framework of acceptability. Context-acceptability theories (CATs) were developed, which summarised constructs of acceptability triggered for specific actors in specified contexts. Examples of interventions that may be used to trigger acceptability for these actors were described.

RESULTS

Seven CATs were developed for contexts with strong beliefs in religious values and with powerful religious leaders, a traditional desire for large families, stigmatisation of MCM use, male partners who are non-accepting of FP, and rumours or experiences of MCM side effects. Acceptability mechanisms included alignment with values and beliefs without requiring compromise, actors' certainty about their ability to avoid harm and make the intervention work, and understanding the intervention and how it works. Additionally, acceptability by one group of actors was found to alter the context, triggering acceptability mechanisms amongst others.

CONCLUSIONS

This study demonstrated the value of embedding realist approaches within implementation research. CATs are transferable theories that answer the question: given the context, what construct of acceptability does an intervention need to trigger, or more simply, what intervention do we need to apply here to achieve our outcomes? CATs facilitate transfer of interventions across geographies within defined contexts.

Which mechanisms explain motivation the of primary health workers? Insights from the realist evaluation of a maternal and child health programme in Nigeria

INTRODUCTION

Well-trained, adequately skilled and motivated primary healthcare (PHC) workers are essential for attaining universal health coverage (UHC). While there is abundant literature on the drivers of workforce motivation, published knowledge on the mechanisms of motivation within different contexts is limited, particularly in resource-limited countries. This paper contributes to health workforce literature by reporting on how motivation works among PHC workers in a maternal and child health (MCH) programme in Nigeria.

METHODS

We adopted a realist evaluation design combining document review with 56 in-depth interviews of PHC workers, facility managers and policy-makers to assess the impact of the MCH programme in Anambra State, Nigeria. A realist process of theory development, testing and consolidation was used to understand how and under what circumstances the MCH programme impacted on workers' motivation and which mechanisms explain how motivation works. We drew on Herzberg's two-factor and Adam's equity theories to unpack how context shapes worker motivation.

RESULTS

A complex and dynamic interaction between the MCH programme and organisational and wider contexts triggered five mechanisms which explain PHC worker motivation: (1) feeling supported, (2) feeling comfortable with work environment, (3) feeling valued, (4) morale and confidence to perform tasks and (5) companionship. Some mechanisms were mutually reinforcing while others operated in parallel. Other conditions that enabled worker motivation were organisational values of fairness, recognition of workers' contributions and culture of task-sharing and teamwork.

CONCLUSIONS

Policy designs and management strategies for improving workforce performance, particularly in resource-constrained settings should create working environments that foster feelings of being valued and supported while enabling workers to apply their knowledge and skills to improve healthcare delivery and promote UHC. Future research can test the explanatory framework generated by this study and explore differences in motivational mechanisms among different cadres of PHC workers to inform cadre-related motivational interventions.

Using implementation science theories and frameworks in global health

In global health, researchers and decision makers, many of whom have medical, epidemiology or biostatistics background, are increasingly interested in evaluating the implementation of health interventions. Implementation science, particularly for the study of public policies, has existed since at least the 1930s. This science makes compelling use of explicit theories and analytic frameworks that ensure research quality and rigour. Our objective is to inform researchers and decision makers who are not familiar with this research branch about these theories and analytic frameworks. We define four models of causation used in implementation science: intervention theory, frameworks, middle-range theory and grand theory. We then explain how scientists apply these models for three main implementation studies: fidelity assessment, process evaluation and complex evaluation. For each study, we provide concrete examples from research in Cuba and Africa to better understand the implementation of health interventions in global health context. Global health researchers and decision makers with a quantitative background will not become implementation scientists after reading this article. However, we believe they will be more aware of the need for rigorous implementation evaluations of global health interventions, alongside impact evaluations, and in collaboration with social scientists.

An exploratory study of the acceptability of indoor residual spraying for malaria control in upper western Ghana

Background

Despite the implementation of the World Health Organisation’s recommended indoor residual spraying (IRS) intervention in the upper west region of Ghana to reduce malaria morbidity and mortality, the uptake of this intervention remains low. This study explores the facilitators and barriers to the acceptability and community uptake of indoor residual spraying in a highly endemic region of Ghana.

Methods

The health belief model (HBM) and realist evaluation framework were used to inform the study. A qualitative enquiry was conducted between April to October 2016. Data were collected through focus group discussions and semi-structured interviews with program stakeholders including community members, AngloGold Ashanti malaria control (AGA Mal) spray operators, and AGA Mal officials.

Results

A total of 101 people participated in the study. Considerable barriers to community acceptance of indoor residual spraying (IRS) were found, including, dislike of spray insecticides, inadequate information, religious and cultural beliefs, perceived low efficacy of IRS, difficulties with packing, unprofessional conduct of IRS spray operators, and other operational barriers to spraying. Facilitators of IRS uptake included a perceived effectiveness of IRS in preventing malaria and reducing mosquito bites, incidental benefits, respect for authority, training and capacity building, and sensitization activities.

Conclusion

The numerous barriers to indoor residual spraying acceptance and implications show that acceptance levels could be improved. However, measures are required to address householders’ concerns and streamline operational barriers to increase community uptake of indoor residual spraying.

Unravelling the role of leadership in motivation of health workers in a Moroccan public hospital: a realist evaluation

OBJECTIVES

This study aimed at opening the black box of the relationship between leadership and motivation of health workers by focusing on a high-performance hospital in Morocco.

DESIGN

We adopted the realist evaluation approach and used the case study design to test the initial programme theory we formulated on the basis of a scoping review on complex leadership. We used the Intervention-Context-Actors-Mechanism-Outcome Configuration as a heuristic tool to identify plausible causal configurations.

SETTINGS

Since 2000, the Ministry of Health in Morocco initiated many reforms in the frame of the governmental deconcentration process called 'advanced regionalisation'. The implementation of these reforms is hampered by inadequate human resource management capacities of local health system managers. Yet, the National 'Concours Qualité', a national quality assurance programme implemented since 2007, demonstrated that there are many islands of excellence. We explore how leadership may play a role in explaining these islands of excellence.

PARTICIPANTS

We carried out a document review, 18 individual interviews and 3 group discussions (with doctors, administrators and nurses), and non-participant observations during a 2-week field visit in January-February 2018.

RESULTS

We confirmed that effective leaders adopt an appropriate mix of transactional, transformational and distributed leadership styles that fits the mission, goals, organisational culture and nature of tasks of the organisation and the individual characteristics of the personnel when organisational culture is conducive. Leadership effectiveness is conditioned by the degree of responsiveness to the basic psychological needs of autonomy, competence and relatedness, perceived organisational support and perceived supervisor support. Transactional and overcontrolling leadership behaviour decreased the satisfaction of the need for autonomy and mutual respect. By distributing leadership responsibilities, complex leaders create an enabling environment for collective efficacy and creative problem solving.

CONCLUSIONS

We found indications that in the Moroccan context, well-performing hospitals could be characterised by a good fit between leadership styles, organisational characteristics and individual staff attributes.

Preliminary validation of the PRImary care facility Management Evaluation tool (PRIME-Tool), a national facility management survey implemented in Ghana

Background

The management quality of healthcare facilities has consistently been linked to facility performance, but available tools to measure management are costly to implement, often hospital-specific, not designed for low- and middle-income countries (LMICs), nor widely deployed. We addressed this gap by developing the PRImary care facility Management Evaluation Tool (PRIME-Tool), a primary health care facility management survey for integration into routine national surveys in LMICs. We present an analysis of the tool’s psychometric properties and suggest directions for future improvements.

Methods

The PRIME-Tool assesses performance in five core management domains: Target setting, Operations, Human resources, Monitoring, and Community engagement. We evaluated two versions of the PRIME-Tool. We surveyed 142 primary health care (PHC) facilities in Ghana in 2016 using the first version (27 items) and 148 facilities in 2017 using the second version (34 items). We calculated floor and ceiling effects for each item and conducted exploratory factor analyses to examine the factor structure for each year and version of the tool. We developed a revised management framework and PRIME-tool as informed by these exploratory results, further review of management theory literature, and co-author consensus.

Results

The majority (17 items in 2016, 23 items in 2017) of PRIME-Tool items exhibited ceiling effects, but only three (2 items in 2016, 3 items in 2017) showed floor effects. Solutions suggested by factor analyses did not fully fit our initial hypothesized management domains. We found five groupings of items that consistently loaded together across each analysis and named these revised domains as Supportive supervision and target setting, Active monitoring and review, Community engagement, Client feedback for improvement, and Operations and financing.

Conclusion

The revised version of the PRIME-Tool captures a range of important and actionable information on the management of PHC facilities in LMIC contexts. We recommend its use by other investigators and practitioners to further validate its utility in PHC settings. We will continue to refine the PRIME-Tool to arrive at a parsimonious tool for tracking PHC facility management quality. Better understanding the functional components of PHC facility management can help policymakers and frontline managers drive evidence-based improvements in performance.

Working in partnership with communities to improve health and research outcomes. Comparisons and commonalities between the UK and South Africa

Community and public participation and involvement is an underpinning principle of primary health care, an essential component of a social justice-orientated approach to health care and a vehicle to improving health outcomes for patients, public and communities. However, influenced by history and context, there are intrinsic issues surrounding power imbalance and other barriers to partnerships between communities, public, policy makers and researchers. It is important to acknowledge these issues, and through doing so share experiences and learn from those working within very different settings.In South Africa, community participation is seen as a route to decolonisation. It is also integral to the core functions of South African Higher Education Institutes, alongside teaching and research. In the UK, there has also been a history of participation and involvement as part of a social rights movement, but notably public involvement has become embedded in publicly funded health research as a policy imperative.In this paper, we draw on our respective programmes of work in public and community participation and involvement. These include a South African community engagement project to reduce teenage pregnancy and HIV infection working through a partnership between teachers, students and university academics, and a national evaluation in England of public involvement in applied health research. We begin by highlighting the lack of clarity and terms used interchangeably to describe participation, engagement and involvement. Frameworks for partnership working with relevance to South Africa and the UK are then analysed, suggesting key themes of relationships, working together, and evaluation and monitoring. The South African project and examples of public involvement in English primary and community care research are examined through these themes. We conclude the paper by mapping out common enablers and barriers to partnership working within these very different contexts.

A Realist Evaluation of a Community-Centered Radio Initiative for Health and Development in Mindanao, Philippines

A small, local-level communication initiative aimed to bring about social change and development in communities affected by sustained conflict in Mindanao, Philippines. A realist evaluation involved a secondary analysis of existing data sets that revealed previously undetected mechanisms and 13 outcomes for improving dialogue, livelihood, and participatory communication. This article describes the method developed for the realist evaluation and constructs Context-Mechanism-Outcome configurations from the existing data sets. The realist evaluation represents what took place in a context characterized by conflict, disadvantage and disempowerment through 2 key mechanisms, community-centered radio and community radio volunteers. Both mechanisms became voices for the voiceless. The community-centered radio program supported community volunteers to mobilize communities to participate in radio segments, offering opportunities for their voices to be heard on local issues resulting in discussion, provision of services not previously offered, community leaders more responsive to community needs, and coordinated community action that resolved needs.

Health system strengthening-Reflections on its meaning, assessment, and our state of knowledge

Comprehensive reviews of health system strengthening (HSS) interventions are rare, partly because of lack of clarity on definitions of the term but also the potentially huge scale of the evidence. We reflect on the process of undertaking such an evidence review recently, drawing out suggestions on definitions of HSS and approaches to assessment, as well as summarising some key conclusions from the current evidence base. The key elements of a clear definition include, in our view, consideration of scope (with effects cutting across building blocks in practice, even if not in intervention design, and also tackling more than one disease), scale (having national reach and cutting across levels of the system), sustainability (effects being sustained over time and addressing systemic blockages), and effects (impacting on health outcomes, equity, financial risk protection, and responsiveness). We also argue that agreeing a framework for design and evaluation of HSS is urgent. Most HSS interventions have theories of change relating to specific system blocks, but more work is needed on capturing their spillover effects and their contribution to meeting overarching health system process goals. We make some initial suggestions about such goals, to reflect the features that characterise a "strong health system." We highlight that current findings on "what works" are just indicative, given the limitations and biases in what has been studied and how, and argue that there is need to rethink evaluation methods for HSS beyond finite interventions and narrow outcomes. Clearer concepts, frameworks, and methods can support more coherent HSS investment.

Factors supporting and constraining the implementation of robot-assisted surgery: a realist interview study

OBJECTIVE

To capture stakeholders' theories concerning how and in what contexts robot-assisted surgery becomes integrated into routine practice.

DESIGN

A literature review provided tentative theories that were revised through a realist interview study. Literature-based theories were presented to the interviewees, who were asked to describe to what extent and in what ways those theories reflected their experience. Analysis focused on identifying mechanisms through which robot-assisted surgery becomes integrated into practice and contexts in which those mechanisms are triggered.

SETTING

Nine hospitals in England where robot-assisted surgery is used for colorectal operations.

PARTICIPANTS

Forty-four theatre staff with experience of robot-assisted colorectal surgery, including surgeons, surgical trainees, theatre nurses, operating department practitioners and anaesthetists.

RESULTS

Interviewees emphasised the importance of support from hospital management, team leaders and surgical colleagues. Training together as a team was seen as beneficial, increasing trust in each other's knowledge and supporting team bonding, in turn leading to improved teamwork. When first introducing robot-assisted surgery, it is beneficial to have a handpicked dedicated robotic team who are able to quickly gain experience and confidence. A suitably sized operating theatre can reduce operation duration and the risk of de-sterilisation. Motivation among team members to persist with robot-assisted surgery can be achieved without involvement in the initial decision to purchase a robot, but training that enables team members to feel confident as they take on the new tasks is essential.

CONCLUSIONS

We captured accounts of how robot-assisted surgery has been introduced into a range of hospitals. Using a realist approach, we were also able to capture perceptions of the factors that support and constrain the integration of robot-assisted surgery into routine practice. We have translated these into recommendations that can inform future implementations of robot-assisted surgery.

"Posting policies don't change because there is peace or war": the staff deployment challenges for two large health employers during and after conflict in Northern Uganda

BACKGROUND

Between 1986 and 2006, the Acholi region in Uganda experienced armed conflict which disrupted the health system including human resources. Deployment of health workers during and after conflict raises many challenges for managers due to issues of security and staff shortage. We explored how deployment policies and practices were adapted during the conflict and post-conflict periods with the aim of drawing lessons for future responses to similar conflicts.

METHODS

A cross-sectional study with qualitative techniques for data collection to investigate deployment policy and practice during the conflict and post-conflict period (1986-2013) was used. The study was conducted in Amuru, Gulu and Kitgum districts in Northern Uganda in 2013. Two large health employers from Acholi were selected: the district local government and Lacor hospital, a private provider. Twenty-three key informants' interviews were conducted at the national and district level, and in-depth interviews with 10 district managers and 25 health workers. This study focused on recruitment, promotions, transfers and bonding to explore deployment policies and practices.

RESULTS

There was no evidence of change in deployment policy due to conflict, but decentralisation from 1997 had a major effect for the local government employer. Lacor hospital had no formal deployment policy until 2001. Health managers in government and those working for Lacor hospital both implemented deployment policies pragmatically, especially because of the danger to staff in remote facilities. Lacor hospital introduced bonding agreements to recruit and staff their facilities. While managers in both organisations implemented the deployment policies as best as they could, some deployment-related decisions could lead to longer-term problems.

CONCLUSION

It may not be possible or even appropriate to change deployment policy during or after conflict. However, given sufficient autonomy, local managers can adapt deployment policies appropriately to need, but they should also be supported with the necessary human resource management skills to enable them make appropriate decisions for deployment.

Examining clinical leadership in Kenyan public hospitals through the distributed leadership lens

Clinical leadership is recognized as a crucial element in health system strengthening and health policy globally yet it has received relatively little attention in low and middle income countries (LMICs). Moreover, analyses of clinical leadership tend to focus on senior-level individual leaders, overlooking a wider constellation of middle-level leaders delivering health care in practice in a way affected by their health care context. Using the theoretical lens of ‘distributed leadership’, this article examines how middle-level leadership is practised and affected by context in Kenyan county hospitals, providing insights relevant to health care in other LMICs. The article is based on empirical qualitative case studies of clinical departmental leadership in two Kenyan public hospitals, drawing on data gathered through ethnographic observation, interviews and focus groups. We inductively and iteratively coded, analysed and theorized our findings. We found the distributed leadership lens useful for the purpose of analysing middle-level leadership in Kenyan hospitals, although clinical departmental leadership was understood locally in more individualized terms. Our distributed lens revealed medical and nursing leadership occurring in parallel and how only doctors in leadership roles were able to directly influence behaviour among their medical colleagues, using inter-personal skills, power and professional expertize. Finally, we found that Kenyan hospital contexts were characterized by cultures, norms and structures that constrained the way leadership was practiced. We make a theoretical contribution by demonstrating the utility of using distributed leadership as a lens for analysing leadership in LIMC health care contexts, revealing how context, power and inter-professional relationships moderate individual leaders’ ability to bring about change. Our findings, have important implications for how leadership is conceptualized and the way leadership development and training are provided in LMICs health systems.

A translational case study of empowerment into practice: A realist evaluation of a member-led dementia empowerment service

Involving people with dementia in decision-making is widely accepted as a means of empowering them to lead more independent lives and have more meaningful roles in shaping their care. However, there is a need to conduct rigorous evaluations of empowerment-driven services and policies in order to develop a deeper understanding about how to optimise successful implementation. This paper presents the results of an evaluation of Dementia Northern Ireland, an organisation initiated and led by people with dementia. We used a realist evaluation approach that comprised interviews with 15 people with dementia, three staff and two board members, ethnographic observations, along with documentary analysis to identify ‘what works, for whom, under what circumstances’. The analysis used realist logic to build up context-mechanism-outcome configurations. The Dementia Northern Ireland service model of empowerment revolved around the formation and maintenance of social groups of people with dementia. Facilitators, recruited and selected by people with dementia, supported six groups, consisting of one to four members with mild to moderate cognitive impairment. Facilitators helped expand empowerment groups, facilitate decision-making, awareness raising and consultation opportunities with group members. The ‘Empowerment Groups’ appeared to lead to the development of a shared social identity and a sense of collective strength as indicated by interview and observational data demonstrating an activist mentality among group members to challenge the stigma surrounding dementia. Group members also reported improved quality of life. Widespread implementation of the empowerment model has the potential to lead to reduced stigma and greater social inclusion, increased involvement of people with dementia as active co-producers of policy and service development, better services and support. This case study of Dementia Northern Ireland illustrates that there are boundaries and challenges to empowerment in terms of requiring additional support from staff without dementia. However, despite these challenges, empowerment-driven organisations can and should be committed to involving members in lead roles and key decision-making.

Characterizing performance improvement in primary care systems in Mesoamerica: A realist evaluation protocol

Background. Evaluations of performance measurement and management interventions in public, primary care delivery systems of low- and middle-income countries are scarce. In such contexts, few studies to date have focused on characterizing how, why and under what contextual conditions do such complex, multifaceted arrangements lead to intended and unintended consequences for the healthcare workforce, the healthcare organizations involved, and the communities that are served. Methods. Case-study design with purposeful outlier sampling of high-performing primary care delivery systems in El Salvador and Honduras, as part of the Salud Mesoamerica Initiative. Case study design is suitable for characterizing individual, interpersonal and collective mechanisms of change in complex adaptive systems. The protocol design includes literature review, document review, non-participant observation, and qualitative analysis of in-depth interviews. Data analysis will use inductive and deductive approaches to identify causal patterns organized as ‘context-mechanism-outcome’ configurations. Findings will be triangulated with existing secondary data sources collected including country-specific performance measurement data, impact, and process evaluations conducted by the Salud Mesoamerica Initiative. Discussion. This realist evaluation protocol aims to characterize how, why and under what conditions do performance measurement and management arrangements contribute to the improvement of primary care system performance in two low-income countries.

A realistic evaluation approach highlighted the success factors and difficulties of an innovative and comprehensive malnutrition programme in Madagascar

AIM

Malnutrition has multiple causes, not just a lack of calories. We explored the successes and barriers of an innovative, comprehensive programme in Madagascar that educated mothers in particular about nutrition.

METHODS

The outpatient programme in Antananarivo treats 2400 malnourished children from 6 to 59 months each year. The 35-day programme comprises health checks, cooking demonstrations, the distribution of enriched flour supplements and two food-related games that are called Nutricartes and are based on boards and picture cards. In 2014, we involved 74 of the children's mothers in a realistic evaluation process, which investigates the mechanisms that produce a particular outcome in a given context.

RESULTS

The mechanisms of success were the community-based aspects of the programme, the kindness and equality displayed by the nutritional assistants and improved household budgeting and hygiene. Children ate better and improved their health, which reduced medical visits and costs. Finally, the mothers became proud of their achievements and told other mothers what they had learnt. Adherence was greatest when the women were on a minimum wage and strongly motivated.

CONCLUSION

The effect of this multifaceted programme was that the mothers received sustainable education about healthy eating and improved childhood health and nutrition.

Community based interventions for strengthening adolescent sexual reproductive health and rights: how can they be integrated and sustained? A realist evaluation protocol from Zambia

BACKGROUND

Research that explores how community-based interventions for strengthening adolescent sexual reproductive health and rights (SRHR) can be integrated and sustained in community health systems, is, to the best of our knowledge, very scarce, if not absent. It is important to document mechanisms that shape integration process in order to improve health systems' responsiveness towards adolescents' SRHR. This realist evaluation protocol will contribute to this knowledge in Zambia where there is increased attention towards promoting maternal, neonatal and child health as a means of addressing the current high early pregnancy and marriage rates. The protocol will ascertain: why, how, and under what conditions the integration of SRHR interventions into Zambian community health systems will optimise (or not) acceptability and adoption of SRHR services. This study is embedded within a randomized controlled trial - "Research Initiative to Support the Empowerment of Girls (RISE)"- which aims to reduce adolescent girl pregnancies and marriages through a package of interventions including economic support to families, payment of school fees to keep girls in school, pocket money for girls, as well as youth club and community meetings on reproductive health.

METHODS

This is a multiple-case study design. Data will be collected from schools, health facilities and communities through individual and group interviews, photovoice, documentary review, and observations. The study process will involve 1) developing an initial causal theory that proposes an explanation of how the integration of a community-based intervention that aimed to integrate adolescent SRHR into the community health system may lead to adolescent-friendly services; 2) refining the causal theory through case studies; 3) identifying contextual conditions and mechanisms that shape the integration process; and 4) finally proposing a refined causal theory and set of recommendations to guide policy makers, steer further research, and inform teaching programmes.

DISCUSSION

The study will document relevant values as well as less formal and horizontal mechanisms which shape the integration process of SRHR interventions at community level. Knowledge on mechanisms is essential for guiding development of strategies for effectively facilitating the integration process, scaling up processes and sustainability of interventions aimed at reducing SRH problems and health inequalities among adolescents.

'We are planning to leave, all of us'-a realist study of mechanisms explaining healthcare employee turnover in rural Ethiopia

BACKGROUND

We study healthcare employees' turnover intentions in the Afar National Regional State of Ethiopia. This rural region is experiencing the globally felt crisis in human resources, which is inhibiting its ability to meet health-related sustainable development goals.

METHODS

Realist case study which combines literature study and qualitative analysis of interview and focus group discussion data, following a realist case study protocol.

RESULTS

A large majority of employees has turnover intentions. Building on Herzberg's two-factor theory, person-environment fit theory, as well as recent sub-Saharan evidence, analysis of the collected data yields four turnover mechanisms: (1) lack of social and personal opportunities in the region, (2) dissonance between management logic and professional logic, (3) standards of service operations are hard to accept, and (4) lack of financial improvement opportunities.

CONCLUSIONS

While the first and fourth mechanisms may be out of reach for local (human resource) management interventions, the second and third mechanisms proposed to explain health workforce turnover appear to be amenable to local (human resource) management interventions to strengthen healthcare. These mechanisms are likely to play a role in other remote sub-Saharan regions as well.

The effect of human resource management on performance in hospitals in Sub-Saharan Africa: a systematic literature review

Hospitals in Sub-Saharan Africa (SSA) face major workforce challenges while having to deal with extraordinary high burdens of disease. The effectiveness of human resource management (HRM) is therefore of particular interest for these SSA hospitals. While, in general, the relationship between HRM and hospital performance is extensively investigated, most of the underlying empirical evidence is from western countries and may have limited validity in SSA. Evidence on this relationship for SSA hospitals is scarce and scattered. We present a systematic review of empirical studies investigating the relationship between HRM and performance in SSA hospitals.Following the PRISMA protocol, searching in seven databases (i.e., Embase, MEDLINE, Web of Science, Cochrane, PubMed, CINAHL, Google Scholar) yielded 2252 hits and a total of 111 included studies that represent 19 out of 48 SSA countries.From a HRM perspective, most studies researched HRM bundles that combined practices from motivation-enhancing, skills-enhancing, and empowerment-enhancing domains. Motivation-enhancing practices were most frequently researched, followed by skills-enhancing practices and empowerment-enhancing practices. Few studies focused on single HRM practices (instead of bundles). Training and education were the most researched single practices, followed by task shifting.From a performance perspective, our review reveals that employee outcomes and organizational outcomes are frequently researched, whereas team outcomes and patient outcomes are significantly less researched. Most studies report HRM interventions to have positively impacted performance in one way or another. As researchers have studied a wide variety of (bundled) interventions and outcomes, our analysis does not allow to present a structured set of effective one-to-one relationships between specific HRM interventions and performance measures. Instead, we find that specific outcome improvements can be accomplished by different HRM interventions and conversely that similar HRM interventions are reported to affect different outcome measures.In view of the high burden of disease, our review identified remarkable little evidence on the relationship between HRM and patient outcomes. Moreover, the presented evidence often fails to provide contextual characteristics which are likely to induce variety in the performance effects of HRM interventions. Coordinated research efforts to advance the evidence base are called for.

Pathways to high and low performance: factors differentiating primary care facilities under performance-based financing in Nigeria

The determinants of primary health facility performance in developing countries have not been well studied. One of the most under-researched areas is health facility management. This study investigated health facilities under the pilot performance-based financing (PBF) scheme in Nigeria, and aimed to understand which factors differentiated primary health care centres (PHCCs) which had performed well, vs those which had not, with a focus on health facility management practices. We used a multiple case study where we compared two high-performing PHCCs and two low-performing PHCCs for each of the two PBF target states. Two teams of two trained local researchers spent 1 week at each PHCC and collected semi-structured interview, observation and documentary data. Data from interviews were transcribed, translated and coded using a framework approach. The data for each PHCC were synthesized to understand dynamic interactions of different elements in each case. We then compared the characteristics of high and low performers. The areas in which critical differences between high and low-performers emerged were: community engagement and support; and performance and staff management. We also found that (i) contextual and health system factors particularly staffing, access and competition with other providers; (ii) health centre management including community engagement, performance management and staff management; and (iii) community leader support interacted and drove performance improvement among the PHCCs. Among them, we found that good health centre management can overcome some contextual and health system barriers and enhance community leader support. This study findings suggest a strong need to select capable and motivated health centre managers, provide long-term coaching in managerial skills, and motivate them to improve their practices. The study also highlights the need to position engagement with community leaders as a key management practice and a central element of interventions to improve PHCC performance.

Enabling relational leadership in primary healthcare settings: lessons from the DIALHS collaboration

Strong management and leadership competencies have been identified as critical in enhancing health system performance. While the need for strong health system leadership has been raised, an important undertaking for health policy and systems researchers is to generate lessons about how to support leadership development (LD), particularly within the crisis-prone, resource poor contexts that are characteristic of Low- and Middle-Income health systems. As part of the broader DIALHS (District Innovation and Action Learning for Health Systems Development) collaboration, this article reflects on 5 years of action learning and engagement around leadership and LD within primary healthcare (PHC) services. Working in one sub-district in Cape Town, we co-created LD processes with managers from nine PHC facilities and with the six members of the sub-district management team. Within this article, we seek to provide insights into how leadership is currently practiced and to highlight lessons about whether and how our approach to LD enabled a strengthening of leadership within this setting. Findings suggest that the sub-district is located within a hierarchical governance context, with performance monitored through the use of multiple accountability mechanisms including standard operating procedures, facility audits and target setting processes. This context presents an important constraint to the development of a more distributed, relational leadership. While our data suggest that gains in leadership were emerging, our experience is of a system struggling to shift from a hierarchical to a more relational understanding of how to enable improvements in performance, and to implement these changes in practice.

Developing programme theories as part of a realist evaluation of a healthcare quality improvement programme

The application of realist approaches to health services research and evaluation has continued to grow over the past two decades. However, difficulties in defining and operationalising key realist concepts of contexts, mechanisms and outcomes in healthcare settings continue to be recognised within the realist evaluation literature. Reflecting on an ongoing realist evaluation of a healthcare quality improvement programme across an inter-organisational context, this article explores some of the methodological challenges encountered by the author in the early stages of programme theory development. An individualised operationalisation and application of realist concepts are presented to demonstrate how initial programme theories can be developed despite the methodological difficulties presented.

Optimum hospice at home services for end-of-life care: protocol of a mixed-methods study employing realist evaluation

INTRODUCTION

Hospice at home (HAH) services aim to enable patients to be cared for and die in their place of choice, if that is at home, and to achieve a 'good death'. There is a considerable range of HAH services operating in England. The published evidence focuses on evaluations of individual services which vary considerably, and there is a lack of consistency in terms of the outcome measures reported. The evidence, therefore, does not provide generalisable information, so the question 'What are the features of hospice at home service models that work, for whom, and under what circumstances?' remains unanswered. The study aims to answer this question.

METHODS AND ANALYSIS

This is a mixed-methods study in three phases informed by realist evaluation methodology. All HAH services in England will be invited to participate in a telephone survey to enable the development of a typology of services. In the second phase, case study sites representing the different service types will collect patient data and recruit carers, service managers and commissioners to gather quantitative and qualitative data about service provision and outcomes. A third phase will synthesise and refine the results through consensus workshops.

ETHICS AND DISSEMINATION

The first survey phase has university ethics approval and the second phase, Integrated Research Application System (IRAS) and Health Research Authority (HRA) approval (IRAS ID:205986, REC:17/LO/0880); the third phase does not require ethics approval. Dissemination will be facilitated by project coapplicants with established connections to national policy-making forums, in addition to publications, conference presentations and reports targeted to service providers and commissioners.

Use of clinical guidelines in remote Australia: A realist evaluation

AIM

The aim of this evaluation was to assess the acceptability, accessibility, and compliance with the 2014 editions of the Remote Primary Health Care Manuals (RPHCM) in health care centres across remote areas of Northern and Central Australia.

METHOD

To undertake a comprehensive evaluation that considered context, the evaluation used a realist evaluation framework. The evaluation used a variety of methods including interviews and survey to develop and test a programme theory.

RESULTS

Many remote health practitioners have adopted standardized, evidence-based practice because of the use of the RPHCM. The mechanisms that led to the use of the manuals include acceptance of the worth of the protocols to their clinical practice, reliance on manual content to guide their practice, the perception of credibility, the applicability of RPHCM content to the context, and a fear of the consequences of not using the RPHCMs. Some remote health practitioners are less inclined to use the RPHCM regularly because of a perception that the content is less suited to their needs and daily practice or it is hard to navigate or understand.

CONCLUSION

The evaluation concluded that there is work to be done to widen the RPHCM user base, and organizations need to increase support for their staff to use the RPHCM protocols better. These measures are expected to enable standardized clinical practice in the remote context.

Evaluation of a health system strengthening initiative in the Zambian prison system

Introduction

In 2013, the Zambian Correctional Service (ZCS) partnered with the Centre for Infectious Disease Research in Zambia on the Zambian Prisons Health System Strengthening project, seeking to tackle structural, organisational and cultural weaknesses within the prison health system. We present findings from a nested evaluation of the project impact on high, mid-level and facility-level health governance and health service arrangements in the Zambian Correctional Service.

Methods

Mixed methods were used, including document review, indepth interviews with ministry (11) and prison facility (6) officials, focus group discussions (12) with male and female inmates in six of the eleven intervention prisons, and participant observation during project workshops and meetings. Ethical clearance and verbal informed consent were obtained for all activities. Analysis incorporated deductive and iterative inductive coding.

Results

Outcomes: Improved knowledge of the prison health system strengthened political and bureaucratic will to materially address prison health needs. This found expression in a tripartite Memorandum of Understanding between the Ministry of Home Affairs, Ministry of Health (MOH) and Ministry of Community Development, and in the appointment of a permanent liaison between MOH and ZCS. Capacity-building workshops for ZCS Command resulted in strengthened health planning and management outcomes, including doubling ZCS health professional workforce (from 37 to78 between 2014 and 2016), new preservice basic health training for incoming ZCS officers and formation of facility-based prison health committees with a mandate for health promotion and protection. Mechanisms: continuous and facilitated communication among major stakeholders and the emergence of interorganisational trust were critical. Enabling contextual factors included a permissive political environment, a shift within ZCS from a ‘punitive’ to ‘correctional’ organisational culture, and prevailing political and public health concerns about the spread of HIV and tuberculosis.

Conclusion

While not a panacea, findings demonstrate that a ‘systems’ approach to seemingly intractable prison health system problems yielded a number of short-term tactical and long-term strategic improvements in the Zambian setting. Context-sensitive application of such an approach to other settings may yield positive outcomes.

Strengthening community mental health competence-A realist informed case study from Dehradun, North India

Few accounts exist of programmes in low- and middle-income countries seeking to strengthen community knowledge and skills in mental health. This case study uses a realist lens to explore how a mental health project in a context with few mental health services, strengthened community mental health competence by increasing community knowledge, creating safer social spaces and engaging partnerships for action. We used predominantly qualitative methods to explore relationships between context, interventions, mechanisms and outcomes in the "natural setting" of a community-based mental health project in Dehradun district, Uttarakhand, North India. Qualitative data came from focus group discussions, participant observation and document reviews of community teams' monthly reports on changes in behaviour, attitudes and relationships among stakeholder groups. Data analysis initially involved thematic analysis of three domains: knowledge, safe social spaces and partnerships for action. By exploring patterns within the identified themes for each domain, we were able to infer the mechanisms and contextual elements contributing to observed outcomes. Community knowledge was effectively increased by allowing communities to absorb new understanding into pre-existing social and cultural constructs. Non-hierarchical informal community conversations allowed "organic" integration of unfamiliar biomedical knowledge into local explanatory frameworks. People with psycho-social disability and caregivers found increased social support and inclusion by participating in groups. Building skills in respectful communication through role plays and reflexive discussion increased the receptivity of social environments to people with psycho-social disabilities participation, thereby creating safe social spaces. Facilitating social networks through groups increases women's capacity for collective action to promote mental health. In summary, locally appropriate methods contribute most to learning, stigma reduction and help-seeking. The complex social change progress was patchy and often slow. This study demonstrates a participatory, iterative, reflexive project design which is generating evidence indicating substantial improvements in community mental health competence.

Policymakers' experience of a capacity-building intervention designed to increase their use of research: a realist process evaluation

BACKGROUND

An intervention's success depends on how participants interact with it in local settings. Process evaluation examines these interactions, indicating why an intervention was or was not effective, and how it (and similar interventions) can be improved for better contextual fit. This is particularly important for innovative trials like Supporting Policy In health with Research: an Intervention Trial (SPIRIT), where causal mechanisms are poorly understood. SPIRIT was testing a multi-component intervention designed to increase the capacity of health policymakers to use research.

METHODS

Our mixed-methods process evaluation sought to explain variation in observed process effects across the six agencies that participated in SPIRIT. Data collection included observations of intervention workshops (n = 59), purposively sampled interviews (n = 76) and participant feedback forms (n = 553). Using a realist approach, data was coded for context-mechanism-process effect configurations (retroductive analysis) by two authors.

RESULTS

Intervention workshops were very well received. There was greater variation of views regarding other aspects of SPIRIT such as data collection, communication and the intervention's overall value. We identified nine inter-related mechanisms that were crucial for engaging participants in these policy settings: (1) Accepting the premise (agreeing with the study's assumptions); (2) Self-determination (participative choice); (3) The Value Proposition (seeing potential gain); (4) 'Getting good stuff' (identifying useful ideas, resources or connections); (5) Self-efficacy (believing 'we can do this!'); (6) Respect (feeling that SPIRIT understands and values one's work); (7) Confidence (believing in the study's integrity and validity); (8) Persuasive leadership (authentic and compelling advocacy from leaders); and (9) Strategic insider facilitation (local translation and mediation). These findings were used to develop tentative explanatory propositions and to revise the programme theory.

CONCLUSION

This paper describes how SPIRIT functioned in six policy agencies, including why strategies that worked well in one site were less effective in others. Findings indicate a complex interaction between participants' perception of the intervention, shifting contextual factors, and the form that the intervention took in each site. Our propositions provide transferable lessons about contextualised areas of strength and weakness that may be useful in the development and implementation of similar studies.

Optimal NHS service delivery to care homes: a realist evaluation of the features and mechanisms that support effective working for the continuing care of older people in residential settings

Background

Care homes are the institutional providers of long-term care for older people. The OPTIMAL study argued that it is probable that there are key activities within different models of health-care provision that are important for residents’ health care.

Objectives

To understand ‘what works, for whom, why and in what circumstances?’. Study questions focused on how different mechanisms within the various models of service delivery act as the ‘active ingredients’ associated with positive health-related outcomes for care home residents.

Methods

Using realist methods we focused on five outcomes: (1) medication use and review; (2) use of out-of-hours services; (3) hospital admissions, including emergency department attendances and length of hospital stay; (4) resource use; and (5) user satisfaction. Phase 1: interviewed stakeholders and reviewed the evidence to develop an explanatory theory of what supported good health-care provision for further testing in phase 2. Phase 2 developed a minimum data set of resident characteristics and tracked their care for 12 months. We also interviewed residents, family and staff receiving and providing health care to residents. The 12 study care homes were located on the south coast, the Midlands and the east of England. Health-care provision to care homes was distinctive in each site.

Findings

Phase 1 found that health-care provision to care homes is reactive and inequitable. The realist review argued that incentives or sanctions, agreed protocols, clinical expertise and structured approaches to assessment and care planning could support improved health-related outcomes; however, to achieve change NHS professionals and care home staff needed to work together from the outset to identify, co-design and implement agreed approaches to health care. Phase 2 tested this further and found that, although there were few differences between the sites in residents’ use of resources, the differences in service integration between the NHS and care homes did reflect how these institutions approached activities that supported relational working. Key to this was how much time NHS staff and care home staff had had to learn how to work together and if the work was seen as legitimate, requiring ongoing investment by commissioners and engagement from practitioners. Residents appreciated the general practitioner (GP) input and, when supported by other care home-specific NHS services, GPs reported that it was sustainable and valued work. Access to dementia expertise, ongoing training and support was essential to ensure that both NHS and care home staff were equipped to provide appropriate care.

Limitations

Findings were constrained by the numbers of residents recruited and retained in phase 2 for the 12 months of data collection.

Conclusions

NHS services work well with care homes when payments and role specification endorse the importance of this work at an institutional level as well as with individual residents. GP involvement is important but needs additional support from other services to be sustainable. A focus on strategies that promote co-design-based approaches between the NHS and care homes has the potential to improve residents’ access to and experience of health care.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Combining Theory-Driven Evaluation and Causal Loop Diagramming for Opening the 'Black Box' of an Intervention in the Health Sector: A Case of Performance-Based Financing in Western Uganda

Increased attention on "complexity" in health systems evaluation has resulted in many different methodological responses. Theory-driven evaluations and systems thinking are two such responses that aim for better understanding of the mechanisms underlying given outcomes. Here, we studied the implementation of a performance-based financing intervention by the Belgian Technical Cooperation in Western Uganda to illustrate a methodological strategy of combining these two approaches. We utilized a systems dynamics tool called causal loop diagramming (CLD) to generate hypotheses feeding into a theory-driven evaluation. Semi-structured interviews were conducted with 30 health workers from two districts (Kasese and Kyenjojo) and with 16 key informants. After CLD, we identified three relevant hypotheses: "success to the successful", "growth and underinvestment", and "supervision conundrum". The first hypothesis leads to increasing improvements in performance, as better performance leads to more incentives, which in turn leads to better performance. The latter two hypotheses point to potential bottlenecks. Thus, the proposed methodological strategy was a useful tool for identifying hypotheses that can inform a theory-driven evaluation. The hypotheses are represented in a comprehensible way while highlighting the underlying assumptions, and are more easily falsifiable than hypotheses identified without using CLD.

The influence of contextual factors on healthcare quality improvement initiatives: what works, for whom and in what setting? Protocol for a realist review

BACKGROUND

Context shapes the effectiveness of knowledge implementation and influences health improvement. Successful healthcare quality improvement (QI) initiatives frequently fail to transfer to different settings, with local contextual factors often cited as the cause. Understanding and overcoming contextual barriers is therefore crucial to implementing effective improvement; yet context is still poorly understood. There is a paucity of information on the mechanisms underlying how and why QI projects succeed or fail in given settings. A realist review of empirical studies of healthcare QI initiatives will be undertaken to examine the influence and impact of contextual factors on quality improvement in healthcare settings and explore whether QI initiatives can work in all contexts.

METHODS

The review will explore which contextual factors are important, and how, why, when and for whom they are important, within varied settings. The dynamic nature of context and change over time will be explored by examining which aspects of context impact at key points in the improvement trajectory. The review will also consider the influence of context on improvement outcomes (provider- and patient-level), spread and sustainability. The review process will follow five iterative steps: (1) clarify scope, (2) search for evidence, (3) appraise primary studies and extract data, (4) synthesise evidence and draw conclusions and (5) disseminate findings. The reviewers will consult with experts and stakeholders in the early stages to focus the review and develop a programme theory consisting of explanatory 'context-mechanism-outcome' configurations. Searches for primary evidence will be conducted iteratively. Data will be extracted and tested against the programme theory. A review advisory group will oversee the review process. Review findings will follow RAMESES guidelines and will be disseminated via a report, presentations and peer-reviewed publications.

DISCUSSION

The review will update and consolidate evidence on the contextual conditions for effective improvement and distil new knowledge to inform the design and development of context-sensitive QI initiatives. This review ties in with the study of improvement programmes as vehicles of change and the development of an evidence base around healthcare improvement by addressing whether QI initiatives can work in all contexts.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42017062135.

Challenge Demcare: management of challenging behaviour in dementia at home and in care homes – development, evaluation and implementation of an online individualised intervention for care homes; and a cohort study of specialist community mental health care for families

Background

Dementia with challenging behaviour (CB) causes significant distress for caregivers and the person with dementia. It is associated with breakdown of care at home and disruption in care homes. Challenge Demcare aimed to assist care home staff and mental health practitioners who support families at home to respond effectively to CB.

Objectives

To study the management of CB in care homes (ResCare) and in family care (FamCare). Following a conceptual overview, two systematic reviews and scrutiny of clinical guidelines, we (1) developed and tested a computerised intervention; (2) conducted a cluster randomised trial (CRT) of the intervention for dementia with CB in care homes; (3) conducted a process evaluation of implementation of the intervention; and (4) conducted a longitudinal observational cohort study of the management of people with dementia with CB living at home, and their carers.

Review methods

Cochrane review of randomised controlled trials; systematic meta-ethnographic review of quantitative and qualitative studies.

Design

ResCare – survey, CRT, process evaluation and stakeholder consultations. FamCare – survey, longitudinal cohort study, participatory development design process and stakeholder consultations. Comparative examination of baseline levels of CB in the ResCare trial and the FamCare study participants.

Settings

ResCare – 63 care homes in Yorkshire. FamCare – 33 community mental health teams for older people (CMHTsOP) in seven NHS organisations across England.

Participants

ResCare – 2386 residents and 861 staff screened for eligibility; 555 residents with dementia and CB; 277 ‘other’ residents; 632 care staff; and 92 staff champions. FamCare – every new referral (n = 5360) reviewed for eligibility; 157 patients with dementia and CB, with their carer; and 26 mental health practitioners. Stakeholder consultations – initial workshops with 83 practitioners and managers from participating organisations; and 70 additional stakeholders using eight group discussions and nine individual interviews.

Intervention

An online application for case-specific action plans to reduce CB in dementia, consisting of e-learning and bespoke decision support care home and family care e-tools.

Main outcome measures

ResCare – survey with the Challenging Behaviour Scale; measurement of CB with the Neuropsychiatric Inventory (NPI) and medications taken from prescriptions; implementation with thematic views from participants and stakeholders. FamCare – case identification from all referrals to CMHTsOP; measurement of CB with the Revised Memory and Behaviour Problems Checklist and NPI; medications taken from prescriptions; and thematic views from stakeholders. Costs of care calculated for both settings. Comparison of the ResCare trial and FamCare study participants used the NPI, Clinical Dementia Rating and prescribed medications.

Results

ResCare – training with group discussion and decision support for individualised interventions did not change practice enough to have an impact on CB in dementia. Worksite e-learning opportunities were not readily taken up by care home staff. Smaller homes with a less hierarchical management appear more ready than others to engage in innovation. FamCare – home-dwelling people with dementia and CB are referred to specialist NHS services, but treatment over 6 months, averaging nine contacts per family, had no overall impact on CB. Over 60% of people with CB had mild dementia. Families bear the majority of the care costs of dementia with CB. A care gap in the delivery of post-diagnostic help for families supporting relatives with dementia and significant CB at home has emerged. Higher levels of CB were recorded in family settings; and prescribing practices were suboptimal in both care home and family settings.

Limitations

Functionality of the software was unreliable, resulting in delays. This compromised the feasibility studies and undermined delivery of the intervention in care homes. A planned FamCare CRT could not proceed because of insufficient referrals.

Conclusions

A Cochrane review of individualised functional analysis-based interventions suggests that these show promise, although delivery requires a trained dementia care workforce. Like many staff training interventions, our interactive e-learning course was well received by staff when delivered in groups with facilitated discussion. Our e-learning and decision support e-tool intervention in care homes, in its current form, without ongoing review of implementation of recommended action plans, is not effective at reducing CB when compared with usual care. This may also be true for staff training in general. A shift in priorities from early diagnosis to early recognition of dementia with clinically significant CB could bridge the emerging gap and inequities of care to families. Formalised service improvements in the NHS, to co-ordinate such interventions, may stimulate better opportunities for practice models and pathways. Separate services for care homes and family care may enhance the efficiency of delivery and the quality of research on implementation into routine care.

Future work

There is scope for extending functional analysis-based interventions with communication and interaction training for carers. Our clinical workbooks, video material of real-life episodes of CB and process evaluation tool resources require further testing. There is an urgent need for evaluation of interventions for home-dwelling people with dementia with clinically significant CB, delivered by trained dementia practitioners. Realist evaluation designs may illuminate how the intervention might work, and for whom, within varying service contexts.

Trial registration

Current Controlled Trials ISRCTN02553381 (the ResCare trial) and ISRCTN58876649 (the FamCare study).

Funding

This project was funded by the National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 5, No. 15. See the NIHR Journals Library website for further project information.

Exploring the Functioning of Decision Space: A Review of the Available Health Systems Literature

BACKGROUND

The concept of decision space holds appeal as an approach to disaggregating the elements that may influence decision-making in decentralized systems. This narrative review aims to explore the functioning of decision space and the factors that influence decision space.

METHODS

A narrative review of the literature was conducted with searches of online databases and academic journals including PubMed Central, Emerald, Wiley, Science Direct, JSTOR, and Sage. The articles were included in the review based on the criteria that they provided insight into the functioning of decision space either through the explicit application of or reference to decision space, or implicitly through discussion of decision-making related to organizational capacity or accountability mechanisms.

RESULTS

The articles included in the review encompass literature related to decentralisation, management and decision space. The majority of the studies utilise qualitative methodologies to assess accountability mechanisms, organisational capacities such as finance, human resources and management, and the extent of decision space. Of the 138 articles retrieved, 76 articles were included in the final review.

CONCLUSION

The literature supports Bossert's conceptualization of decision space as being related to organizational capacities and accountability mechanisms. These functions influence the decision space available within decentralized systems. The exact relationship between decision space and financial and human resource capacities needs to be explored in greater detail to determine the potential influence on system functioning.

Understanding and measuring quality of care: dealing with complexity

Existing definitions and measurement approaches of quality of health care often fail to address the complexities involved in understanding quality of care. It is perceptions of quality, rather than clinical indicators of quality, that drive service utilization and are essential to increasing demand. Here we reflect on the nature of quality, how perceptions of quality influence health systems and what such perceptions indicate about measurement of quality within health systems. We discuss six specific challenges related to the conceptualization and measurement of the quality of care: perceived quality as a driver of service utilization; quality as a concept shaped over time through experience; responsiveness as a key attribute of quality; the role of management and other so-called upstream factors; quality as a social construct co-produced by families, individuals, networks and providers; and the implications of our observations for measurement. Within the communities and societies where care is provided, quality of care cannot be understood outside social norms, relationships, trust and values. We need to improve not only technical quality but also acceptability, responsiveness and levels of patient-provider trust. Measurement approaches need to be reconsidered. An improved understanding of all the attributes of quality in health systems and their interrelationships could support the expansion of access to essential health interventions.

Understanding the factors that make public participation effective in health policy and planning: a realist synthesis

Although researchers argue for the importance of involving the public in developing health policy, there has been little focus on central research questions – such as what techniques of public participation work, in what circumstances, and why. This paper presents a realist synthesis which identifies and explains the underlying mechanisms and specific contextual factors that lead to effective public participation in health policy and planning. Peer-reviewed, English language literature was searched, which resulted in 77 articles for review and synthesis. This synthesis uncovered the underlying mechanism of ‘political commitment’ that generates public participation effectiveness. The other three possible underlying mechanisms, namely: ‘partnership synergy’, ‘inclusiveness’ and ‘deliberativeness’, were found to potentially provide further explanation on public participation effectiveness for health policy and planning. The findings of this review provide evidence that can be useful to health practitioners and decision-makers to actively involve the public when drafting public health policies and programs and, more importantly, guide them in deciding which strategies to best employ for which contexts.

Factors associated with the failure of obstetric fistula repair in Guinea: implications for practice

Background

The prevention and treatment of obstetric fistula still remains a concern and a challenge in low income countries.

The objective of this study was to estimate the overall proportions of failure of fistula closure and incontinence among women undergoing repair for obstetric fistula in Guinea and identify its associated factors.

Methods

This was a retrospective cohort study using data extracted from medical records of fistula repairs between 1 January 2012 and 30 September 2013. The outcome was the failure of fistula closure and incontinence at hospital discharge evaluated by a dye test. A sub-sample of women with vesicovaginal fistula was used to identify the factors associated with these outcomes.

Results

Overall, 109 women out of 754 (14.5 %; 95 % CI:11.9–17.0) unsuccessful repaired fistula at discharge and 132 (17.5 %; 95 % CI:14.8–20.2) were not continent.

Failure of fistula closure was associated with vaginal delivery (AOR: 1.9; 95 % CI: 1.0–3.6), partially (AOR: 2.0; 95 % CI: 1.1–5.6) or totally damaged urethra (AOR: 5.9; 95 % CI: 2.9–12.3) and surgical repair at Jean Paul II Hospital (AOR: 2.5; 95 % CI: 1.2–4.9).

Women who had a partially damaged urethra (AOR: 2.5; 95 % CI: 1.5–4.4) or a totally damaged urethra (AOR: 6.3; 95 % CI: 3.0–13.0) were more likely to experience post-repair urinary incontinence than women who had their urethra intact.

Conclusion

At programmatic level in Guinea, caution should be paid to the repair of women who present with a damaged urethra and those who delivered vaginally as they carry greater risks of experiencing a failure of fistula closure and incontinence.