"I wanted a skeleton … they brought a prince": A qualitative investigation of factors mediating the implementation of a Performance Based Incentive program in Malawi

Implementation fidelity of intravenous ferric carboxymaltose administration for iron deficiency anaemia in pregnancy: a mixed-methods study nested in a clinical trial in Nigeria

BACKGROUND

Iron deficiency anaemia is common among pregnant women in Nigeria. The standard treatment is oral iron therapy, which can be sub-optimal due to side effects. Intravenous ferric carboxymaltose (FCM) is an evidenced-based alternative treatment with a more favourable side effect profile requiring administration according to a standardized protocol. In this study, we assessed the fidelity of administering a single dose of FCM according to protocol and identified factors influencing implementation fidelity.

METHODS

We used a mixed-method approach with a sequential explanatory design nested in a clinical trial across 11 facilities in Lagos and Kano States, Nigeria. Guided by a conceptual framework of implementation fidelity, we quantitatively assessed adherence to protocol by directly observing every alternate FCM administration, using an intervention procedure checklist, and compared median adherence by facility and state. Qualitative fidelity assessment was conducted via in-depth interviews with 14 skilled health personnel (SHP) from nine purposively selected health facilities, using a semi-structured interview guide. We analyzed quantitative data using descriptive and inferential statistics in Stata and used thematic analysis to analyze the transcribed interviews in NVivo.

RESULTS

A total of 254 FCM administrations were observed across the 11 study sites, with the majority in secondary (63%), followed by primary healthcare facilities (PHCs) (30%). Overall, adherence to FCM administration as per protocol was moderate (63%) and varied depending on facility level. The lowest level of adherence was observed in PHCs (36%). Median, adherence level showed significant differences by facility level (p = 0.001) but not by state (p = 0.889). Teamwork and availability of protocols are facilitation strategies that contributed to high fidelity. However, institutional/ logistical barriers are contextual factors that influenced the varied fidelity levels observed in some facilities.

CONCLUSIONS

Collaborative teams and access to operating protocols resulted in high fidelity in some facilities. However, in some PHCs, fidelity to FCM was low due to contextual factors and intervention complexities, thereby influencing the quality of delivery. In Nigeria, scale-up of FCM will require attention to staff strength, teamwork and availability of administration protocols, in order to optimize its impact on anaemia in pregnancy.

Factors Affecting the Successful Implementation of a Digital Intervention for Health Financing in a Low-Resource Setting at Scale: Semistructured Interview Study With Health Care Workers and Management Staff

BACKGROUND

Digital interventions for health financing, if implemented at scale, have the potential to improve health system performance by reducing transaction costs and improving data-driven decision-making. However, many interventions never reach sustainability, and evidence on success factors for scale is scarce. The Insurance Management Information System (IMIS) is a digital intervention for health financing, designed to manage an insurance scheme and already implemented on a national scale in Tanzania. A previous study found that the IMIS claim function was poorly adopted by health care workers (HCWs), questioning its potential to enable strategic purchasing and succeed at scale.

OBJECTIVE

This study aimed to understand why the adoption of the IMIS claim function by HCWs remained low in Tanzania and to assess implications for use at scale.

METHODS

We conducted 21 semistructured interviews with HCWs and management staff in 4 districts where IMIS was first implemented. We sampled respondents by using a maximum variation strategy. We used the framework method for data analysis, applying a combination of inductive and deductive coding to organize codes in a socioecological model. Finally, we related emerging themes to a framework for digital health interventions for scale.

RESULTS

Respondents appreciated IMIS's intrinsic software characteristics and technical factors and acknowledged IMIS as a valuable tool to simplify claim management. Human factors, extrinsic ecosystem, and health care ecosystem were considered as barriers to widespread adoption.

CONCLUSIONS

Digital interventions for health financing, such as IMIS, may have the potential for scale if careful consideration is given to the environment in which they are placed. Without a sustainable health financing environment, sufficient infrastructure, and human capacity, they cannot unfold their full potential to improve health financing functions and ultimately contribute to universal health coverage.

Why did performance-based financing in Burkina Faso fail to achieve the intended equity effects? A process tracing study

In recent years, performance-based financing (PBF) has attracted attention as a means of reforming provider payment mechanisms in low- and middle-income countries. Particularly in combination with demand-side interventions, PBF has been assumed to benefit also the most vulnerable and disadvantaged groups. However, impact evaluations have often found this not to be the case. In Burkina Faso, PBF was coupled with specific equity measures to enhance healthcare utilization among the ultra-poor, but failed to produce the expected effects. Our study used the process tracing methodology to unravel the reasons for the lack of impact produced by the equity measures. We relied on published evidence, secondary data analysis, and findings from a qualitative study to support or invalidate the hypothesized causal mechanism, that is the reconstructed theory of change of the equity measures. Our findings show how various contextual, design, and implementation challenges hindered the causal mechanism from unfolding as planned. These included issues with the identification and exemption of the ultra-poor on the demand side, and with financial issues and considerations on the supply side. In broader terms, our findings underline the difficulty in improving access to care for the ultra-poor, given the multifaceted and complex nature of barriers to care the most vulnerable face. From a methodological point of view, our study demonstrates the value and applicability of process tracing in complementing other forms of evaluation for complex interventions in global health.

How much does it cost to combine supply-side and demand-side RBF approaches in a single intervention? Full cost analysis of the Results Based Financing for Maternal and Newborn Health Initiative in Malawi

OBJECTIVE

To estimate the economic cost associated with implementing the Results Based Financing for Maternal and Newborn Health (RBF4MNH) Initiative in Malawi. No specific hypotheses were formulated ex-ante.

SETTING

Primary and secondary delivery facilities in rural Malawi.

PARTICIPANTS

Not applicable. The study relied almost exclusively on secondary financial data.

INTERVENTION

The RBF4MNH Initiative was a results-based financing (RBF) intervention including both a demand and a supply-side component.

PRIMARY AND SECONDARY OUTCOME MEASURES

Cost per potential and for actual beneficiaries.

RESULTS

The overall economic cost of the Initiative during 2011-2016 amounted to €12 786 924, equivalent to €24.17 per pregnant woman residing in the intervention districts. The supply side activity cluster absorbed over 40% of all resources, half of which were spent on infrastructure upgrading and equipment supply, and 10% on incentives. Costs for the demand side activity cluster and for verification were equivalent to 14% and 6%, respectively of the Initiative overall cost.

CONCLUSION

Carefully tracing resource consumption across all activities, our study suggests that the full economic cost of implementing RBF interventions may be higher than what was previously reported in published cost-effectiveness studies. More research is urgently needed to carefully trace the costs of implementing RBF and similar health financing innovations, in order to inform decision-making in low-income and middle-income countries around scaling up RBF approaches.

What theories underpin performance-based financing? A scoping review

PURPOSE

The study aims to explore the theoretical bases justifying the use of performance-based financing (PBF) in the health sector in low- and middle-income countries (LMICs).

DESIGN/METHODOLOGY/APPROACH

The authors conducted a scoping review of the literature on PBF so as to identify the theories utilized to underpin it and analyzed its theoretical justifications.

FINDINGS

Sixty-four studies met the inclusion criteria. Economic theories were predominant, with the principal-agent theory being the most commonly-used theory, explicitly referred to by two-thirds of included studies. Psychological theories were also common, with a wide array of motivation theories. Other disciplines in the form of management or organizational science, political and social science and systems approaches also contributed. However, some of the theories referred to contradicted each other. Many of the studies included only casually alluded to one or more theories, and very few used these theories to justify or support PBF. No theory emerged as a dominant, consistent and credible justification of PBF, perhaps except for the principal-agent theory, which was often inappropriately applied in the included studies, and when it included additional assumptions reflecting the contexts of the health sector in LMICs, might actually warn against adopting PBF.

PRACTICAL IMPLICATIONS

Overall, this review has not been able to identify a comprehensive, credible, consistent, theoretical justification for using PBF rather than alternative approaches to health system reforms and healthcare providers' motivation in LMICs.

ORIGINALITY/VALUE

The theoretical justifications of PBF in the health sector in LMICs are under-documented. This review is the first of this kind and should encourage further debate and theoretical exploration of the justifications of PBF.

A realist review to assess for whom, under what conditions and how pay for performance programmes work in low- and middle-income countries

Pay for performance (P4P) programmes are popular health system-focused interventions aiming to improve health outcomes in low-and middle-income countries (LMICs). This realist review aims to understand how, why and under what circumstance P4P works in LMICs.We systematically searched peer-reviewed and grey literature databases, and examined the mechanisms underpinning P4P effects on: utilisation of services, patient satisfaction, provider productivity and broader health system, and contextual factors moderating these. This evidence was then used to construct a causal loop diagram.We included 112 records (19 grey literature; 93 peer-reviewed articles) assessing P4P schemes in 36 countries. Although we found mixed evidence of P4P's effects on identified outcomes, common pathways to improved outcomes include: community outreach; adherence to clinical guidelines, patient-provider interactions, patient trust, facility improvements, access to drugs and equipment, facility autonomy, and lower user fees. Contextual factors shaping the system response to P4P include: degree of facility autonomy, efficiency of banking, role of user charges in financing public services; staffing levels; staff training and motivation, quality of facility infrastructure and community social norms. Programme design features supporting or impeding health system effects of P4P included: scope of incentivised indicators, fairness and reach of incentives, timely payments and a supportive, robust verification system that does not overburden staff. Facility bonuses are a key element of P4P, but rely on provider autonomy for maximum effect. If health system inputs are vastly underperforming pre-P4P, they are unlikely to improve only due to P4P. This is the first realist review describing how and why P4P initiatives work (or fail) in different LMIC contexts by exploring the underlying mechanisms and contextual and programme design moderators. Future studies should systematically examine health system pathways to outcomes for P4P and other health system strengthening initiatives, and offer more understanding of how programme design shapes mechanisms and effects.

Effect Heterogeneity in Responding to Performance-Based Incentives: A Quasi-Experimental Comparison of Impacts on Health Service Indicators Between Hospitals and Health Centers in Malawi

Heterogeneity of effects produced by performance-based incentives (PBIs) at different levels of care provision is not well understood. This study analyzes effect heterogeneities between different facility types resulting from a PBI program in Malawi. Identical PBIs were applied to both district hospitals and health centers to improve the performance of essential health services provision. We conducted two complementary quasi-experiments comparing all 17 interventions with 17 matched independent control facilities (each 12 health centers, five hospitals). A pre- and post-test design with difference-in-differences analysis was used to estimate effects on 14 binary quality indicators; interrupted time series analysis of monthly routine data was used to estimate effects on 11 continuous quantity indicators. Effects were estimated separately for health centers and hospitals. Most quality indicators performed high at baseline, producing ceiling effects on further measurable improvements. Significant positive effects were observed for stocks of iron supplements (hospitals) and partographs (health centers). Four quantity indicators showed similar positive trend improvements across facility types (first-trimester antenatal visits, voluntary HIV-testing of couples, iron supplementation in pregnancy, vitamin A supplementation of children); two showed no change for either type of facility (skilled birth attendance, fully immunized one-year-olds); five indicators revealed different effect patterns for health centers and hospitals. In both health centers and hospitals, the largely positive PBI effects on antenatal care included resilience against interrupted supply chains and improvements in attendance rates. Observed heterogeneity might have been influenced by the availability of specific resources or the redistribution of service use.

Using implementation science theories and frameworks in global health

In global health, researchers and decision makers, many of whom have medical, epidemiology or biostatistics background, are increasingly interested in evaluating the implementation of health interventions. Implementation science, particularly for the study of public policies, has existed since at least the 1930s. This science makes compelling use of explicit theories and analytic frameworks that ensure research quality and rigour. Our objective is to inform researchers and decision makers who are not familiar with this research branch about these theories and analytic frameworks. We define four models of causation used in implementation science: intervention theory, frameworks, middle-range theory and grand theory. We then explain how scientists apply these models for three main implementation studies: fidelity assessment, process evaluation and complex evaluation. For each study, we provide concrete examples from research in Cuba and Africa to better understand the implementation of health interventions in global health context. Global health researchers and decision makers with a quantitative background will not become implementation scientists after reading this article. However, we believe they will be more aware of the need for rigorous implementation evaluations of global health interventions, alongside impact evaluations, and in collaboration with social scientists.

At what cost is performance-based financing implemented? Novel evidence from Malawi

Our study estimated the full economic cost of implementing performance-based financing [PBF, the Support for Service Delivery Integration Performance-Based Incentives (SSDI-PBI) programme], as a means of first introducing strategic purchasing in a low-income setting, Malawi. Our analysis distinguished design from implementation costs and traces costs across personnel and non-personnel cost categories over the 2012-15 period. The full cost of the SSDI-PBI programme amounted to USD 3 402 187, equivalent to USD 6.46 per targeted beneficiary. The design phase accounted for about one-third (USD 1 161 332) of the total costs, while the incentives (USD 1 140 436) represented about one-third of the total cost of the intervention and about half the cost of the implementation phase. With a cost of USD 1 605 178, personnel costs represented the dominant cost category. Our study indicated that the introduction of PBF entailed consumption of a substantial amount of resources, hence representing an important opportunity cost for the health system.

To the bone: Comment on "I wanted a skeleton … they brought a prince": A qualitative investigation of factors mediating the implementation of a Performance Based Incentive program in Malawi

Recently, McMahon and colleagues set out to build on a widely-used fidelity framework, assessing the role of moderating factors during the implementation of performance-based financing programs in Malawi. Their attempt draws again the attention to the importance of approaching real word implementation issues from a theoretical perspective. It also highlights the importance of fidelity assessment within process evaluation of health programs. In this comment we argue that theoretical developments in the field of implementation science in global health would benefit from an accurate understanding of existing conceptual frameworks as well as from taking into account all contemporary contributions.