Design of an impact evaluation using a mixed methods model--an explanatory assessment of the effects of results-based financing mechanisms on maternal healthcare services in Malawi

Good while it lasted? Estimating the long-term and withdrawal effects of results-based financing in Malawi on maternal care utilisation using routine data

OBJECTIVES

This study aimed to evaluate the effect of introduction and subsequent withdrawal of the Results-based Financing for Maternal and Newborn Health Initiative (RBF4MNH) in Malawi on utilisation of facility-based childbirths, antenatal care (ANC) and postnatal care (PNC).

DESIGN

A controlled interrupted time series design was used with secondary data from the Malawian Health Management Information System.

SETTING

Healthcare facilities at all levels identified as providing maternity services in four intervention districts and 20 non-intervention districts in Malawi.

PARTICIPANTS

Routinely collected, secondary data of total monthly service utilisation of facility-based childbirths, ANC and PNC services.

INTERVENTIONS

The intervention is the RBF4MNH initiative, introduced by the Malawian government in 2013 to improve maternal and infant health outcomes and withdrawn in 2018 after ceasing of donor funding.

OUTCOME MEASURES

Differences in total volume and trends of utilisation of facility-based childbirths, ANC and PNC services, compared between intervention versus non-intervention districts, for the study period of 90 consecutive months.

RESULTS

No significant effect was observed, on utilisation trends for any of the three services during the first 2.5 years of intervention. In the following 2.5 years after full implementation, we observed a small positive increase for facility-based childbirths (+0.62 childbirths/month/facility) and decrease for PNC (-0.55 consultations/month/facility) trends of utilisation respectively. After withdrawal, facility-based childbirths and ANC consultations dropped both in immediate volume after removal (-10.84 childbirths/facility and -20.66 consultations/facility, respectively), and in trends of utilisation over time (-0.27 childbirths/month/facility and -1.38 consultations/month/facility, respectively). PNC utilisation levels seemed unaffected in intervention districts against a decline in the rest of the country.

CONCLUSIONS

Concurrent with wider literature, our results suggest that effects of complex health financing interventions, such as RBF4MNH, can take a long time to be seen. They might not be sustained beyond the implementation period if measures are not adopted to reform existing health financing structures.

Perceptions of the benefits of the basic medical insurance system among the insured: a mixed methods research of a northern city in China

Background

The perceptions of the benefits of the basic medical insurance system among the insured not only reflect the system's performance but also the public's basic medical insurance policy literacy, valuable information for countries that have entered the stage of deepening reform. This study aims to examine the factors that affect the perceptions of the benefits of the basic medical insurance system in China, diagnose the key problems, and propose corresponding measures for improvement.

Methods

A mixed method design was used. Data for the quantitative study were obtained from a cross-sectional questionnaire survey (n = 1,045) of residents of Harbin who had enrolled for basic medical insurance system. A quota sampling method was further adopted. A multivariate logistic regression model was then employed to identify the factors influencing the perceptions of the benefits of the basic medical insurance system, followed by semi-structured interviews with 30 conveniently selected key informants. Interpretative phenomenological analysis was used to analyze the interview data.

Results

Approximately 44% of insured persons reported low perceptions of benefits. The logistic regression model showed that low perceptions of the benefits of the basic medical insurance system was positively correlated with the experience of daily drug purchases (OR = 1.967), perceptions of recognition with basic medical insurance system (OR = 1.948), perceptions of the financial burden of participation costs (OR = 1.887), perceptions of the convenience of using basic medical insurance for medical treatment (OR = 1.770), perceptions of the financial burden of daily drug purchases costs (OR = 1.721), perceptions of the financial burden of hospitalization costs (OR = 1.570), and type of basic medical insurance system (OR = 1.456). The results of the qualitative analysis showed that the key problem areas of perceptions of the benefits of the basic medical insurance system were: (I) system design of basic medical insurance; (II) intuitive cognition of the insured; (III) rational cognition of the insured; and (IV) the system environment.

Conclusions

Improving the perceptions of the benefits of the basic medical insurance system of the insured requires joint efforts in improving system design and implementation, exploring effective publicity methods of basic medical insurance system information, supporting public policy literacy, and promoting the health system environment.

Mixed methods study design, pre-analysis plan, process evaluation and baseline results of trailbridges in rural Rwanda

We present a study design, pre-analysis plan, process evaluation and baseline results designed to establish the impact of trailbridges on health, education, agricultural and economic outcomes of households in rural Rwanda. This intervention and study is being implemented in communities that face barriers to socioeconomic development through periodic isolation caused by flooding. We describe a mixed methods approach to measure the impacts of these trailbridges on outcomes at the village level. The study is anchored on a stepped-wedge randomized controlled trial (RCT) implemented in 147 sites: 97 phased-in intervention sites and 50 long-term control sites. These sites are being monitored in four annual waves comprising of a baseline period and three subsequent follow-up waves. We will supplement the RCT with three sub-studies. First, we are investigating the role of weather events and streamflow variability on temporal and spatial bridge use patterns among intervention sites. We will then find the relationship between the weather events, streamflow and bridge use from motion-activated cameras installed in intervention sites. Secondly, we are following 42 markets serving study sites to investigate the impact of the trailbridges on the market prices of key goods including crops, livestock and agricultural inputs. Lastly, we are following 30 villages that are more distant from the river crossings to determine the spatial extent of the trailbridge impacts. Our study will advance knowledge by generating new data on the impact of rural infrastructure and providing the opportunity to explore a range of outcomes for future evaluation of infrastructure in low- and middle-income countries. We will enable an outcomes-based funding model that ties implementer payments to demonstrated positive impacts of these trailbridges. Furthermore, we will identify cost-effective, easily assessed measures that are highly correlated to the economic and health benefits of the intervention. These measures may then be used by a portfolio of interventions across multiple geographies without always requiring complex trials.

Implementation of a performance-based financing scheme in Malawi and resulting externalities on the quality of care of non-incentivized services

Background

Countries in Africa progressively implement performance-based financing schemes to improve the quality of care provided by maternal, newborn and child health services. Beyond its direct effects on service provision, evidence suggests that performance-based financing can also generate positive externalities on service utilization, such as increased use of those services that reached higher quality standards after effective scheme implementation. Little, however, is known about externalities generated within non-incentivized health services, such as positive or negative effects on the quality of services within the continuum of maternal care.

Methods

We explored whether a performance-based financing scheme in Malawi designed to improve the quality of childbirth service provision resulted positive or negative externalities on the quality of non-targeted antenatal care provision. This non-randomized controlled pre-post-test study followed the phased enrolment of facilities into a performance-based financing scheme across four districts over a two-year period. Effects of the scheme were assessed by various composite scores measuring facilities’ readiness to provide quality antenatal care, as well as the quality of screening, prevention, and education processes offered during observed antenatal care consultations.

Results

Our study did not identify any statistically significant effects on the quality of ANC provision attributable to the implemented performance-based financing scheme. Our findings therefore suggest not only the absence of positive externalities, but also the absence of any negative externalities generated within antenatal care service provision as a result of the scheme implementation in Malawi.

Conclusions

Prior research has shown that the Malawian performance-based financing scheme was sufficiently effective to improve the quality of incentivized childbirth service provision. Our findings further indicate that scheme implementation did not affect the quality of non-incentivized but clinically related antenatal care services. While no positive externalities could be identified, we also did not observe any negative externalities attributable to the scheme’s implementation. While performance-based incentives might be successful in improving targeted health care processes, they have limited potential in producing externalities – neither positive nor negative – on the provision quality of related non-incentivized services.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12884-021-03880-9.

Effects of combined decision-support and performance-based incentives on reported client satisfaction with maternal health services in primary facilities: A quasi-experimental study in the Upper East Region of Ghana

BACKGROUND

Computerized decision-support systems (CDSS) and performance-based incentives (PBIs) have potential to contribute to client satisfaction with health services. However, rigorous evidence is lacking on the effectiveness of these strategies in lower-income countries such as Ghana. This study aimed to determine the effect of a combined CDSS-PBI intervention on client satisfaction with maternal health services in primary facilities in the Upper East Region of Ghana.

METHODS

We employed a quasi-experimental controlled baseline and endline design to assess the effect of the combined interventions on client satisfaction with maternal health services, measured by quantitative pre/post-test client satisfaction survey. Our analysis used difference-in-difference logistic regression, controlling for potential covariates, to compare variables across intervention and comparison facilities at baseline and endline.

RESULTS

The combined CDSS-PBI intervention was associated with increased or unchanged client satisfaction with all maternal health services compared at endline. Antenatal client difference-in-difference of mean satisfaction scores were significant at endline for intervention (n = 378) and comparison (n = 362) healthcare facilities for overall satisfaction (DiD 0.058, p = 0.014), perception of providers' technical performance (DiD = 0.142; p = 0.006), client-provider interaction (DiD = 0.152; p = 0.001), and provider availability (DiD = 0.173; p = 0.001). Delivery client difference-in-difference of satisfaction scores were significant at endline for intervention (n = 318) and comparison (n = 240) healthcare facilities for overall satisfaction with delivery services (DiD = 0.072; p = 0.02) and client-provider interaction (DiD = 0.146; p = 0.02). However, mean overall satisfaction actually reduced slightly in intervention facilities, while DiD for technical performance and provider availability were not significant.

CONCLUSION

This combined CDSS-PBI intervention was associated with greater antenatal and delivery client satisfaction with some aspects of maternity services within two years of implementation. It could be expanded elsewhere if funds allow, though further research is still required to assess cost-effectiveness and long-term effects on client satisfaction and maternal health outcomes.

Exploiting the emergent nature of mixed methods designs: insights from a mixed methods impact evaluation in Malawi

The application of mixed methods in Health Policy and Systems Research (HPSR) has expanded remarkably. Nevertheless, a recent review has highlighted how many mixed methods studies do not conceptualize the quantitative and the qualitative component as part of a single research effort, failing to make use of integrated approaches to data collection and analysis. More specifically, current mixed methods studies rarely rely on emergent designs as a specific feature of this methodological approach. In our work, we postulate that explicitly acknowledging the emergent nature of mixed methods research by building on a continuous exchange between quantitative and qualitative strains of data collection and analysis leads to a richer and more informative application in the field of HPSR. We illustrate our point by reflecting on our own experience conducting the mixed methods impact evaluation of a complex health system intervention in Malawi, the Results Based Financing for Maternal and Newborn Health Initiative. We describe how in the light of a contradiction between the initial set of quantitative and qualitative findings, we modified our design multiple times to include additional sources of quantitative and qualitative data and analytical approaches. To find an answer to the initial riddle, we made use of household survey data, routine health facility data, and multiple rounds of interviews with both healthcare workers and service users. We highlight what contextual factors made it possible for us to maintain the high level of methodological flexibility that ultimately allowed us to solve the riddle. This process of constant reiteration between quantitative and qualitative data allowed us to provide policymakers with a more credible and comprehensive picture of what dynamics the intervention had triggered and with what effects, in a way that we would have never been able to do had we kept faithful to our original mixed methods design.

Mixed methods research on satisfaction with basic medical insurance for urban and rural residents in China

Background

There have been few studies on satisfaction with integrated basic medical insurance for urban and rural residents (URRBMI), and satisfaction with URRBMI is not very high because of the complexity of its policies and differences among the insured. The aim of the present study was to explore the factors that influence satisfaction with URRBMI in China and to provide scientific suggestions to the government for how to effectively manage and improve the policy.

Methods

An explanatory sequential design of mixed methods research was used. A quantitative research using a three-stage stratified cluster sampling method was used to randomly select the guardians of pupils who participated in URRBMI (n = 1335). The quantitative research was conducted to calculate the latent variables’ scores and path coefficients between latent variables using SmartPLS3.0. With public trust, public satisfaction, and perceived quality as the target variables, important-performance analysis (IPA) was used to explore the important but underperforming factors, which were the key elements to improving satisfaction with URRBMI. A purposeful sampling strategy according to satisfaction level was used to obtain qualitative research subjects from among the quantitative research subjects. A qualitative research was conducted using semi-structured interviews, and the thematic analysis method was used to summarize the interview data.

Results

The three strongest paths were perceived quality to public satisfaction, with a total effect of 0.737 (t = 41.270, P < 0.001); perceived quality to perceived value, with a total effect of 0.676 (t = 31.964, P < 0.001); and public satisfaction to public trust, with a total effect of 0.634 (t = 31.305, P < 0.001). IPA revealed that public satisfaction and perceived quality were key factors for public trust and that perceived quality was of high importance for public satisfaction but had low performance. The policy quality was a determining factor for perceived quality. The qualitative research results showed that the most unsatisfactory aspect for the insured was the policy quality.

Conclusions

This study found that improving quality is key to improving public satisfaction with and public trust in URRBMI. The government should improve the compensation level by broadening the channel of financing for the URRBMI fund, rationally formulating reimbursement standards, and broadening the scope of the drug catalog and the medical treatment projects. The government should establish a stable financing growth mechanism and effective methods of providing health education to improve public satisfaction and public trust.

The cost-effectiveness of using results-based financing to reduce maternal and perinatal mortality in Malawi

INTRODUCTION

Results-based financing (RBF) is being promoted to increase coverage and quality of maternal and perinatal healthcare in sub-Saharan Africa (SSA) countries. Evidence on the cost-effectiveness of RBF is limited. We assessed the cost-effectiveness within the context of an RBF intervention, including performance-based financing and conditional cash transfers, in rural Malawi.

METHODS

We used a decision tree model to estimate expected costs and effects of RBF compared with status quo care during single pregnancy episodes. RBF effects on maternal case fatality rates were modelled based on data from a maternal and perinatal programme evaluation in Zambia and Uganda. We obtained complementary epidemiological information from the published literature. Service utilisation rates for normal and complicated deliveries and associated costs of care were based on the RBF intervention in Malawi. Costs were estimated from a societal perspective. We estimated incremental cost-effectiveness ratios per disability adjusted life year (DALY) averted, death averted and life-year gained (LYG) and conducted sensitivity analyses to how robust results were to variations in key model parameters.

RESULTS

Relative to status quo, RBF implied incremental costs of US$1122, US$26 220 and US$987 per additional DALY averted, death averted and LYG, respectively. The share of non-RBF facilities that provide quality care, life expectancy of mothers at time of delivery and the share of births in non-RBF facilities strongly influenced cost-effectiveness values. At a willingness to pay of US$1485 (3 times Malawi gross domestic product per capita) per DALY averted, RBF has a 77% probability of being cost-effective.

CONCLUSIONS

At high thresholds of wiliness-to-pay, RBF is a cost-effective intervention to improve quality of maternal and perinatal healthcare and outcomes, compared with the non-RBF based approach. More RBF cost-effectiveness analyses are needed in the SSA region to complement the few published studies and narrow the uncertainties surrounding cost-effectiveness estimates.

Intended and unintended effects: community perspectives on a performance-based financing programme in Malawi

Background

Several performance-based financing (PBF) evaluations have been undertaken in low-income countries, yet few have examined community perspectives of care amid PBF programme implementation. We assessed community members' perspectives of Support for Service Delivery Integration - Performance-Based Incentives ('SSDI-PBI'), a PBF intervention in Malawi, and explored some of the unintended effects that emerged amid implementation.

Methods

We conducted 30 focus group discussions: 17 with community leaders and 13 with mothers within catchment areas of SSDI-PBI implementing facilities. We analysed data using the framework approach.

Results

Community leaders and women had mixed impressions regarding the effect of SSDI-PBI on service delivery in facilities. They highlighted several improvements (including improved dialogue between staff and community, and cleaner, better-equipped facilities with enhanced privacy), but also persisting challenges (including inadequate and overworked staff, overcrowded facilities and long distances to facilities) related to services in SSDI-PBI-implementing facilities. Further, respondents described how four targeted service indicators related to maternal risk factor management, antenatal care (ANC) in the first trimester, skilled birth attendance and couple's HIV testing sparked unintended negative effects as experienced by women and communities. The unintended effects included women returning home for delivery, women feeling uncertain about their pregnancy status, women feeling betrayed or frustrated by the quality of care provided and partnerless women being denied ANC.

Conclusion

PBF programmes such as SSDI-PBI may improve some aspects of service delivery. However, to achieve system improvement, not only should necessary tools (such as medicines, equipment and human resources) be in place, but also programme priorities must be congruent with cultural expectations. Finally, facilities must be better supported to expect and then address increases in client load and heightened expectations in relation to services.

Effects in spite of tough constraints - A theory of change based investigation of contextual and implementation factors affecting the results of a performance based financing scheme extended to malnutrition in Burundi

BACKGROUND

From January 2015 to December 2016, the health authorities in Burundi piloted the inclusion of child nutrition services into the pre-existing performance-based financing free health care policy (PBF-FHC). An impact evaluation, focused on health centres, found positive effects both in terms of volume of services and quality of care. To some extent, this result is puzzling given the harshness of the contextual constraints related to the fragile setting.

METHODS

With a multi-methods approach, we explored how contextual and implementation constraints interacted with the pre-identified tracks of effect transmission embodied in the intervention. For our analysis, we used a hypothetical Theory of Change (ToC) that mapped a set of seven tracks through which the intervention might develop positive effects for children suffering from malnutrition. We built our analysis on (1) findings from the facility surveys and (2) extra qualitative data (logbooks, interviews and operational document reviews).

FINDINGS

Our results suggest that six constraints have weighted upon the intervention: (1) initial low skills of health workers; (2) unavailability of resources (including nutritional dietary inputs and equipment); (3) payment delays; (4) suboptimal information; (5) restrictions on autonomy; and (6) low intensity of supervision. Together, they have affected the intensity of the intervention, especially during its first year. From our analysis of the ToC, we noted that the positive effects largely occurred as a result of the incentive and information tracks. Qualitative data suggests that health centres have circumvented the many constraints by relying on a community-based recruitment strategy and a better management of inputs at the level of the facility and the patient himself.

CONCLUSION

Frontline actors have agency: when incentives are right, they take the initiative and find solutions. However, they cannot perform miracles: Burundi needs a holistic societal strategy to resolve the structural problem of child malnutrition.

TRIAL REGISTRATION

Clinical Trials.gov Identifier: NCT02721160; March 2016 (retrospectively registered).

Psychological wellbeing in a resource-limited work environment: examining levels and determinants among health workers in rural Malawi

BACKGROUND

A competent, responsive, and productive health workforce is central to a well-performing health system capable of providing universal access to high-quality care. Ensuring health workers' psychological wellbeing is critical to sustaining their availability and productivity. This is particularly true in heavily constrained health systems in low- and lower-middle-income countries. Research on the issue, however, is scarce. This study aimed to contribute to filling the gap in knowledge by investigating levels of and factors associated with psychological wellbeing of mid-level health workers in Malawi.

METHODS

The study relied on a cross-sectional sample of 174 health workers from 33 primary- and secondary-level health facilities in four districts of Malawi. Psychological wellbeing was measured using the WHO-5 Wellbeing Index. Data were analyzed using linear and logistic regression models.

RESULTS

Twenty-five percent of respondents had WHO-5 scores indicative of poor psychological wellbeing. Analyses of factors related to psychological wellbeing showed no association with sex, cadre, having dependents, supervision, perceived coworker support, satisfaction with the physical work environment, satisfaction with remuneration, and motivation; a positive association with respondents' satisfaction with interpersonal relationships at work; and a negative association with having received professional training recently. Results were inconclusive in regard to personal relationship status, seniority and responsibility at the health facility, clinical knowledge, perceived competence, perceived supervisor support, satisfaction with job demands, health facility level, data collection year, and exposure to performance-based financing.

CONCLUSIONS

The high proportion of health workers with poor wellbeing scores is concerning in light of the general health workforce shortage in Malawi and strong links between wellbeing and work performance. While more research is needed to draw conclusions and provide recommendations as to how to enhance wellbeing, our results underline the importance of considering this as a key concern for human resources for health.

Quality of maternal obstetric and neonatal care in low-income countries: development of a composite index

Background

In low-income countries, studies demonstrate greater access and utilization of maternal and neonatal health services, yet mortality rates remain high with poor quality increasingly scrutinized as a potential point of failure in achieving expected goals. Comprehensive measures reflecting the multi-dimensional nature of quality of care could prove useful to quality improvement. However, existing tools often lack a systematic approach reflecting all aspects of quality considered relevant to maternal and newborn care. We aim to address this gap by illustrating the development of a composite index using a step-wise approach to evaluate the quality of maternal obstetric and neonatal healthcare in low-income countries.

Methods

The following steps were employed in creating a composite index: 1) developing a theoretical framework; 2) metric selection; 3) imputation of missing data; 4) initial data analysis 5) normalization 6) weighting and aggregating; 7) uncertainty and sensitivity analysis of resulting composite score; 8) and deconstruction of the index into its components. Based on this approach, we developed a base composite index and tested alternatives by altering the decisions taken at different stages of the construction process to account for missing values, normalization, and aggregation. The resulting single composite scores representing overall maternal obstetric and neonatal healthcare quality were used to create facility rankings and further disaggregated into sub-composites of quality of care.

Results

The resulting composite scores varied considerably in absolute values and ranges based on method choice. However, the respective coefficients produced by the Spearman rank correlations comparing facility rankings by method choice showed a high degree of correlation. Differences in method of aggregation had the greatest amount of variation in facility rankings compared to the base case. Z-score standardization most closely aligned with the base case, but limited comparability at disaggregated levels.

Conclusions

This paper illustrates development of a composite index reflecting the multi-dimensional nature of maternal obstetric and neonatal healthcare. We employ a step-wise process applicable to a wide range of obstetric quality of care assessment programs in low-income countries which is adaptable to setting and context. In exploring alternative approaches, certain decisions influencing the interpretation of a given index are highlighted.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0790-0) contains supplementary material, which is available to authorized users.

Impact of results-based financing on effective obstetric care coverage: evidence from a quasi-experimental study in Malawi

BACKGROUND

Results-based financing (RBF) describes health system approaches addressing both service quality and use. Effective coverage is a metric measuring progress towards universal health coverage (UHC). Although considered a means towards achieving UHC in settings with weak health financing modalities, the impact of RBF on effective coverage has not been explicitly studied.

METHODS

Malawi introduced the Results-Based Financing For Maternal and Neonatal Health (RBF4MNH) Initiative in 2013 to improve quality of maternal and newborn health services at emergency obstetric care facilities. Using a quasi-experimental design, we examined the impact of the RBF4MNH on both crude and effective coverage of pregnant women across four districts during the two years following implementation.

RESULTS

There was no effect on crude coverage. With a larger proportion of women in intervention areas receiving more effective care over time, the overall net increase in effective coverage was 7.1%-points (p = 0.07). The strongest impact on effective coverage (31.0%-point increase, p = 0.02) occurred only at lower cut-off level (60% of maximum score) of obstetric care effectiveness. Design-specific and wider health system factors likely limited the program's potential to produce stronger effects.

CONCLUSION

The RBF4MNH improved effective coverage of pregnant women and seems to be a promising reform approach towards reaching UHC. Given the short study period, the full potential of the current RBF scheme has likely not yet been reached.

"I wanted a skeleton … they brought a prince": A qualitative investigation of factors mediating the implementation of a Performance Based Incentive program in Malawi

While several evaluations have examined the extent to which performance based financing (PBF) programs induce changes in the quantity and quality of health services provided, less is known about the process of implementing PBF. We conducted a process evaluation of a PBF intervention in Malawi that focused on understanding moderators of program implementation. Informed by a seminal theory of implementation, we first created a timeline and taxonomy of key events in the program lifeline and then undertook 25 in-depth interviews with stakeholders including implementers, central-level ministry officials and district-level health staff. While seven "moderator categories" emerged in this study, two categories (program complexity and quality of delivery) proved especially crucial in terms of moderating implementation and sparking adaptations. Complexity refers primarily to the manner in which PBF requires that those implementing the program have business acumen and forecasting skills, which are often beyond the purview of a clinician's training and thus proved challenging. Regarding quality of delivery, the program struggled to issue rewards in a timely and adequate manner, which proved highly problematic as it undermined a bedrock feature of PBF. Adaptations and adaptability refers here to a program's ability to make changes; the program proved rigid in several respects although nimble in terms of adjusting the verification process (upon noticing revengeful behaviors in peer verification). This PBF program is unique in several respects and findings cannot be generalized to all PBF programs. Nevertheless, process evaluations that draw from or expand upon existing implementation theories can allow researchers to better disentangle complex programming. We hope that more process evaluations, which track both core elements and necessary adaptations of PBF implementation, can further advance understandings of why PBF implementation functions or fails within a given setting, thereby enhancing implementers' abilities to replicate facilitators and bypass barriers.

The gendered micropolitics of hiding and disclosing: assessing the spread and stagnation of information on two new EMTCT policies in a Malawian village

Analysing why certain information spreads-or not-can be highly relevant for understanding an intervention's potential impact. Two recently implemented policy changes related to EMTCT (elimination of mother-to-child transmission of HIV) in the Balaka district of Malawi give ample opportunity to assess how new information trickles through a targeted rural community. One of the policies entails the lifetime provision of ART (anti-retroviral therapy) to all HIV+ pregnant women-a governmental strategy to EMTCT first initiated in Malawi and now being expanded throughout the region. The second new policy concerns a pilot project in which women are financially rewarded for attending antenatal care and delivering in the hospital. An in-depth anthropological approach was used to assess what women in one village community know about the policy changes and how they had come to know about it. Although the policies were implemented more or less at the same time, awareness and knowledge levels among village women differed largely: In case of the first, awareness stagnated at the level of those who directly received the information from health professionals. In the case of the second, highly accurate and up-to-date knowledge had spread throughout the village community. I suggest three reasons for this divergence: (i) perceived talk-worthiness of (issues addressed by) the interventions, (ii) motives for hiding or disclosing involvement in either of the interventions and (iii) the visibility of each intervention, or in other words, the (im)possibility to hide involvement. I argue that these reasons for women's structural silence on one policy change and prompt sharing of information on another follow a distinctly gendered logic. The findings underline that the diffusion of new information is to a great extent shaped by the social particularities of the context in which it is introduced.

How does performance-based financing affect health workers' intrinsic motivation? A Self-Determination Theory-based mixed-methods study in Malawi

"Intrinsic motivation crowding out", the erosion of high-quality, sustainable motivation through the introduction of financial incentives, is one of the most frequently discussed but yet little researched potential unfavorable consequence of Performance-based Financing (PBF). We used the opportunity of the introduction of PBF in Malawi to investigate whether and how PBF affected intrinsic motivation, using a mixed-methods research design theoretically grounded in Self-Determination Theory (SDT). The quantitative component served to estimate the impact of PBF on intrinsic motivation, relying on a controlled pre- and post-test design, with data collected from health workers in 23 intervention and 10 comparison facilities before (March/April 2013; n = 70) and approximately two years after (June/July 2015; n = 71) the start of the intervention. The qualitative component, relying on in-depth interviews with health workers in selected intervention facilities one (April 2014; n = 21) and two (September 2015; n = 20) years after the start of PBF, served to understand how PBF did or did not bring about change in intrinsic motivation. Specifically, it allowed us to examine how the various motivation-relevant elements and consequences of PBF impacted health workers' basic psychological needs for autonomy, competence, and relatedness, which SDT postulates as central to intrinsic motivation. Our results suggest that PBF did not affect health workers' overall intrinsic motivation levels, with the intervention having had both positive and negative effects on psychological needs satisfaction. To maximize positive PBF effects on intrinsic motivation, our results underline the potential value of explicit strategies to mitigate unintended negative impact of unavoidable design, implementation, and contextual challenges, for instance by building autonomy support activities into PBF designs.

Unraveling PBF effects beyond impact evaluation: results from a qualitative study in Cameroon

INTRODUCTION

Performance-based financing (PBF) has acquired increased prominence as a means of reforming health system purchasing structures in low-income and middle-income countries. A number of impact evaluations have noted that PBF often produces mixed and heterogeneous effects. Still, little systematic effort has been channelled towards understanding what causes such heterogeneity, including looking more closely at implementation processes.

METHODS

Our qualitative study aimed at closing this gap in knowledge by attempting to unpack the mixed and heterogeneous effects detected by the PBF impact evaluation in Cameroon to inform further implementation as the country scales up the PBF approach. We collected data at all levels of the health system (national, district, facility) and at the community level, using a mixture of in-depth interviews and focus group discussions. We combined deductive and inductive analytical techniques and applied analyst triangulation.

RESULTS

Our findings indicate that heterogeneity in effects across facilities could be explained by pre-existing infrastructural weaknesses coupled with rigid administrative processes and implementation challenges, while heterogeneity across indicators could be explained by providers' practices, privileging services where demand-side barriers were less substantive.

CONCLUSION

In light of the country's commitment to scaling up PBF, it follows that substantial efforts (particularly entrusting facilities with more financial autonomy) should be made to overcome infrastructural and demand-side barriers and to smooth implementation processes, thus, enabling healthcare providers to use PBF resources and management models to a fuller potential.

Household costs and time to seek care for pregnancy related complications: The role of results-based financing

Results-based financing (RBF) schemes-including performance based financing (PBF) and conditional cash transfers (CCT)-are increasingly being used to encourage use and improve quality of institutional health care for pregnant women in order to reduce maternal and neonatal mortality in low-income countries. While there is emerging evidence that RBF can increase service use and quality, little is known on the impact of RBF on costs and time to seek care for obstetric complications, although the two represent important dimensions of access. We conducted this study to fill the existing gap in knowledge by investigating the impact of RBF (PBF+CCT) on household costs and time to seek care for obstetric complications in four districts in Malawi. The analysis included data on 2,219 women with obstetric complications from three waves of a population-based survey conducted at baseline in 2013 and repeated in 2014(midline) and 2015(endline). Using a before and after approach with controls, we applied generalized linear models to study the association between RBF and household costs and time to seek care. Results indicated that receipt of RBF was associated with a significant reduction in the expected mean time to seek care for women experiencing an obstetric complication. Relative to non-RBF, time to seek care in RBF areas decreased by 27.3% (95%CI: 28.4-25.9) at midline and 34.2% (95%CI: 37.8-30.4) at endline. No substantial change in household costs was observed. We conclude that the reduced time to seek care is a manifestation of RBF induced quality improvements, prompting faster decisions on care seeking at household level. Our results suggest RBF may contribute to timely emergency care seeking and thus ultimately reduce maternal and neonatal mortality in beneficiary populations.

Evaluating the comparative effectiveness of different demand side interventions to increase maternal health service utilization and practice of birth spacing in South Kivu, Democratic Republic of Congo: an innovative, mixed methods approach

Background

In this protocol we describe a mixed methods study in the province of South Kivu, Democratic Republic of Congo evaluating the effectiveness of different demand side strategies to increase maternal health service utilization and the practice of birth spacing. Conditional service subsidization, conditional cash transfers and non-monetary incentives aim to encourage women to use maternal health services and practice birth spacing in two different health districts. Our methodology will comparatively evaluate the effectiveness of different approaches against each other and no intervention.

Methods/design

This study comprises four main research activities: 1) Formative qualitative research to determine feasibility of planned activities and inform development of the quantitative survey; 2) A community-based, longitudinal survey; 3) A retrospective review of health facility records; 4) Qualitative exploration of intervention acceptability and emergent themes through in-depth interviews with program participants, non-participants, their partners and health providers. Female community health workers are engaged as core members of the research team, working in tandem with female survey teams to identify women in the community who meet eligibility criteria. Female community health workers also act as key informants and community entry points during methods design and qualitative exploration. Main study outcomes are completion of antenatal care, institutional delivery, practice of birth spacing, family planning uptake and intervention acceptability in the communities. Qualitative methods also explore decision making around maternal health service use, fertility preference and perceptions of family planning.

Discussion

The innovative mixed methods design allows quantitative data to inform the relationships and phenomena to be explored in qualitative collection. In turn, qualitative findings will be triangulated with quantitative findings. Inspired by the principles of grounded theory, qualitative analysis will begin while data collection is ongoing. This “conversation” between quantitative and qualitative data will result in a more holistic, context-specific exploration and understanding of research topics, including the mechanisms through which the interventions are or are not effective. In addition, engagement of female community health workers as core members of the research team roots research methods in the realities of the community and provides teams with key informants who are simultaneously implicated in the health system, community and target population.

Perceptions of quality across the maternal care continuum in the context of a health financing intervention: Evidence from a mixed methods study in rural Malawi

BACKGROUND

In 2013, Malawi with its development partners introduced a Results-Based Financing for Maternal and Newborn Health (RBF4MNH) intervention to improve the quality of maternal and newborn health-care services. Financial incentives are awarded to health facilities conditional on their performance and to women for delivering in the health facility. We assessed the effect of the RBF4MNH on quality of care from women's perspectives.

METHODS

We used a mixed-method prospective sequential controlled pre- and post-test design. We conducted 3060 structured client exit interviews, 36 in-depth interviews and 29 focus group discussions (FGDs) with women and 24 in-depth interviews with health service providers between 2013 and 2015. We used difference-in-differences regression models to measure the effect of the RBF4MNH on experiences and perceived quality of care. We used qualitative data to explore the matter more in depth.

RESULTS

We did not observe a statistically significant effect of the intervention on women's perceptions of technical care, quality of amenities and interpersonal relations. However, in the qualitative interviews, most women reported improved health service provision as a result of the intervention. RBF4MNH increased the proportion of women reporting to have received medications/treatment during childbirth. Participants in interviews expressed that drugs, equipment and supplies were readily available due to the RBF4MNH. However, women also reported instances of neglect, disrespect and verbal abuse during the process of care. Providers attributed these negative instances to an increased workload resulting from an increased number of women seeking services at RBF4MNH facilities.

CONCLUSION

Our qualitative findings suggest improvements in the availability of drugs and supplies due to RBF4MNH. Despite the intervention, challenges in the provision of quality care persisted, especially with regard to interpersonal relations. RBF interventions may need to consider including indicators that specifically target the provision of respectful maternity care as a means to foster providers' positive attitudes towards women in labour. In parallel, governments should consider enhancing staff and infrastructural capacity before implementing RBF.

Implementation research to improve quality of maternal and newborn health care, Malawi

Objective

To evaluate the impact of a performance-based financing scheme on maternal and neonatal health service quality in Malawi.

Methods

We conducted a non-randomized controlled before and after study to evaluate the effects of district- and facility-level performance incentives for health workers and management teams. We assessed changes in the facilities’ essential drug stocks, equipment maintenance and clinical obstetric care processes. Difference-in-difference regression models were used to analyse effects of the scheme on adherence to obstetric care treatment protocols and provision of essential drugs, supplies and equipment.

Findings

We observed 33 health facilities, 23 intervention facilities and 10 control facilities and 401 pregnant women across four districts. The scheme improved the availability of both functional equipment and essential drug stocks in the intervention facilities. We observed positive effects in respect to drug procurement and clinical care activities at non-intervention facilities, likely in response to improved district management performance. Birth assistants’ adherence to clinical protocols improved across all studied facilities as district health managers supervised and coached clinical staff more actively.

Conclusion

Despite nation-wide stock-outs and extreme health worker shortages, facilities in the study districts managed to improve maternal and neonatal health service quality by overcoming bottlenecks related to supply procurement, equipment maintenance and clinical performance. To strengthen and reform health management structures, performance-based financing may be a promising approach to sustainable improvements in quality of health care.

Does performance-based financing increase value for money in low- and middle- income countries? A systematic review

Governments of low- and middle-income countries (LMICs) are widely implementing performance-based financing (PBF) to improve healthcare services. However, it is unclear whether PBF provides good value for money compared to status quo or other interventions aimed at strengthening the healthcare system in LMICs. The objective of this systematic review is to identify and synthesize the existing literature that examines whether PBF represents an efficient manner of investing resources. We considered PBF to be efficient when improved care quality or quantity was achieved with equal or lower costs, or alternatively, when the same quality of care was achieved using less financial resources. A manual search of the reference lists of two recent systematic reviews on economic evaluations of PBF was conducted to identify articles that met our inclusion and exclusion criteria. Subsequently, a search strategy was developed with the help of a librarian. The following databases and search engines were used: PubMed, EconLit, Google Scholar and Google. Experts on economic evaluations were consulted for validation of the selected studies. A total of seven articles from five LMICs were selected for this review. We found the overall strength of the evidence to be weak. None of the articles were full economic evaluations; they did not make clear connections between the costs and effects of PBF. Only one study reported using a randomized controlled trial, but issues with the randomization procedure were reported. Important alternative interventions to strengthen the capacities of the healthcare system have not been considered. Few studies examined the costs and consequences of PBF in the long term. Important costs and consequences were omitted from the evaluations. Few LMICs are represented in the literature, despite wide implementation. Lastly, most articles had at least one author employed by an organization involved in the implementation of PBF, thereby resulting in potential conflicts of interest. Stronger empirical evidence on whether PBF represents good value for money in LMICs is needed.

Evaluating complex health financing interventions: using mixed methods to inform further implementation of a novel PBI intervention in rural Malawi

Background

Gaps remain in understanding how performance-based incentive (PBI) programs affect quality of care and service quantity, whether programs are cost effective and how programs could be tailored to meet client and provider needs while remaining operationally viable. In 2014, Malawi’s Ministry of Health launched the Service Delivery Integration-PBI (SSDI-PBI) program. The program is unique in that no portion of performance bonuses are paid to individual health workers, and it shifts responsibility for infrastructure and equipment procurement from facility staff to implementing partners. This protocol outlines an approach that analyzes processes and outcomes, considers expected and unexpected consequences of the program and frames the program’s outputs relative to its costs. Findings from this evaluation will inform the intended future scale-up of PBI in Malawi.

Methods/design

This study employs a prospective controlled before-and-after triangulation design to assess effects of the PBI program by analyzing quantitative and qualitative data from intervention and control facilities. Guided by a theoretical framework, the evaluation consists of four main components: service provision, health worker motivation, implementation processes and costing. Quality and access outcomes are assessed along four dimensions: (1) structural elements (related to equipment, drugs, staff); (2) process elements (providers’ compliance with standards); (3) outputs (service utilization); (4) experiential elements (experiences of service delivery). The costing component includes costs related to start-up, ongoing management, and the cost of incentives themselves. The cost analysis considers costs incurred within the Ministry of Health, funders, and the implementing agency. The evaluation relies on primary data (including interviews and surveys) and secondary data (including costing and health management information system data).

Discussion

Through the lens of a PBI program, we illustrate how complex interventions can be evaluated via not only primary, mixed-methods data collection, but also through a wealth of secondary data from program implementers (including monitoring, evaluation and financial data), and the health system (including service utilization and service readiness data). We also highlight the importance of crafting a theory and using theory to inform the nature of data collected. Finally, we highlight the need to be responsive to stakeholders in order to enhance a study’s relevance.

A qualitative study assessing the acceptability and adoption of implementing a results based financing intervention to improve maternal and neonatal health in Malawi

Background

Results Based Financing (RBF) interventions have recently gained significant momentum, especially in sub-Saharan Africa. However, most of the research has focused on the evaluation of the impacts of this approach, providing little insight into how the contextual circumstances surrounding the implementation have contributed to its success or failure. This study aims to fill a void in the current literature on RBF by focusing explicitly on the process of implementing a RBF intervention rather than on its impact. Specifically, this study focuses on the acceptability and adoption of the RBF intervention’s implementation among local and international key stakeholders with the aim to inform further implementation.

Methods

The Results Based Financing for Maternal and Neonatal Health (RBF4MNH) Initiative is currently being implemented in Malawi. Our study employed an exploratory cross-sectional qualitative design to explore the factors affecting the acceptability and adoption of the intervention’s implementation. Purposeful sampling techniques were used to identify each key stakeholder who participated in all or parts of the implementation process. In-depth interviews were conducted and analyzed using a deductive open coding approach. The final interpretation of the findings emerged through active discussion among the co-authors.

Results

Despite encountering several challenges, such as delay in procurement of equipment and difficulties in arranging local bank accounts, all stakeholders responded positively to the RBF4MNH Initiative. Stakeholders’ acceptance of the RBF4MNH Initiative grew stronger over time as understanding of the intervention improved and was supported by early inclusion during the design and implementation process. In addition, stakeholders took on functions not directly incentivized by the intervention, suggesting that they turned adoption into actual ownership. All stakeholders raised concerns that the intervention may not be sustainable after its initial program phase would end, which contributed to hesitancy in fully accepting the intervention.

Conclusions

Based on the results of this study, we recommend the inclusion of local stakeholders into the intervention’s implementation process at the earliest stages. We also recommend setting up continuous feedback mechanisms to tackle challenges encountered during the implementation process. The sustainability of the intervention and its incorporation into national budgets should be addressed from the earliest stages.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-016-1652-7) contains supplementary material, which is available to authorized users.

Difference-in-Differences Method in Comparative Effectiveness Research: Utility with Unbalanced Groups

BACKGROUND

Comparative effectiveness research (CER) often includes observational studies utilizing administrative data. Multiple conditioning methods can be used for CER to adjust for group differences, including difference-in-differences (DiD) estimation.

OBJECTIVE

This study presents DiD and demonstrates how to apply this conditioning method to estimate treatment outcomes in the CER setting by utilizing the MarketScan® Databases for multiple sclerosis (MS) patients receiving different therapies.

METHODS

The sample included 6762 patients, with 363 in the Test Cohort [glatiramer acetate (GA) switched to fingolimod (FTY)] and 6399 in the Control Cohort (GA only, no switch) from a US administrative claims database. A trend analysis was conducted to rule out concerns regarding regression to the mean and to compare relapse rates among treatment cohorts. DiD analysis was used to enable comparisons among the Test and Control Cohorts. Logistic regression was used to estimate the probability of relapse after switching from GA to FTY, and to compare group differences in the pre- and post-index periods.

RESULTS

Crude DiD analysis showed that in the pre-index period more patients in the Test Cohort experienced an MS relapse and had a higher mean number of relapses than in the Control Cohort. During the pre-index period, numeric and relative data for MS relapses in patients in the Test Cohort were significantly higher than in the Control Cohort, while no significant between-group differences emerged during the post-index period. Generalized linear modeling with DiD regression estimation showed that the mean number of MS relapses decreased significantly in the post-index period among patients in the Test Cohort compared with patients in the Control Cohort.

CONCLUSION

In this study, an MS population was utilized to demonstrate how DiD can be applied to estimate treatment effects in a heterogeneous population, where the Test and Control Cohorts varied greatly. The results show that DiD offers a robust method for comparing diverse cohorts when other risk-adjustment methods may not be adequate.

Measuring and understanding the effects of a performance based financing scheme applied to nutrition services in Burundi-a mixed method impact evaluation design

BACKGROUND

Malnutrition is a huge problem in Burundi. In order to improve the provision of services at hospital, health centre and community levels, the Ministry of Health is piloting the introduction of malnutrition prevention and care indicators within its performance based financing (PBF) scheme. Paying for units of services and for qualitative indicators is expected to enhance provision and quality of these nutrition services, as PBF has done, in Burundi and elsewhere, for several other services.

METHODS

This paper presents the protocol for the impact evaluation of the PBF scheme applied to malnutrition. The research design consists in a mixed methods model adopting a sequential explanatory design. The quantitative component is a cluster-randomized controlled evaluation design: among the 90 health centres selected for the study, half receive payment related to their results in malnutrition activities, while the other half get a budget allocation. Qualitative research will be carried out both during the intervention period and at the end of the quantitative evaluation. Data are collected from 1) baseline and follow-up surveys of 90 health centres and 6,480 households with children aged 6 to 23 months, 2) logbooks filled in weekly in health centres, and 3) in-depth interviews and focus group discussions. The evaluation aims to provide the best estimate of the impact of the project on malnutrition outcomes in the community as well as outputs at the health centre level (malnutrition care outputs) and to describe quantitatively and qualitatively the changes that took place (or did not take place) within health centres as a result of the program.

DISCUSSION

Although PBF schemes are blooming in low in-come countries, there is still a need for evidence, especially on the impact of revising the list of remunerated indicators. It is expected that this impact evaluation will be helpful for the national policy dialogue in Burundi, but it will also provide key evidence for countries with an existing PBF scheme and confronted with malnutrition problems on the appropriateness to extend the strategy to nutrition services.

TRIAL REGISTRATION

ClinicalTrials.gov PRS Identifier: NCT02721160; registered March 2016.

Health worker preferences for performance-based payment schemes in a rural health district in Burkina Faso

BACKGROUND

One promising way to improve the motivation of healthcare providers and the quality of healthcare services is performance-based incentives (PBIs) also referred as performance-based financing. Our study aims to explore healthcare providers' preferences for an incentive scheme based on local resources, which aimed at improving the quality of maternal and child health care in the Nouna Health District.

DESIGN

A qualitative and quantitative survey was carried out in 2010 involving 94 healthcare providers within 34 health facilities. In addition, in-depth interviews involving a total of 33 key informants were conducted at health facility levels.

RESULTS

Overall, 85% of health workers were in favour of an incentive scheme based on the health district's own financial resources (95% CI: [71.91; 88.08]). Most health workers (95 and 96%) expressed a preference for financial incentives (95% CI: [66.64; 85.36]) and team-based incentives (95% CI: [67.78; 86.22]), respectively. The suggested performance indicators were those linked to antenatal care services, prevention of mother-to-child human immunodeficiency virus transmission, neonatal care, and immunization.

CONCLUSIONS

The early involvement of health workers and other stakeholders in designing an incentive scheme proved to be valuable. It ensured their effective participation in the process and overall acceptance of the scheme at the end. This study is an important contribution towards the designing of effective PBI schemes.

Effect of diagnostic testing on medicines used by febrile children less than five years in 12 malaria-endemic African countries: a mixed-methods study

BACKGROUND

In 2010, WHO revised guidelines to recommend testing all suspected malaria cases prior to treatment. Yet, evidence to assess programmes is largely derived from limited facility settings in a limited number of countries. National surveys from 12 sub-Saharan African countries were used to examine the effect of diagnostic testing on medicines used by febrile children under five years at the population level, including stratification by malaria risk, transmission season, source of care, symptoms, and age.

METHODS

Data were compiled from 12 Demographic and Health Surveys in 2010-2012 that reported fever prevalence, diagnostic test and medicine use, and socio-economic covariates (n=16,323 febrile under-fives taken to care). Mixed-effects logistic regression models quantified the influence of diagnostic testing on three outcomes (artemisinin combination therapy (ACT), any anti-malarial or any antibiotic use) after adjusting for data clustering and confounding covariates. For each outcome, interactions between diagnostic testing and the following covariates were separately tested: malaria risk, season, source of care, symptoms, and age. A multiple case study design was used to understand varying results across selected countries and sub-national groups, which drew on programme documents, published research and expert consultations. A descriptive typology of plausible explanations for quantitative results was derived from a cross-case synthesis.

RESULTS

Significant variability was found in the effect of diagnostic testing on ACT use across countries (e.g., Uganda OR: 0.84, 95% CI: 0.66-1.06; Mozambique OR: 3.54, 95% CI: 2.33-5.39). Four main themes emerged to explain results: available diagnostics and medicines; quality of care; care-seeking behaviour; and, malaria epidemiology.

CONCLUSIONS

Significant country variation was found in the effect of diagnostic testing on paediatric fever treatment at the population level, and qualitative results suggest the impact of diagnostic scale-up on treatment practices may not be straightforward in routine conditions given contextual factors (e.g., access to care, treatment-seeking behaviour or supply stock-outs). Despite limitations, quantitative results could help identify countries (e.g., Mozambique) or issues (e.g., malaria risk) where facility-based research or programme attention may be warranted. The mixed-methods approach triangulates different evidence to potentially provide a standard framework to assess routine programmes across countries or over time to fill critical evidence gaps.

Factors associated with delivery outside a health facility: cross-sectional study in rural Malawi

OBJECTIVE

To identify factors associated with delivery outside a health facility in rural Malawi.

METHOD

A cross-sectional survey was conducted in Balaka, Dedza, Mchinji and Ntcheu districts in Malawi in 2013 among women who had completed a pregnancy 12 months prior to the day of the survey. Multilevel logistic regression was used to assess factors associated with delivery outside a facility.

RESULTS

Of the 1812 study respondents, 9% (n = 159) reported to have delivered outside a facility. Unmarried women were significantly more likely [OR = 1.88; 95% CI (1.086-3.173)] to deliver outside a facility, while women from households with higher socio-economic status [third-quartile OR = 0.51; 95% CI (0.28-0.95) and fourth-quartile OR = 0.48; 95% CI (0.29-0.79)] and in urban areas [OR = 0.39; 95%-CI (0.23-0.67)] were significantly less likely to deliver outside a facility. Women without formal education [OR 1.43; 95% CI (0.96-2.14)] and multigravidae [OR = 1.14; 95% CI (0.98-1.73)] were more likely to deliver outside a health facility at 10% level of significance.

CONCLUSION

About 9% of women deliver outside a facility. Policies to encourage facility delivery should not only focus on health systems but also be multisectoral to address women's vulnerability and inequality. Facility-based delivery can contribute to curbing the high maternal illness burden if authorities provide incentives to those not delivering at the facility without losing existing users.

The quality of clinical maternal and neonatal healthcare - a strategy for identifying 'routine care signal functions'

BACKGROUND

A variety of clinical process indicators exists to measure the quality of care provided by maternal and neonatal health (MNH) programs. To allow comparison across MNH programs in low- and middle-income countries (LMICs), a core set of essential process indicators is needed. Although such a core set is available for emergency obstetric care (EmOC), the 'EmOC signal functions', a similar approach is currently missing for MNH routine care evaluation. We describe a strategy for identifying core process indicators for routine care and illustrate their usefulness in a field example.

METHODS

We first developed an indicator selection strategy by combining epidemiological and programmatic aspects relevant to MNH in LMICs. We then identified routine care process indicators meeting our selection criteria by reviewing existing quality of care assessment protocols. We grouped these indicators into three categories based on their main function in addressing risk factors of maternal or neonatal complications. We then tested this indicator set in a study assessing MNH quality of clinical care in 33 health facilities in Malawi.

RESULTS

Our strategy identified 51 routine care processes: 23 related to initial patient risk assessment, 17 to risk monitoring, 11 to risk prevention. During the clinical performance assessment a total of 82 cases were observed. Birth attendants' adherence to clinical standards was lowest in relation to risk monitoring processes. In relation to major complications, routine care processes addressing fetal and newborn distress were performed relatively consistently, but there were major gaps in the performance of routine care processes addressing bleeding, infection, and pre-eclampsia risks.

CONCLUSION

The identified set of process indicators could identify major gaps in the quality of obstetric and neonatal care provided during the intra- and immediate postpartum period. We hope our suggested indicators for essential routine care processes will contribute to streamlining MNH program evaluations in LMICs.

Maternal and neonatal tetanus elimination: from protecting women and newborns to protecting all

A total of 35 of the 59 countries that had not eliminated maternal and neonatal tetanus (MNT) as a public health problem in 1999 have since achieved the MNT-elimination goal. Neonatal tetanus deaths have decreased globally from 200,000 in 2000 to 49,000 in 2013. This is the result of increased immunization coverage with tetanus toxoid-containing vaccines among pregnant women, improved access to skilled birth attendance during delivery, and targeted campaigns with these vaccines for women of reproductive age in high-risk areas. In the process, inequities have been reduced, private–public partnerships fostered, and innovations triggered. However, lack of funding, poor accessibility to some areas, suboptimal surveillance, and a perceived low priority for the disease are among the main obstacles. To ensure MNT elimination is sustained, countries must build and maintain strong routine programs that reach people with vaccination and with clean deliveries. This should also be an opportunity to shift programs into preventing tetanus among all people. Regular assessments, and where needed appropriate action, are key to prevent increases in MNT incidence over time, especially in areas that are at higher risk. The main objective of the paper is to provide a detailed update on the progress toward MNT elimination between 1999 and 2014. It elaborates on the challenges and opportunities, and discusses how MNT elimination can be sustained and to shift the program to protect wider populations against tetanus.