Exploiting the emergent nature of mixed methods designs: insights from a mixed methods impact evaluation in Malawi

Implementation of a performance-based financing scheme in Malawi and resulting externalities on the quality of care of non-incentivized services

Background

Countries in Africa progressively implement performance-based financing schemes to improve the quality of care provided by maternal, newborn and child health services. Beyond its direct effects on service provision, evidence suggests that performance-based financing can also generate positive externalities on service utilization, such as increased use of those services that reached higher quality standards after effective scheme implementation. Little, however, is known about externalities generated within non-incentivized health services, such as positive or negative effects on the quality of services within the continuum of maternal care.

Methods

We explored whether a performance-based financing scheme in Malawi designed to improve the quality of childbirth service provision resulted positive or negative externalities on the quality of non-targeted antenatal care provision. This non-randomized controlled pre-post-test study followed the phased enrolment of facilities into a performance-based financing scheme across four districts over a two-year period. Effects of the scheme were assessed by various composite scores measuring facilities’ readiness to provide quality antenatal care, as well as the quality of screening, prevention, and education processes offered during observed antenatal care consultations.

Results

Our study did not identify any statistically significant effects on the quality of ANC provision attributable to the implemented performance-based financing scheme. Our findings therefore suggest not only the absence of positive externalities, but also the absence of any negative externalities generated within antenatal care service provision as a result of the scheme implementation in Malawi.

Conclusions

Prior research has shown that the Malawian performance-based financing scheme was sufficiently effective to improve the quality of incentivized childbirth service provision. Our findings further indicate that scheme implementation did not affect the quality of non-incentivized but clinically related antenatal care services. While no positive externalities could be identified, we also did not observe any negative externalities attributable to the scheme’s implementation. While performance-based incentives might be successful in improving targeted health care processes, they have limited potential in producing externalities – neither positive nor negative – on the provision quality of related non-incentivized services.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12884-021-03880-9.

Mixed and Multi-Methods Protocol to Evaluate Implementation Processes and Early Effects of the Pradhan Mantri Jan Arogya Yojana Scheme in Seven Indian States

In September 2018, India launched Pradhan Mantri Jan Arogya Yojana (PM-JAY), a nationally implemented government-funded health insurance scheme to improve access to quality inpatient care, increase financial protection, and reduce unmet need for the most vulnerable population groups. This protocol describes the methodology adopted to evaluate implementation processes and early effects of PM-JAY in seven Indian states. The study adopts a mixed and multi-methods concurrent triangulation design including three components: 1. demand-side household study, including a structured survey and qualitative elements, to quantify and understand PM-JAY reach and its effect on insurance awareness, health service utilization, and financial protection; 2. supply-side hospital-based survey encompassing both quantitative and qualitative elements to assess the effect of PM-JAY on quality of service delivery and to explore healthcare providers' experiences with scheme implementation; and 3. process documentation to examine implementation processes in selected states transitioning from either no or prior health insurance to PM-JAY. Descriptive statistics and quasi-experimental methods will be used to analyze quantitative data, while thematic analysis will be used to analyze qualitative data. The study design presented represents the first effort to jointly evaluate implementation processes and early effects of the largest government-funded health insurance scheme ever launched in India.