Assessing performance enhancing tools: experiences with the open performance review and appraisal system (OPRAS) and expectations towards payment for performance (P4P) in the public health sector in Tanzania

Kidanto H. Scaling up Locally Adapted Clinical Practice Guidelines for Improving Childbirth Care in Tanzania: A Protocol for Programme Theory and Qualitative Methods of the PartoMa Scale-up Study

Effective, low-cost clinical interventions to improve facility-based care during childbirth are critical to reduce maternal and perinatal mortality and morbidity in low-resource settings. While health interventions for low- and lower-middle-income countries are often developed and implemented top-down, needs and circumstances vary greatly across locations. Our pilot study in Zanzibar improved care through locally co-created intrapartum clinical practice guidelines (CPGs) and associated training (the PartoMa intervention). This intervention was context-tailored with health-care providers in Zanzibar and now scaled up within five maternity units in Dar es Salaam, Tanzania. This PartoMa Scale-up Study thereby provides an opportunity to explore the co-creation process and modification of the intervention in another context and how scale-up might be successfully achieved. The overall protocol is presented in a separate paper. The aim of the present paper is to account for the Scale-up Study's programme theory and qualitative methodology. We introduce social practice theory and argue for its value within the programme theory and towards qualitative explorations of shifts in clinical practice. The theory recognizes that the practice we aim to strengthen - safe and respectful clinical childbirth care - is not practiced in a vacuum but embedded within a socio-material context and intertwined with other practices. Methodologically, the project draws on ethnographic and participatory methodologies to explore current childbirth care practices. In line with our programme theory, explorations will focus on meanings of childbirth care, material tools and competencies that are being drawn upon, birth attendants' motivations and relational contexts, as well as other everyday practices of childbirth care. Insights generated from this study will not only elucidate active ingredients that make the PartoMa intervention feasible (or not) but develop the knowledge foundation for scaling-up and replicability of future interventions based on the principles of co-creation and contextualisation.

Perceptions and Experiences of Health Care Workers on Accountability Mechanisms for Enhancing Quality Improvement in the Delivery of Maternal Newborns and Child Health Services in Mkuranga, Tanzania

Background

Maternal mortality estimates globally show that by 2017 there were still 211 deaths per 100,000 live births; more strikingly, 99% of them happen in low and middle-income countries, including Tanzania. There has been insufficient progress in improving maternal and newborn health despite the efforts to strengthen the health systems, to improve the quality of maternal health in terms of training and deploying human resources for health, constructing health facilities, and supplying medical products. However, fewer efforts are invested in enhancing accountability toward the improvement of the quality of maternal health care. This the study was conducted to explore the perceptions of healthcare workers regarding accountability mechanisms for enhancing quality improvement in the delivery of maternal newborn and child health services in Tanzania.

Methods

We adopted phenomenology as a study design to understand how health workers perceive accountability and data were collected using semi-structured interviews. We then used thematic analysis to analyze themes and sub- themes.

Results

The study revealed four categories of perceptions namely, differences in the conceptualization of accountability and accountability mechanisms, varied opinions about the existing accountability mechanisms, perceived the usefulness of accountability mechanisms, together with perceived challenges in the enforcement of accountability mechanisms.

Conclusion

Perceived variations in the understanding of accountability among healthcare workers signaled a proper but fragmented understanding of accountability in maternal care. Accountability mechanisms are perceived to be useful for enhancing hard work in the provision of maternal health services. Moreover, inadequate motivation resulting from health system bottlenecks tend to constrain enforcement of accountability in the provision of maternal care services. Thus, we recommend that the government should deal with health system constraints and enforce regular monitoring and supervision.

Evaluation of the Direct Health Facility Financing Program in Improving Maternal Health Services in Pangani District, Tanzania

BACKGROUND

Maternal morbidity and mortality remain significant public health concerns globally, with Tanzania reporting 398 deaths per 100,000 live births annually. While national level data provide some insights into the issue, a focus on sub-national levels is required because of differences in contexts such as rural-urban disparities in maternal mortality. This study examined Direct Health Facility Financing (DHFF) and its effects on the quality of maternal health services in Pangani, a rural district in Tanzania.

METHODS

This study was conducted in Pangani district of Tanga region in Tanzania. The study used both qualitative and quantitative methods, including 16 in-depth interviews with the council health management teams, facility in charges, maternity nurse in charge, and 5 focus group discussions with community health governing committee members. The number of deliveries that occurred in health facilities, as well as medical supplies, equipment, and reagents purchased by the facilities, were compared using descriptive statistics before and after the DHFF implementation.

RESULTS

Direct disbursement of funds from the central government through the Ministry of Finance and Planning to the primary health facilities reduced delays in procurement, improved community outreach services, and improved community leaders' engagements. Deliveries occurring at health facilities increased by 33.6% (p < 0.001) one year after the HDFF implementation. Various medicines, delivery kits, and some reagents increased significantly (p < 0.05). However, the lack of computers and poor internet connectivity, an insufficient supply of medical equipment and unstable stock of the Medical Stores Department increased the difficulty of obtaining the missed items from the selected prime vendor.

CONCLUSION

Overall, this study shows a positive impact of the DHFF on maternal health service delivery in Pangani district. Specifically, an increase in the number of medical supplies, equipment, and reagents necessary to provide maternal health services contributed to the observed increase in facility deliveries by 33.6%. Moreover, the system minimizes unnecessary delays in the procurement processes of required drugs, supplies, and other facility reagents. To maximize the impact of the HDFF system, lack of computers, unstable internet, limited knowledge of the staff about the system, and inadequate health workforce should be addressed. Therefore, strengthening the DHFF system and staff training in-service and on the job is essential for smooth implementation.

Does the gap between health workers' expectations and the realities of implementing a performance-based financing project in Mali create frustration?

Background

Performance-Based Financing (PBF), an innovative health financing initiative, was recently implemented in Mali. PBF aims to improve quality of care by motivating health workers. The purpose of this research was to identify and understand how health workers’ expectations related to their experiences of the first cycle of payment of PBF subsidies, and how this experience affected their motivation and sentiments towards the intervention. We pose the research question, “how does the process of PBF subsidies impact the motivation of health workers in Mali?”

Methods

We adopted a qualitative approach using multiple case studies. We chose three district hospitals (DH 1, 2 and 3) in three health districts (district 1, 2 and 3) among the ten in the Koulikoro region. Our cases correspond to the three DHs. We followed the principle of data source triangulation; we used 53 semi-directive interviews conducted with health workers (to follow the principle of saturuation), field notes, and documents relating to the distribution grids of subsidies for each DH. We analyzed data in a mixed deductive and inductive manner.

Results

The results show that the PBF subsidies led to health workers feeling more motivated to perform their tasks overall. Beyond financial motivation, this was primarily due to PBF allowing them to work more efficiently. However, respondents perceived a discrepancy between the efforts made and the subsidies received. The fact that their expectations were not met led to a sense of frustration and disappointment. Similarly, the way in which the subsidies were distributed and the lack of transparency in the distribution process led to feelings of unfairness among the vast majority of respondents. The results show that frustrations can build up in the early days of the intervention.

Conclusion

The PBF implementation in Mali left health workers frustrated. The short overall implementation period did not allow actors to adjust their initial expectations and motivational responses, neither positive nor negative. This underlines how short-term interventions might not just lack impact, but instil negative sentiments likely to carry on into the future.

A realist review to assess for whom, under what conditions and how pay for performance programmes work in low- and middle-income countries

Pay for performance (P4P) programmes are popular health system-focused interventions aiming to improve health outcomes in low-and middle-income countries (LMICs). This realist review aims to understand how, why and under what circumstance P4P works in LMICs.We systematically searched peer-reviewed and grey literature databases, and examined the mechanisms underpinning P4P effects on: utilisation of services, patient satisfaction, provider productivity and broader health system, and contextual factors moderating these. This evidence was then used to construct a causal loop diagram.We included 112 records (19 grey literature; 93 peer-reviewed articles) assessing P4P schemes in 36 countries. Although we found mixed evidence of P4P's effects on identified outcomes, common pathways to improved outcomes include: community outreach; adherence to clinical guidelines, patient-provider interactions, patient trust, facility improvements, access to drugs and equipment, facility autonomy, and lower user fees. Contextual factors shaping the system response to P4P include: degree of facility autonomy, efficiency of banking, role of user charges in financing public services; staffing levels; staff training and motivation, quality of facility infrastructure and community social norms. Programme design features supporting or impeding health system effects of P4P included: scope of incentivised indicators, fairness and reach of incentives, timely payments and a supportive, robust verification system that does not overburden staff. Facility bonuses are a key element of P4P, but rely on provider autonomy for maximum effect. If health system inputs are vastly underperforming pre-P4P, they are unlikely to improve only due to P4P. This is the first realist review describing how and why P4P initiatives work (or fail) in different LMIC contexts by exploring the underlying mechanisms and contextual and programme design moderators. Future studies should systematically examine health system pathways to outcomes for P4P and other health system strengthening initiatives, and offer more understanding of how programme design shapes mechanisms and effects.

How are pay-for-performance schemes in healthcare designed in low- and middle-income countries? Typology and systematic literature review

BACKGROUND

Pay for performance (P4P) schemes provide financial incentives to health workers or facilities based on the achievement of pre-specified performance targets and have been widely implemented in health systems across low and middle-income countries (LMICs). The growing evidence base on P4P highlights that (i) there is substantial variation in the effect of P4P schemes on outcomes and (ii) there appears to be heterogeneity in incentive design. Even though scheme design is likely a key determinant of scheme effectiveness, we currently lack systematic evidence on how P4P schemes are designed in LMICs.

METHODS

We develop a typology to classify the design of P4P schemes in LMICs, which highlights different design features that are a priori likely to affect the behaviour of incentivised actors. We then use results from a systematic literature review to classify and describe the design of P4P schemes that have been evaluated in LMICs. To capture academic publications, Medline, Embase, and EconLit databases were searched. To include relevant grey literature, Google Scholar, Emerald Insight, and websites of the World Bank, WHO, Cordaid, Norad, DfID, USAID and PEPFAR were searched.

RESULTS

We identify 41 different P4P schemes implemented in 29 LMICs. We find that there is substantial heterogeneity in the design of P4P schemes in LMICs and pinpoint precisely how scheme design varies across settings. Our results also highlight that incentive design is not adequately being reported on in the literature - with many studies failing to report key design features.

CONCLUSIONS

We encourage authors to make a greater effort to report information on P4P scheme design in the future and suggest using the typology laid out in this paper as a starting point.

Intended and unintended effects: community perspectives on a performance-based financing programme in Malawi

Background

Several performance-based financing (PBF) evaluations have been undertaken in low-income countries, yet few have examined community perspectives of care amid PBF programme implementation. We assessed community members' perspectives of Support for Service Delivery Integration - Performance-Based Incentives ('SSDI-PBI'), a PBF intervention in Malawi, and explored some of the unintended effects that emerged amid implementation.

Methods

We conducted 30 focus group discussions: 17 with community leaders and 13 with mothers within catchment areas of SSDI-PBI implementing facilities. We analysed data using the framework approach.

Results

Community leaders and women had mixed impressions regarding the effect of SSDI-PBI on service delivery in facilities. They highlighted several improvements (including improved dialogue between staff and community, and cleaner, better-equipped facilities with enhanced privacy), but also persisting challenges (including inadequate and overworked staff, overcrowded facilities and long distances to facilities) related to services in SSDI-PBI-implementing facilities. Further, respondents described how four targeted service indicators related to maternal risk factor management, antenatal care (ANC) in the first trimester, skilled birth attendance and couple's HIV testing sparked unintended negative effects as experienced by women and communities. The unintended effects included women returning home for delivery, women feeling uncertain about their pregnancy status, women feeling betrayed or frustrated by the quality of care provided and partnerless women being denied ANC.

Conclusion

PBF programmes such as SSDI-PBI may improve some aspects of service delivery. However, to achieve system improvement, not only should necessary tools (such as medicines, equipment and human resources) be in place, but also programme priorities must be congruent with cultural expectations. Finally, facilities must be better supported to expect and then address increases in client load and heightened expectations in relation to services.

Brazilian Payment for Performance (PMAQ) Seen From a Global Health and Public Policy Perspective: What Does It Mean for Research and Policy?

This supplement of the Journal of Ambulatory Care Management on the Brazilian National Program for Improving Access and Quality of Primary Care (PMAQ) reveals a relevant gap in the Brazilian literature on pay for performance/PMAQ, and is therefore an opportunity to bring contributions from global health and public policy to the debate. We discuss the relevant gap in the light of developments in evaluation and policy analysis. We afterward present the state of knowledge regarding global health and public policy in pay for performance, giving attention to diverse themes, methods, types of analyses, theoretical contributions, and limitations. Finally, we suggest some possible implications for research and policy in Brazil.

Support and performance improvement for primary health care workers in low- and middle-income countries: a scoping review of intervention design and methods

Primary health care workers (HCWs) in low- and middle-income settings (LMIC) often work in challenging conditions in remote, rural areas, in isolation from the rest of the health system and particularly specialist care. Much attention has been given to implementation of interventions to support quality and performance improvement for workers in such settings. However, little is known about the design of such initiatives and which approaches predominate, let alone those that are most effective. We aimed for a broad understanding of what distinguishes different approaches to primary HCW support and performance improvement and to clarify the existing evidence as well as gaps in evidence in order to inform decision-making and design of programs intended to support and improve the performance of health workers in these settings. We systematically searched the literature for articles addressing this topic, and undertook a comparative review to document the principal approaches to performance and quality improvement for primary HCWs in LMIC settings. We identified 40 eligible papers reporting on interventions that we categorized into five different approaches: (1) supervision and supportive supervision; (2) mentoring; (3) tools and aids; (4) quality improvement methods, and (5) coaching. The variety of study designs and quality/performance indicators precluded a formal quantitative data synthesis. The most extensive literature was on supervision, but there was little clarity on what defines the most effective approach to the supervision activities themselves, let alone the design and implementation of supervision programs. The mentoring literature was limited, and largely focused on clinical skills building and educational strategies. Further research on how best to incorporate mentorship into pre-service clinical training, while maintaining its function within the routine health system, is needed. There is insufficient evidence to draw conclusions about coaching in this setting, however a review of the corporate and the business school literature is warranted to identify transferrable approaches. A substantial literature exists on tools, but significant variation in approaches makes comparison challenging. We found examples of effective individual projects and designs in specific settings, but there was a lack of comparative research on tools across approaches or across settings, and no systematic analysis within specific approaches to provide evidence with clear generalizability. Future research should prioritize comparative intervention trials to establish clear global standards for performance and quality improvement initiatives. Such standards will be critical to creating and sustaining a well-functioning health workforce and for global initiatives such as universal health coverage.

A qualitative analysis of health professionals' job descriptions for surgical service delivery in Uganda

BACKGROUND

The ever increasing demand for surgical services in sub-Saharan Africa is creating a need to increase the number of health workers able to provide surgical care. This calls for the optimisation of all available human resources to provide universal access to essential and emergency surgical services. One way of optimising already scarce human resources for health is by clarifying job descriptions to guide the scope of practice, measuring rewards/benefits for the health workers providing surgical care, and informing education and training for health professionals. This study set out to determine the scope of the mandate to perform surgical procedures in current job descriptions of surgical care health professionals in Uganda.

METHODS

A document review was conducted of job descriptions for the health professionals responsible for surgical service delivery in the Ugandan Health care system. The job descriptions were extracted and subjected to a qualitative content data analysis approach using a text based RQDA package of the open source R statistical computing software.

RESULTS

It was observed that there was no explicit mention of assignment of delivery of surgical services to a particular cadre. Instead the bulk of direct patient related care, including surgical attention, was assigned to the lower cadres, in particular the medical officer. Senior cadres were assigned to perform predominantly advisory and managerial roles in the health care system. In addition, a no cost opportunity to task shift surgical service delivery to the senior clinical officers was identified.

CONCLUSIONS

There is a need to specifically assign the mandate to provide surgical care tasks, according to degree of complexity, to adequately trained cadres of health workers. Health professionals' current job descriptions are not explicit, and therefore do not adequately support proper training, deployment, defined scope of practice, and remuneration for equitable surgical service delivery in Uganda. Such deliberate assignment of mandates will provide a means of increasing surgical service delivery through further optimisation of the available human resources for health.

Design of an impact evaluation using a mixed methods model--an explanatory assessment of the effects of results-based financing mechanisms on maternal healthcare services in Malawi

BACKGROUND

In this article we present a study design to evaluate the causal impact of providing supply-side performance-based financing incentives in combination with a demand-side cash transfer component on equitable access to and quality of maternal and neonatal healthcare services. This intervention is introduced to selected emergency obstetric care facilities and catchment area populations in four districts in Malawi. We here describe and discuss our study protocol with regard to the research aims, the local implementation context, and our rationale for selecting a mixed methods explanatory design with a quasi-experimental quantitative component.

DESIGN

The quantitative research component consists of a controlled pre- and post-test design with multiple post-test measurements. This allows us to quantitatively measure 'equitable access to healthcare services' at the community level and 'healthcare quality' at the health facility level. Guided by a theoretical framework of causal relationships, we determined a number of input, process, and output indicators to evaluate both intended and unintended effects of the intervention. Overall causal impact estimates will result from a difference-in-difference analysis comparing selected indicators across intervention and control facilities/catchment populations over time.To further explain heterogeneity of quantitatively observed effects and to understand the experiential dimensions of financial incentives on clients and providers, we designed a qualitative component in line with the overall explanatory mixed methods approach. This component consists of in-depth interviews and focus group discussions with providers, service user, non-users, and policy stakeholders. In this explanatory design comprehensive understanding of expected and unexpected effects of the intervention on both access and quality will emerge through careful triangulation at two levels: across multiple quantitative elements and across quantitative and qualitative elements.

DISCUSSION

Combining a traditional quasi-experimental controlled pre- and post-test design with an explanatory mixed methods model permits an additional assessment of organizational and behavioral changes affecting complex processes. Through this impact evaluation approach, our design will not only create robust evidence measures for the outcome of interest, but also generate insights on how and why the investigated interventions produce certain intended and unintended effects and allows for a more in-depth evaluation approach.

Does training on performance based financing make a difference in performance and quality of health care delivery? Health care provider's perspective in Rungwe Tanzania

Background

In recent years, Performance Based Financing (PBF); a form of result based financing, has attracted a global attention in health systems in developing countries. PBF promotes autonomous health facilities, motivates and introduces financial incentives to motivate health facilities and health workers to attain pre-determined targets. To achieve this, the Tanzanian government through the Christian Social Services Commission initiated a PBF pilot project in Rungwe district, Mbeya region. Kilimanjaro Christian Medical Center was given the role of training health workers on PBF principles in Rungwe. The aim of this study was to explore health care providers’ perception on a three years training on PBF principles in a PBF pilot project at Rungwe District in Mbeya, Tanzania.

Methods

This was an explorative qualitative study, which took place at Rungwe PBF pilot area in October 2012. Twenty six (26) participants were purposively selected. Six took part in- depth interviews (IDIs) and twenty (20) in the group discussions. Both the IDIs and the GDs explored the perceived benefit and challenges of implementing PBF in their workplace. Data were manually analyzed using content analysis approach.

Results

Overall informants had positive perspectives on PBF training. Most of the health facilities were able to implement some of the PBF concepts in their work places after the training, such as developing job descriptions for their staff, creating quarterly business plans for their facilities, costing for their services and entering service agreement with the government, improved record keeping, customer care and involving community as partners in running their facilities. The most common principle of paying individual performance bonuses was mentioned as a major challenge due to inadequate funding and poor design of Rungwe PBF pilot project.

Conclusion

Despite poor design and inadequate funding, our findings have shown some promising results after PBF training in the study area. The findings have highlighted the potential of PBF to act as leverage for initiating innovative and proactive actions, which may motivate health personnel performance and quality of care in the study setting with minimal support. However, key policy issues at the national level should be addressed in order to exploit this opportunity.

When incentives work too well: locally implemented pay for performance (P4P) and adverse sanctions towards home birth in Tanzania - a qualitative study

BACKGROUND

Despite limited evidence of its effectiveness, performance-based payments (P4P) are seen by leading policymakers as a potential solution to the slow progress in reaching Millennium Development Goal 5: improved maternal health. This paper offers insights into two of the aspects that are lacking in the current literature on P4P, namely what strategies health workers employ to reach set targets, and how the intervention plays out when implemented by local government as part of a national programme that does not receive donor funding.

METHODS

A total of 28 in-depth interviews (IDIs) with 25 individuals were conducted in Mvomero district over a period of 15 months in 2010 and 2011, both before and after P4P payments. Seven facilities, including six dispensaries and one health centre, were covered. Informants included 17 nurses, three clinical officers, two medical attendants, one lab technician and two district health administrators.

RESULTS

Health workers reported a number of strategies to increase the number of deliveries at their facility, including health education and cooperation with traditional health providers. The staff at all facilities also reported that they had told the women that they would be sanctioned if they gave birth at home, such as being fined or denied clinical cards and/or vaccinations for their babies. There is a great uncertainty in relation to the potential health impacts of the behavioural changes that have come with P4P, as the reported strategies may increase the numbers, but not necessarily the quality. Contrary to the design of the P4P programme, payments were not based on performance. We argue that this was due in part to a lack of resources within the District Administration, and in part as a result of egalitarian fairness principles.

CONCLUSIONS

Our results suggest that particular attention should be paid to adverse effects when using external rewards for improved health outcomes, and secondly, that P4P may take on a different form when implemented by local implementers without the assistance of professional P4P specialists.