Evidence gap map of performance measurement and management in primary care delivery systems in low- and middle-income countries - Study protocol

PROTOCOL: Effectiveness of social accountability interventions in low- and middle-income countries: An evidence and gap map

This is the protocol for an evidence and gap map, which aims to map the existing evidence on the effectiveness of social accountability interventions in low- and middle-income countries. This map will help users identify the size and quality of the evidence base, guide strategic program development, and highlight gaps for future research. The map will cover studies published after 2000, including systematic reviews, randomized controlled trials, non-experimental designs, and before-after designs.

Improving primary health care delivery in Bihar, India: Learning from piloting and statewide scale-up of Ananya

In 2010, the Bill and Melinda Gates Foundation (BMGF) partnered with the Government of Bihar (GoB), India to launch the Ananya program to improve reproductive, maternal, newborn and child health and nutrition (RMNCHN) outcomes. The program sought to address supply- and demand-side barriers to the adoption, coverage, quality, equity and health impact of select RMNCHN interventions. Approaches included strengthening frontline worker service delivery; social and behavior change communications; layering of health, nutrition and sanitation into women's self-help groups (SHGs); and quality improvement in maternal and newborn care at primary health care facilities. Ananya program interventions were piloted in approximately 28 million population in eight innovation districts from 2011-2013, and then beginning in 2014, were scaled up by the GoB across the rest of the state's population of 104 million. A Bihar Technical Support Program provided techno-managerial support to governmental Health as well as Integrated Child Development Services, and the JEEViKA Technical Support Program supported health layering and scale-up of the GoB's SHG program. The level of support at the block level during statewide scale-up in 2014 onwards was approximately one-fourth that provided in the pilot phase of Ananya in 2011-2013. This paper - the first manuscript in an 11-manuscript and 2-viewpoint collection on Learning from Ananya: Lessons for primary health care performance improvement - seeks to provide a broad description of Ananya and subsequent statewide adaptation and scale-up, and capture the background and context, key objectives, interventions, delivery approaches and evaluation methods of this expansive program. Subsequent papers in this collection focus on specific intervention delivery platforms. For the analyses in this series, Stanford University held key informant interviews and worked with the technical support and evaluation grantees of the Ananya program, as well as leadership from the India Country Office of the BMGF, to analyse and synthesise data from multiple sources. Capturing lessons from the Ananya pilot program and statewide scale-up will assist program managers and policymakers to more effectively design and implement RMNCHN programs at scale through technical assistance to governments.

Development of an Evidence and Gap Map (EGM) of interventions to increase smoking cessation: A study protocol

Smoking remains one of the leading risk factors contributing to the global burden of disease. Sub-optimal implementation of evidence-based tobacco control and smoking cessation practice is a major challenge despite a substantial evidence base for interventions to increase smoking cessation globally. We aim to develop an Evidence and Gap Map (EGM) to collate the existing evidence and identify the gap in research on interventions to increase smoking cessation worldwide. A conceptual framework was developed followed by the formulation of a search strategy with key search terms and search period (1970 - date of search). The search will be conducted in relevant bibliographic databases (e.g. MEDLINE, Embase, SCOPUS), systematic reviews databases (e.g. Cochrane Library, Joanna Briggs systematic reviews, EPPI-Reviewer) and impact evaluation databases (e.g. 3ie Impact Evaluation repository and Cochrane tobacco addiction group specialized register) with support from a research librarian. Subsequently, two coders will screen and retrieve systematic reviews and individual impact evaluation studies. The adapted SURE (Supporting the Use of Research Evidence) checklist will be used to evaluate the quality of the included systematic reviews. A narrative synthesis from the systematic review findings and line listing of the impact evaluations will form the basis of this EGM. The EGM report will be presented in an interactive visual format. The proposed EGM will organise the pieces of evidence generated in systematic reviews and impact evaluations on smoking cessation interventions and identify the current research gaps, if any. The findings will inform evidence-based practice and future research.

Effects of team-based goals and non-monetary incentives on front-line health worker performance and maternal health behaviours: a cluster randomised controlled trial in Bihar, India

Introduction

We evaluated the impact of a ‘Team-Based Goals and Incentives’ (TBGI) intervention in Bihar, India, designed to improve front-line (community health) worker (FLW) performance and health-promoting behaviours related to reproductive, maternal, newborn and child health and nutrition.

Methods

This study used a cluster randomised controlled trial design and difference-in-difference analyses of improvements in maternal health-related behaviours related to the intervention’s team-based goals (primary), and interactions of FLWs with each other and with maternal beneficiaries (secondary). Evaluation participants included approximately 1300 FLWs and 3600 mothers at baseline (May to June 2012) and after 2.5 years of implementation (November to December 2014) who had delivered an infant in the previous year.

Results

The TBGI intervention resulted in significant increases in the frequency of antenatal home visits (15 absolute percentage points (PP), p=0.03) and receipt of iron-folic acid (IFA) tablets (7 PP, p=0.02), but non-significant changes in other health behaviours related to the trial’s goals. Improvements were seen in selected attitudes related to coordination and teamwork among FLWs, and in the provision of advice to beneficiaries (ranging from 8 to 14 PP) related to IFA, cord care, breast feeding, complementary feeding and family planning.

Conclusion

Results suggest that combining an integrated set of team-based coverage goals and targets, small non-cash incentives for teams who meet targets and team building to motivate FLWs resulted in improvements in FLW coordination and teamwork, and in the quality and quantity of FLW–beneficiary interactions. These improvements represent programmatically meaningful steps towards improving health behaviours and outcomes.

Trial registration number

NCT03406221

Evidence gap map of performance measurement and management in primary healthcare systems in low-income and middle-income countries

Introduction

We mapped available evidence on performance measurement and management (PMM) strategies in primary healthcare (PHC) systems of low-income and middle-income countries (LMICs). Widely used, their effectiveness remains inconclusive. This evidence gap map characterises existing research and evidence gaps.

Methods

Systematic mapping of performance measurement and management research in LMICs from 2000 to mid-2018; literature searches of seven academic databases and institutional repositories of impact evaluations and systematic reviews. Using a combination of manual screening and machine learning, four reviewers appraised 38 088 titles and abstracts, and extracted metadata from 137 impact evaluations and 18 systematic reviews that met the inclusion criteria. The resulting visual representation of the evidence base was uploaded to a web-based platform.

Results

Since 2000, the number of studies has increased; the first systematic reviews were completed in 2010. Two-thirds of the studies were conducted in sub-Saharan Africa and South Asia. Randomised controlled trials were the most frequently used study design. The evidence is concentrated in two types of PMM strategies: implementation strategies (in-service training, continuing education, supervision) and performance-based financing. Major gaps exist in accountability arrangements particularly the use of audit and feedback. The least studied types of outcomes were unintended effects, harm and social equity.

Conclusions

The evidence is clustered around interventions that are unlikely to achieve transformational change in health outcomes. The gaps identified suggest that routinely used PMM strategies are implemented without sufficient knowledge of their effects. Future efforts at redesigning PHC systems need to be informed by evidence on the most effective approaches for using PMM strategies.

Evidence gap map of performance measurement and management in primary care delivery systems in low- and middle-income countries - Study protocol

Background . For the last two decades there has been growing interest in governmental and global health stakeholders about the role that performance measurement and management systems can play for the production of high-quality and safely delivered primary care services. Despite recognition and interest, the gaps in evidence in this field of research and practice in low- and middle-income countries remain poorly characterized. This study will develop an evidence gap map in the area of performance management in primary care delivery systems in low- and middle-income countries. Methods. The evidence gap map will follow the methodology developed by 3Ie, the International Initiative for Impact Evaluation, to systematically map evidence and research gaps. The process starts with the development of the scope by creating an evidence-informed framework that helps identify the interventions and outcomes of relevance as well as help define inclusion and exclusion criteria. A search strategy is then developed to guide the systematic search of the literature, covering the following databases: Medline (Ovid), Embase (Ovid), CAB Global Health (Ovid), CINAHL (Ebsco), Cochrane Library, Scopus (Elsevier), and Econlit (Ovid). Sources of grey literature are also searched. Studies that meet the inclusion criteria are systematically coded, extracting data on intervention, outcome, measures, context, geography, equity, and study design. Systematic reviews are also critically appraised using an existing standard checklist. Impact evaluations are not appraised but will be coded according to study design. The process of map-building ends with the creation of an evidence gap map graphic that displays the available evidence according to the intervention and outcome framework of interest. Discussion . Implications arising from the evidence map will be discussed in a separate paper that will summarize findings and make recommendations for the development of a prioritized research agenda.