Measuring coverage of essential maternal and newborn care interventions: An unfinished agenda to define the data matrix for action in maternal and newborn health

Multisite, mixed methods study to validate 10 maternal health system and policy indicators in Argentina, Ghana and India: a research protocol

INTRODUCTION

Most efforts to assess maternal health indicator validity focus on measures of service coverage. Fewer measures focus on the upstream enabling environment, and such measures are typically not research validated. Thus, methods for validating system and policy-level indicators are not well described. This protocol describes original multicountry research to be conducted in Argentina, Ghana and India, to validate 10 indicators from the monitoring framework for the 'Strategies toward Ending Preventable Maternal Mortality' (EPMM). The overall aim is to improve capacity to drive and track progress towards achieving the priority recommendations in the EPMM strategies. This work is expected to contribute new knowledge on validation methodology and reveal important information about the indicators under study and the phenomena they target for monitoring. Validating the indicators in three diverse settings will explore the external validity of results.

METHODS AND ANALYSIS

This observational study explores the validity of 10 indicators from the EPMM monitoring framework via seven discrete validation exercises that will use mixed methods: (1) cross-sectional review of policy data, (2) retrospective review of facility-level patient and administrative data and (3) collection of primary quantitative and qualitative cross-sectional data from health service providers and clients. There is a specific methodological approach and analytic plan for each indicator, directed by unique, relevant validation research questions.

ETHICS AND DISSEMINATION

The protocol was approved by the Office of Human Research Administration at Harvard University in November 2019. Individual study sites received approval via local institutional review boards by January 2020 except La Pampa, Argentina, approved June 2020. Our dissemination plan enables unrestricted access and reuse of all published research, including data sets. We expect to publish at least one peer-reviewed publication per validation exercise. We will disseminate results at conferences and engage local stakeholders in dissemination activities in each study country.

Can We Use Routine Data for Strategic Decision Making? A Time Trend Comparison Between Survey and Routine Data in Mali

Routine data, which is available more regularly than the "gold standard" survey data, can be used to inform programmatic decisions in Mali at the national level. However, caution must be used if using data at a subnational level.

Background:

Countries with scarce resources need timely and high-quality data on coverage of health interventions to make strategic decisions about where to allocate investments in health. Household survey data are generally regarded as “gold standard,” high-quality data. This study assessed the comparability of intervention coverage time trends from routine and survey data at national and subnational levels in Mali.

Methods:

We compared 3 coverage indicators: contraceptive prevalence rate, institutional delivery, and 3 doses of diphtheria, pertussis, and tetanus (DPT3) vaccine, using 3 Mali Demographic and Health Surveys (DHS 2001, 2006, and 2012–2013) and routine health system data covering 2001–2012. For routine data, we used local health information system (HIS) annual reports and an HIS database. To compare time trends between the data sources, we calculated the percentage point change and 95% confidence interval from 2001–2006 and 2006–2012. We then computed the absolute and relative differences between the 2 data sources for each indicator over time at national and regional levels and assessed their level of significance.

Results:

The direction and magnitude of the time trends of contraceptive prevalence rate, institutional delivery, and DPT3 vaccine from 2001 to 2012 were similar at the national level between data sources. At the regional level, there were significant differences in the magnitude and direction of time trends for institutional delivery and the DPT3 vaccine; contraceptive prevalence trends were more consistent. Routine data tended to overestimate DPT3 coverage, and underestimate institutional delivery and contraceptive prevalence relative to survey data.

Conclusion:

Routine data in Mali—particularly at the national level—appear to be appropriate for use to inform program planning and prioritization, but routine time trends should be interpreted with caution at the subnational level. For program evaluations, routine data may not be appropriate to draw accurate inferences about program impact.

Quality of routine facility data for monitoring priority maternal and newborn indicators in DHIS2: A case study from Gombe State, Nigeria

INTRODUCTION

Routine health information systems are critical for monitoring service delivery. District Heath Information System, version 2 (DHIS2) is an open source software platform used in more than 60 countries, on which global initiatives increasingly rely for such monitoring. We used facility-reported data in DHIS2 for Gombe State, north-eastern Nigeria, to present a case study of data quality to monitor priority maternal and neonatal health indicators.

METHODS

For all health facilities in DHIS2 offering antenatal and postnatal care services (n = 497) and labor and delivery services (n = 486), we assessed the quality of data for July 2016-June 2017 according to the World Health Organization data quality review guidance. Using data from DHIS2 as well as external facility-level and population-level household surveys, we reviewed three data quality dimensions-completeness and timeliness, internal consistency, and external consistency-and considered the opportunities for improvement.

RESULTS

Of 14 priority maternal and neonatal health indicators that could be tracked through facility-based data, 12 were included in Gombe's DHIS2. During July 2016-June 2017, facility-reported data in DHIS2 were incomplete at least 40% of the time, under-reported 10%-60% of the events documented in facility registers, and showed inconsistencies over time, between related indicators, and with an external data source. The best quality data elements were those that aligned with Gombe's health program priorities, particularly older health programs, and those that reflected contact indicators rather than indicators related to the provision of commodities or content of care.

CONCLUSION

This case study from Gombe State, Nigeria, demonstrates the high potential for effective monitoring of maternal and neonatal health using DHIS2. However, coordinated action at multiple levels of the health system is needed to maximize reporting of existing data; rationalize data flow; routinize data quality review, feedback, and supervision; and ensure ongoing maintenance of DHIS2.

Measures matter: A scoping review of maternal and newborn indicators

BACKGROUND

A variety of global-level monitoring initiatives have recommended indicators for tracking progress in maternal and newborn health. As a first step supporting the work of WHO's Mother and Newborn Information for Tracking Outcomes and Results (MoNITOR) Technical Advisory Group, we aimed to compile and synthesize recommended indicators in order to document the landscape of maternal and newborn measurement and monitoring.

METHODS

We conducted a scoping review of indicators proposed by global multi-stakeholder groups to suggest next steps to further support maternal and newborn measurement and monitoring. Indicators pertaining to pregnancy, childbirth, and postpartum/postnatal and newborn care were extracted and included in the indicator compilation, together with key indicator metadata. We examined patterns and relationships across the compiled indicators.

RESULTS

We identified 140 indicators linked to maternal and newborn health topics across the continuum of service provision. Fifty-five indicators relate to inputs and processes, 30 indicators relate to outputs, outcomes comprise 37 indicators in the database, and 18 impact indicators. A quarter of indicators proposed by global groups is either under development/discussion or is considered "aspirational", highlighting the currently evolving monitoring landscape. Although considerable efforts have been made to harmonize indicator recommendations, there are still relatively few indicators shared across key monitoring initiatives and some of those that are shared may have definitional variation.

CONCLUSION

Rapid, wide-ranging work by a number of multi-stakeholder groups has resulted in a substantial number of indicators, many of which partially overlap and many are not supported with adequate documentation or guidance. The volume of indicators, coupled with the number of initiatives promoting different indicator lists, highlight the need for strengthened coordination and technical leadership to harmonize recommendations for improved measurement and monitoring of data related to maternal and newborn heath.