Using health-facility data to assess subnational coverage of maternal and child health indicators, Kenya

Evolving narratives on signal functions for monitoring maternal and newborn health services: A meta-narrative inspired review

Emergency obstetric care (EmOC) signal functions are a shortlist of key clinical interventions capable of averting deaths from the five main direct causes of maternal mortality; they have been used since 1997 as a part of an EmOC monitoring framework to track the availability of EmOC services in low- and middle-income settings. Their widespread use and proposed adaptation to include other types of care, such as care for newborns, is testimony to their legacy as part of the measurement architecture within reproductive health. Yet, much has changed in the landscape of maternal and newborn health (MNH) since the initial introduction of EmOC signal functions. As part of a project to revise the EmOC monitoring framework, we carried out a meta-narrative inspired review to reflect on how signal functions have been developed and conceptualised over the past two decades, and how different narratives, which have emerged alongside the evolving MNH landscape, have played a role in the conceptualisation of the signal function measurement. We identified three overarching narrative traditions: 1) clinical 2) health systems and 3) human rights, that dominated the discourse and critique around the use of signal functions. Through an iterative synthesis process including 19 final articles selected for the review, we explored patterns of conciliation and areas of contradiction between the three narrative traditions. We summarised five meta-themes around the use of signal functions: i) framing the boundaries; ii) moving beyond clinical capability; iii) capturing the woods versus the trees; iv) grouping signal functions and v) measurement challenges. We intend for this review to contribute to a better understanding of the discourses around signal functions, and to provide insight for the future roles of this monitoring approach for emergency obstetric and newborn care.

Quality of routine health facility data for monitoring maternal, newborn and child health indicators: A desk review of DHIS2 data in Lumbini Province, Nepal

INTRODUCTION

Health-facility data serves as a primary source for monitoring service provision and guiding the attainment of health targets. District Health Information Software (DHIS2) is a free open software predominantly used in low and middle-income countries to manage the facility-based data and monitor program wise service delivery. Evidence suggests the lack of quality in the routine maternal and child health information, however there is no robust analysis to evaluate the extent of its inaccuracy. We aim to bridge this gap by accessing the quality of DHIS2 data reported by health facilities to monitor priority maternal, newborn and child health indicators in Lumbini Province, Nepal.

METHODS

A facility-based descriptive study design involving desk review of Maternal, Neonatal and Child Health (MNCH) data was used. In 2021/22, DHIS2 contained a total of 12873 reports in safe motherhood, 12182 reports in immunization, 12673 reports in nutrition and 12568 reports in IMNCI program in Lumbini Province. Of those, monthly aggregated DHIS2 data were downloaded at one time and included 23 priority maternal and child health related data items. Of these 23 items, nine were chosen to assess consistency over time and identify outliers in reference years. Twelve items were selected to examine consistency between related data, while five items were chosen to assess the external consistency of coverage rates. We reviewed the completeness, timeliness and consistency of these data items and considered the prospects for improvement.

RESULTS

The overall completeness of facility reporting was found within 98% to 100% while timeliness of facility reporting ranged from 94% to 96% in each Maternal, Newborn and Child Health (MNCH) datasets. DHIS2 reported data for all 9 MNCH data items are consistent over time in 4 of 12 districts as all the selected data items are within ±33% difference from the provincial ratio. Of the eight MNCH data items assessed, four districts reported ≥5% monthly values that were moderate outliers in a reference year with no extreme outliers in any districts. Consistency between six-pairs of data items that are expected to show similar patterns are compared and found that three pairs are within ±10% of each other in all 12 districts. Comparison between the coverage rates of selected tracer indicators fall within ±33% of the DHS survey result.

CONCLUSION

Given the WHO data quality guidance and national benchmark, facilities in the Lumbini province well maintained the completeness and timeliness of MNCH datasets. Nevertheless, there is room for improvement in maintaining consistency over time, plausibility and predicted relationship of reported data. Encouraging the promotion of data review through the data management committee, strengthening the system inbuilt data validation mechanism in DHIS2, and promoting routine data quality assessment systems should be greatly encouraged.

Quality of reporting and trends of emergency obstetric and neonatal care indicators: an analysis from Tanzania district health information system data between 2016 and 2020

BACKGROUND

Routine health facility data provides the opportunity to monitor progress in quality and uptake of health care continuously. Our study aimed to assess the reliability and usefulness of emergency obstetric care data including temporal and regional variations over the past five years in Tanzania Mainland.

METHODS

Data were compiled from the routine monthly district reports compiled as part of the health management information systems for 2016-2020. Key indicators for maternal and neonatal care coverage, emergency obstetric and neonatal complications, and interventions indicators were computed. Assessment on reliability and consistency of reports was conducted and compared with annual rates and proportions over time, across the 26 regions in of Tanzania Mainland and by institutional delivery coverage.

RESULTS

Facility reporting was near complete with 98% in 2018-2020. Estimated population coverage of institutional births increased by 10% points from 71.2% to 2016 to 81.7% in 2020 in Tanzania Mainland, driven by increased use of dispensaries and health centres compared to hospitals. This trend was more pronounced in regions with lower institutional birth rates. The Caesarean section rate remained stable at around 10% of institutional births. Trends in the occurrence of complications such as antepartum haemorrhage, premature rupture of membranes, pre-eclampsia, eclampsia or post-partum bleeding were consistent over time but at low levels (1% of institutional births). Prophylactic uterotonics were provided to nearly all births while curative uterotonics were reported to be used in less than 10% of post-partum bleeding and retained placenta cases.

CONCLUSION

Our results show a mixed picture in terms of usefulness of the District Health Information System(DHIS2) data. Key indicators of institutional delivery and Caesarean section rates were plausible and provide useful information on regional disparities and trends. However, obstetric complications and several interventions were underreported thus diminishing the usefulness of these data for monitoring. Further research is needed on why complications and interventions to address them are not documented reliably.

Malaria vaccine coverage estimation using age-eligible populations and service user denominators in Kenya

BACKGROUND

The World Health Organization approved the RTS,S/AS01 malaria vaccine for wider rollout, and Kenya participated in a phased pilot implementation from 2019 to understand its impact under routine conditions. Vaccine delivery requires coverage measures at national and sub-national levels to evaluate progress over time. This study aimed to estimate the coverage of the RTS,S/AS01 vaccine during the first 36 months of the Kenyan pilot implementation.

METHODS

Monthly dose-specific immunization data for 23 sub-counties were obtained from routine health information systems at the facility level for 2019-2022. Coverage of each RTS,S/AS01 dose was determined using reported doses as a numerator and service-based (Penta 1 and Measles) or population (projected infant populations from WorldPop) as denominators. Descriptive statistics of vaccine delivery, dropout rates and coverage estimates were computed across the 36-month implementation period.

RESULTS

Over 36 months, 818,648 RTSS/AS01 doses were administered. Facilities managed by the Ministry of Health and faith-based organizations accounted for over 88% of all vaccines delivered. Overall, service-based malaria vaccine coverage was 96%, 87%, 78%, and 39% for doses 1-4 respectively. Using a population-derived denominator for age-eligible children, vaccine coverage was 78%, 68%, 57%, and 24% for doses 1-4, respectively. Of the children that received measles dose 1 vaccines delivered at 9 months (coverage: 95%), 82% received RTSS/AS01 dose 3, only 66% of children who received measles dose 2 at 18 months (coverage: 59%) also received dose 4.

CONCLUSION

The implementation programme successfully maintained high levels of coverage for the first three doses of RTSS/AS01 among children defined as EPI service users up to 9 months of age but had much lower coverage within the community with up to 1 in 5 children not receiving the vaccine. Consistent with vaccines delivered over the age of 1 year, coverage of the fourth malaria dose was low. Vaccine uptake, service access and dropout rates for malaria vaccines require constant monitoring and intervention to ensure maximum protection is conferred.

Improving health information system for malaria program management: Malaria Frontline Project lessons learned from Kano and Zamfara States, Nigeria, 2016-2019

The U.S. Centers for Disease Control and Prevention in collaboration with the National Malaria Elimination Program and the African Field Epidemiology Network established the Malaria Frontline Project to provide innovative approaches to improve the malaria program implementation in Kano and Zamfara States, Nigeria. Innovative approaches such as malaria bulletin, malaria monitoring wall chart, conduct of ward level data validation meetings and malaria dashboard have helped improve the use of data for decision making at all levels. Innovative approaches deployed during the project implementation facilitated data analysis and a better understanding of malaria program performance and data utilization for decision making at all levels. These innovative approaches may improve malaria control program performance in Nigeria and other resource limited countries.

Reduced incidence of respiratory, gastrointestinal and malaria infections among children during the COVID-19 pandemic in Western Kenya: An analysis of facility-based and weekly diaries data

Background

Epidemics can cause significant disruptions of essential health care services. This was evident in West-Africa during the 2014-2016 Ebola outbreak, raising concerns that COVID-19 would have similar devastating consequences for the continent. Indeed, official facility-based records show a reduction in health care visits after the onset of COVID-19 in Kenya. Our question is whether this observed reduction was caused by lower access to health care or by reduced incidence of communicable diseases resulting from reduced mobility and social contacts.

Methods

We analysed monthly facility-based data from 2018 to 2020, and weekly health diaries data digitally collected by trained fieldworkers between February and November 2020 from 342 households, including 1974 individuals, in Kisumu and Kakamega Counties, Kenya. Diaries data was collected as part of an ongoing longitudinal study of a digital health insurance scheme (Kakamega), and universal health coverage implementation (Kisumu). We assessed the weekly incidence of self-reported medical symptoms, formal and informal health-seeking behaviour, and foregone care in the diaries and compared it with facility-based records. Linear probability regressions with household fixed-effects were performed to compare the weekly incidence of health outcomes before and after COVID-19.

Results

Facility-based data showed a decrease in health care utilization for respiratory infections, enteric illnesses, and malaria, after start of COVID-19 measures in Kenya in March 2020. The weekly diaries confirmed this decrease in respiratory and enteric symptoms, and malaria / fever, mainly in the paediatric population. In terms of health care seeking behaviour, our diaries data find a temporary shift in consultations from health care centres to pharmacists / chemists / medicine vendors for a few weeks during the pandemic, but no increase in foregone care. According to the diaries, for adults the incidence of communicable diseases/symptoms rebounded after COVID-19 mobility restrictions were lifted, while for children the effects persisted.

Conclusions

COVID-19-related containment measures in Western Kenya were accompanied by a decline in respiratory infections, enteric illnesses, and malaria / fever mainly in children. Data from a population-based survey and facility-based records aligned regarding this finding despite the temporary shift to non-facility-based consultations and confirmed that the drop in utilization of health care services was not due to decreased accessibility, but rather to a lower incidence of these infections.

Investigating rapid diagnostic testing in Kenya's health system, 2018-2020: validating non-reporting in routine data using a health facility service assessment survey

BACKGROUND

Understanding the availability of rapid diagnostic tests (RDTs) is essential for attaining universal health care and reducing health inequalities. Although routine data helps measure RDT coverage and health access gaps, many healthcare facilities fail to report their monthly diagnostic test data to routine health systems, impacting routine data quality. This study sought to understand whether non-reporting by facilities is due to a lack of diagnostic and/or service provision capacity by triangulating routine and health service assessment survey data in Kenya.

METHODS

Routine facility-level data on RDT administration were sourced from the Kenya health information system for the years 2018-2020. Data on diagnostic capacity (RDT availability) and service provision (screening, diagnosis, and treatment) were obtained from a national health facility assessment conducted in 2018. The two sources were linked and compared obtaining information on 10 RDTs from both sources. The study then assessed reporting in the routine system among facilities with (i) diagnostic capacity only, (ii) both confirmed diagnostic capacity and service provision and (iii) without diagnostic capacity. Analyses were conducted nationally, disaggregated by RDT, facility level and ownership.

RESULTS

Twenty-one per cent (2821) of all facilities expected to report routine diagnostic data in Kenya were included in the triangulation. Most (86%) were primary-level facilities under public ownership (70%). Overall, survey response rates on diagnostic capacity were high (> 70%). Malaria and HIV had the highest response rate (> 96%) and the broadest coverage in diagnostic capacity across facilities (> 76%). Reporting among facilities with diagnostic capacity varied by test, with HIV and malaria having the lowest reporting rates, 58% and 52%, respectively, while the rest ranged between 69% and 85%. Among facilities with both service provision and diagnostic capacity, reporting ranged between 52% and 83% across tests. Public and secondary facilities had the highest reporting rates across all tests. A small proportion of health facilities without diagnostic capacity submitted testing reports in 2018, most of which were primary facilities.

CONCLUSION

Non-reporting in routine health systems is not always due to a lack of capacity. Further analyses are required to inform other drivers of non-reporting to ensure reliable routine health data.

Spatio-Temporal Bayesian Models for Malaria Risk Using Survey and Health Facility Routine Data in Rwanda

INTRODUCTION

Malaria is a life-threatening disease ocuring mainly in developing countries. Almost half of the world's population was at risk of malaria in 2020. Children under five years age are among the population groups at considerably higher risk of contracting malaria and developing severe disease. Most countries use Demographic and Health Survey (DHS) data for health programs and evaluation. However, malaria elimination strategies require a real-time, locally-tailored response based on malaria risk estimates at the lowest administrative levels. In this paper, we propose a two-step modeling framework using survey and routine data to improve estimates of malaria risk incidence in small areas and enable quantifying malaria trends.

METHODS

To improve estimates, we suggest an alternative approach to modeling malaria relative risk by combining information from survey and routine data through Bayesian spatio-temporal models. We model malaria risk using two steps: (1) fitting a binomial model to the survey data, and (2) extracting fitted values and using them in the Poison model as nonlinear effects in the routine data. We modeled malaria relative risk among under-five-year old children in Rwanda.

RESULTS

The estimation of malaria prevalence among children who are under five years old using Rwanda demographic and health survey data for the years 2019-2020 alone showed a higher prevalence in the southwest, central, and northeast of Rwanda than the rest of the country. Combining with routine health facility data, we detected clusters that were undetected based on the survey data alone. The proposed approach enabled spatial and temporal trend effect estimation of relative risk in local/small areas in Rwanda.

CONCLUSIONS

The findings of this analysis suggest that using DHS combined with routine health services data for active malaria surveillance may provide provide more precise estimates of the malaria burden, which can be used toward malaria elimination targets. We compared findings from geostatistical modeling of malaria prevalence among under-five-year old children using DHS 2019-2020 and findings from malaria relative risk spatio-temporal modeling using both DHS survey 2019-2020 and health facility routine data. The strength of routinely collected data at small scales and high-quality data from the survey contributed to a better understanding of the malaria relative risk at the subnational level in Rwanda.

Feasibility, usability and acceptability of a novel digital hybrid-system for reporting of routine maternal health information in Southern Tanzania: A mixed-methods study

Health information systems are important for health planning and progress monitoring. Still, data from health facilities are often of limited quality in Low-and-Middle-Income Countries. Quality deficits are partially rooted in the fact that paper-based documentation is still the norm at facility level, leading to mistakes in summarizing and manual copying. Digitization of data at facility level would allow automatization of these procedural steps. Here we aimed to evaluate the feasibility, usability and acceptability of a scanning innovation called Smart Paper Technology for digital data processing. We used a mixed-methods design to understand users' engagement with Smart Paper Technology and identify potential positive and negative effects of this innovation in three health facilities in Southern Tanzania. Eight focus group discussions and 11 in-depth interviews with users were conducted. We quantified time used by health care providers for documentation and patient care using time-motion methods. Thematic analysis was used to analyze qualitative data. Descriptive statistics and multivariable linear models were generated to compare the difference before and after introduction and adjust for confounders. Health care providers and health care managers appreciated the forms' simple design features and perceived Smart Paper Technology as time-saving and easy to use. The time-motion study with 273.3 and 224.0 hours of observations before and after introduction of Smart Paper Technology, respectively, confirmed that working time spent on documentation did not increase (27.0% at baseline and 26.4% post-introduction; adjusted p = 0.763). Time spent on patient care was not negatively impacted (26.9% at baseline and 37.1% at post-intervention; adjusted p = 0.001). Health care providers described positive effects on their accountability for data and service provision relating to the fact that individually signed forms were filled. Health care providers perceived Smart Paper Technology as feasible, easy to integrate and acceptable in their setting, particularly as it did not add time to documentation.

Geographic accessibility to public and private health facilities in Kenya in 2021: An updated geocoded inventory and spatial analysis

Objectives

To achieve universal health coverage, adequate geographic access to quality healthcare services is vital and should be characterized periodically to support planning. However, in Kenya, previous assessments of geographic accessibility have relied on public health facility lists only, assembled several years ago. Here, for the first time we assemble a geocoded list of public and private health facilities in 2021 and make use of this updated list to interrogate geographical accessibility to all health providers.

Methods

Existing health provider lists in Kenya were accessed, merged, cleaned, harmonized, and assigned a unique geospatial location. The resultant master list was combined with road network, land use, topography, travel barriers and healthcare-seeking behavior within a geospatial framework to estimate travel time to the nearest (i) private, (ii) public, and (iii) both (public and private-PP) health facilities through a travel scenario involving walking, bicycling and motorized transport. The proportion of the population within 1 h and outside 2-h was computed at 300 × 300 spatial resolution and aggregated at subnational units used for decision-making. Areas with a high disease prevalence for common infections that were outside 1-h catchment (dual burden) were also identified to guide prioritization.

Results

The combined database contained 13,579 health facilities, both in the public (55.5%) and private-for-profit sector (44.5%) in 2021. The private health facilities' distribution was skewed toward the urban counties. Nationally, average travel time to the nearest health facility was 130, 254, and 128 min while the population within 1-h was 89.4, 80.5, and 89.6% for the public, private and PP health facility, respectively. The population outside 2-h were 6% for public and PP and 11% for the private sector. Mean travel time across counties was heterogeneous, while the population within 1-h ranged between 38 and 100% in both the public sector and PP. Counties in northwest and southeast Kenya had a dual burden.

Conclusion

Continuous updating and geocoding of health facilities will facilitate an improved understanding of healthcare gaps for planning. Heterogeneities in geographical access continue to persist, with some areas having a dual burden and should be prioritized toward reducing health inequities and attaining universal health coverage.

Critical success factors for routine immunization performance: A case study of Zambia 2000 to 2018

Introduction

The essential components of a vaccine delivery system are well-documented, but robust evidence on how and why the related processes and implementation strategies prove effective at driving coverage is not well-established. To address this gap, we identified critical success factors associated with advancing key policies and programs that may have led to the substantial changes in routine childhood immunization coverage in Zambia between 2000 and 2018.

Methods

We identified Zambia as an exemplar in the delivery of childhood vaccines through analysis of DTP1 and DTP3 coverage data. Through interviews and focus group discussions at the national and subnational levels, we investigated factors that contributed to high and sustained vaccination coverage. We conducted a thematic analysis through application of implementation science frameworks to determine critical success factors. We triangulated these findings with quantitative analyses using publicly available data.

Results

The following success factors emerged: 1) the Inter-agency Coordinating Committee was strengthened for long-term engagement which, complemented by the Zambia Immunization Technical Advisory Group, is valued by the government and integrated into national-level decision-making; 2) the Ministry of Health improved the coordination of data collection and review for informed decision-making across all levels; 3) Regional multi-actor committees identified development priorities, strategies, and funding, and iteratively adjusted policies to account for facilitators, barriers, and lessons learned; 4) Vaccine messaging was disseminated through multiple channels, including the media and community leaders, increasing trust in the government by community members; 5) The Zambia Ministry of Health and Churches Health Association of Zambia formalized a long-term organizational relationship to leverage the strengths of faith-based organizations; and 6) Neighborhood Health Committees spearheaded community-driven strategies via community action planning and ultimately strengthened the link between communities and health facilities.

Conclusion

Broader health systems strengthening and strong partnerships between various levels of the government, communities, and external organizations were critical factors that accelerated vaccine coverage in Zambia. These partnerships were leveraged to strengthen the overall health system and healthcare governance.

Geospatial mapping of timely access to inpatient neonatal care and its relationship to neonatal mortality in Kenya

Globally, 2.4 million newborns die in the first month of life, with neonatal mortality rates (NMR) per 1,000 livebirths being highest in sub-Saharan Africa. Improving access to inpatient newborn care is necessary for reduction of neonatal deaths in the region. We explore the relationship between distance to inpatient hospital newborn care and neonatal mortality in Kenya. Data on service availability from numerous sources were used to map hospitals that care for newborns with very low birth weight (VLBW). Estimates of livebirths needing VLBW services were mapped from population census data at 100 m spatial resolution using a random forest algorithm and adjustments using a systematic review of livebirths needing these services. A cost distance algorithm that adjusted for proximity to roads, road speeds, land use and protected areas was used to define geographic access to hospitals offering VLBW services. County-level access metrics were then regressed against estimates of NMR to assess the contribution of geographic access to VLBW services on newborn deaths while controlling for wealth, maternal education and health workforce. 228 VLBW hospitals were mapped, with 29,729 births predicted as requiring VLBW services in 2019. Approximately 80.3% of these births were within 2 hours of the nearest VLBW hospital. Geographic access to these hospitals, ranged from less than 30% in Wajir and Turkana to as high as 80% in six counties. Regression analysis showed that a one percent increase in population within 2 hours of a VLBW hospital was associated with a reduction of NMR by 0.24. Despite access in the country being above the 80% threshold, 17/47 counties do not achieve this benchmark. To reduce inequities in NMR in Kenya, policies to improve care must reduce geographic barriers to access and progressively improve facilities' capacity to provide quality care for VLBW newborns.

Health service utilisation during the COVID-19 pandemic in sub-Saharan Africa in 2020: a multicountry empirical assessment with a focus on maternal, newborn and child health services

INTRODUCTION

There are concerns about the impact of the COVID-19 pandemic on the continuation of essential health services in sub-Saharan Africa. Through the Countdown to 2030 for Women's, Children's and Adolescents' Health country collaborations, analysts from country and global public health institutions and ministries of health assessed the trends in selected services for maternal, newborn and child health, general service utilisation.

METHODS

Monthly routine health facility data by district for the period 2017-2020 were compiled by 12 country teams and adjusted after extensive quality assessments. Mixed effects linear regressions were used to estimate the size of any change in service utilisation for each month from March to December 2020 and for the whole COVID-19 period in 2020.

RESULTS

The completeness of reporting of health facilities was high in 2020 (median of 12 countries, 96% national and 91% of districts ≥90%), higher than in the preceding years and extreme outliers were few. The country median reduction in utilisation of nine health services for the whole period March-December 2020 was 3.9% (range: -8.2 to 2.4). The greatest reductions were observed for inpatient admissions (median=-17.0%) and outpatient admissions (median=-7.1%), while antenatal, delivery care and immunisation services generally had smaller reductions (median from -2% to -6%). Eastern African countries had greater reductions than those in West Africa, and rural districts were slightly more affected than urban districts. The greatest drop in services was observed for March-June 2020 for general services, when the response was strongest as measured by a stringency index.

CONCLUSION

The district health facility reports provide a solid basis for trend assessment after extensive data quality assessment and adjustment. Even the modest negative impact on service utilisation observed in most countries will require major efforts, supported by the international partners, to maintain progress towards the SDG health targets by 2030.

Innovative approach to improve information accuracy in a two-district cross-sectional study in Bihar, India

OBJECTIVE

Combine Health Management Information Systems (HMIS) and probability survey data using the statistical annealing technique (AT) to produce more accurate health coverage estimates than either source of data and a measure of HMIS data error.

SETTING

This study is set in Bihar, the fifth poorest state in India, where half the population lives below the poverty line. An important source of data, used by health professionals for programme decision making, is routine health facility or HMIS data. Its quality is sometimes poor or unknown, and has no measure of its uncertainty. Using AT, we combine district-level HMIS and probability survey data (n=475) for the first time for 10 indicators assessing antenatal care, institutional delivery and neonatal care from 11 blocks of Aurangabad and 14 blocks of Gopalganj districts (N=6 253 965) in Bihar state, India.

PARTICIPANTS

Both districts are rural. Bihar is 82.7% Hindu and 16.9% Islamic.

PRIMARY OUTCOME MEASURES

Survey prevalence measures for 10 indicators, corresponding prevalences using HMIS data, combined prevalences calculated with AT and SEs for each type of data.

RESULTS

The combined and survey estimates differ by <0.10. The combined and HMIS estimates differ by up to 84.2%, with the HMIS having 1.4-32.3 times larger error. Of 20 HMIS versus survey coverage estimate comparisons across the two districts only five differed by <0.10. Of 250 subdistrict-level comparisons of HMIS versus combined estimates, only 36.4% of the HMIS estimates are within the 95% CI of the combined estimate.

CONCLUSIONS

Our statistical innovation increases the accuracy of information available for local health system decision making, allows evaluation of indicator accuracy and increases the accuracy of HMIS estimates. The combined estimates with a measure of error better informs health system professionals about their risks when using HMIS estimates, so they can reduce waste by making better decisions. Our results show that AT is an effective method ready for additional international assessment while also being used to provide affordable information to improve health services.

Can We Use Routine Data for Strategic Decision Making? A Time Trend Comparison Between Survey and Routine Data in Mali

Routine data, which is available more regularly than the "gold standard" survey data, can be used to inform programmatic decisions in Mali at the national level. However, caution must be used if using data at a subnational level.

Background:

Countries with scarce resources need timely and high-quality data on coverage of health interventions to make strategic decisions about where to allocate investments in health. Household survey data are generally regarded as “gold standard,” high-quality data. This study assessed the comparability of intervention coverage time trends from routine and survey data at national and subnational levels in Mali.

Methods:

We compared 3 coverage indicators: contraceptive prevalence rate, institutional delivery, and 3 doses of diphtheria, pertussis, and tetanus (DPT3) vaccine, using 3 Mali Demographic and Health Surveys (DHS 2001, 2006, and 2012–2013) and routine health system data covering 2001–2012. For routine data, we used local health information system (HIS) annual reports and an HIS database. To compare time trends between the data sources, we calculated the percentage point change and 95% confidence interval from 2001–2006 and 2006–2012. We then computed the absolute and relative differences between the 2 data sources for each indicator over time at national and regional levels and assessed their level of significance.

Results:

The direction and magnitude of the time trends of contraceptive prevalence rate, institutional delivery, and DPT3 vaccine from 2001 to 2012 were similar at the national level between data sources. At the regional level, there were significant differences in the magnitude and direction of time trends for institutional delivery and the DPT3 vaccine; contraceptive prevalence trends were more consistent. Routine data tended to overestimate DPT3 coverage, and underestimate institutional delivery and contraceptive prevalence relative to survey data.

Conclusion:

Routine data in Mali—particularly at the national level—appear to be appropriate for use to inform program planning and prioritization, but routine time trends should be interpreted with caution at the subnational level. For program evaluations, routine data may not be appropriate to draw accurate inferences about program impact.

Joint spatio-temporal modelling of adverse pregnancy outcomes sharing common risk factors at sub-county level in Kenya, 2016-2019

Background

Adverse pregnancy outcomes jointly account for a high proportion of mortality and morbidity among pregnant women and their infants. Furthermore, the burden attributed to adverse pregnancy outcomes remains high and inadequately characterised due to the intricate interplay of its etiology and shared set of important risk factors. This study sought to quantify and map the underlying risk of multiple adverse pregnancy outcomes in Kenya at sub-county level using a shared component space-time modelling framework.

Methods

Reported sub-county level adverse pregnancy outcomes count from January 2016 – December 2019 were obtained from the Kenyan District Health Information System. A Bayesian hierarchical spatio-temporal model was used to estimate the joint burden of adverse pregnancy outcomes in space (sub-county) and time (year). To improve the precision of our estimates over time and space, information across the outcomes were combined via the shared and the outcome-specific components using a shared component model with spatio-temporal interactions.

Results

Overall, the total number of adverse outcomes in pregnancy increased by 14.2% (95% UI: 14.0–14.5) from 88,816 cases in 2016 to 101,455 cases in 2019. Between 2016 and 2019, the estimated low birth weight rate and the pre-term birth rate were 4.5 (95% UI: 4.4–4.7) and 2.3 (95% UI: 2.2–2.5) per 100 live births. The stillbirth and neonatal death rates were estimated to be 18.7 (95% UI: 18.0–19.4) and 6.9 (95% UI: 6.4–7.4) per 1000 live births. The magnitude of the spatio-temporal variation attributed to shared risk was high for pre-term births, low birth weight, neonatal deaths, stillbirths and neonatal deaths, respectively. The shared risk patterns were dominant in sub-counties located along the Indian ocean coastline, central and western Kenya.

Conclusions

This study demonstrates the usefulness of a Bayesian joint spatio-temporal shared component model in exploiting specific and shared risk of adverse pregnancy outcomes sub-nationally. By identifying sub-counties with elevated risks and data gaps, our estimates not only assert the need for bolstering maternal health programs in the identified high-risk sub-counties but also provides a baseline against which to assess the progress towards the attainment of Sustainable Development Goals.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-021-12210-9.

Estimating population-based coverage of reproductive, maternal, newborn, and child health (RMNCH) interventions from health management information systems: a comprehensive review

Background

Routinely collected health facility data usually captured and stored in Health Management Information Systems (HMIS) are potential sources of data for frequent and local disaggregated estimation of the coverage of reproductive, maternal, newborn, and child health interventions (RMNCH), but have been under-utilized due to concerns over data quality. We reviewed methods for estimation of national or subnational coverage of RMNCH interventions using HMIS data exclusively or in conjunction with survey data from low- and middle-income countries (LMICs).

Methods

We conducted a comprehensive review of studies indexed in PubMed and Scopus to identify potential papers based on predefined search terms. Two reviewers screened the papers using defined inclusion and exclusion criteria. Following sequences of title, abstract and full paper reviews, we retained 18 relevant papers.

Results

12 papers used only HMIS data and 6 used both HMIS and survey data. There is enormous lack of standards in the existing methods for estimating RMNCH intervention coverage; all appearing to be highly author dependent. The denominators for coverage measures were estimated using census, non-census and combined projection-based methods. No satisfactory methods were found for treatment-based coverage indicators for which the estimation of target population requires the population prevalence of underlying conditions. The estimates of numerators for the coverage measures were obtained from the count of users or visits and in some cases correction for completeness of reporting in the HMIS following an assessment of data quality.

Conclusions

Standard methods for correcting numerators from HMIS data for accurate estimation of coverage of RMNCH interventions are needed to expand the use of these data. More research and investments are required to improve denominators for health facility-derived statistics. Improvement in routine data quality and analytical methods would allow for timely estimation of RMNCH intervention coverage at the national and subnational levels.

Measuring coverage of maternal and child health services using routine health facility data: a Sierra Leone case study

BACKGROUND

There are limited existing approaches to generate estimates from Routine Health Information Systems (RHIS) data, despite the growing interest to these data. We calculated and assessed the consistency of maternal and child health service coverage estimates from RHIS data, using census-based and health service-based denominators in Sierra Leone.

METHODS

We used Sierra Leone 2016 RHIS data to calculate coverage of first antenatal care contact (ANC1), institutional delivery and diphtheria-pertussis-tetanus 3 (DPT3) immunization service provision. For each indicator, national and district level coverages were calculated using denominators derived from two census-based and three health service-based methods. We compared the coverage estimates from RHIS data to estimates from MICS 2017. We considered the agreement adequate when estimates from RHIS fell within the 95% confidence interval of the survey estimate.

RESULTS

We found an overall poor consistency of the coverage estimates calculated from the census-based methods. ANC1 and institutional delivery coverage estimates from these methods were greater than 100% in about half of the fourteen districts, and only 3 of the 14 districts had estimates consistent with the survey data. Health service-based methods generated better estimates. For institutional delivery coverage, five districts met the agreement criteria using BCG service-based method. We found better agreement for DPT3 coverage estimates using DPT1 service-based method as national coverage was close to survey data, and estimates were consistent for 8 out of 14 districts. DPT3 estimates were consistent in almost half of the districts (6/14) using ANC1 service-based method.

CONCLUSION

The study highlighted the challenge in determining an appropriate denominator for RHIS-based coverage estimates. Systematic and transparent data quality check and correction, as well as rigorous approaches to determining denominators are key considerations to generate accurate coverage statistics using RHIS data.

Can routine health facility data be used to monitor subnational coverage of maternal, newborn and child health services in Uganda?

Background

Routine health facility data are a critical source of local monitoring of progress and performance at the subnational level. Uganda has been using district health statistics from facility data for many years. We aimed to systematically assess data quality and examine different methods to obtain plausible subnational estimates of coverage for maternal, newborn and child health interventions.

Methods

Annual data from the Uganda routine health facility information system 2015–2019 for all 135 districts were used, as well as national surveys for external comparison and the identification of near-universal coverage interventions. The quality of reported data on antenatal and delivery care and child immunization was assessed through completeness of facility reporting, presence of extreme outliers and internal data consistencies. Adjustments were made when necessary. The denominators for the coverage indicators were derived from population projections and health facility data on near-universal coverage interventions. The coverage results with different denominators were compared with the results from household surveys.

Results

Uganda’s completeness of reporting by facilities was near 100% and extreme outliers were rare. Inconsistencies in reported events, measured by annual fluctuations and between intervention consistency, were common and more among the 135 districts than the 15 subregions. The reported numbers of vaccinations were improbably high compared to the projected population of births or first antenatal visits – and especially so in 2015–2016. There were also inconsistencies between the population projections and the expected target population based on reported numbers of antenatal visits or immunizations. An alternative approach with denominators derived from facility data gave results that were more plausible and more consistent with survey results than based on population projections, although inconsistent results remained for substantive number of subregions and districts.

Conclusion

Our systematic assessment of the quality of routine reports of key events and denominators shows that computation of district health statistics is possible with transparent adjustments and methods, providing a general idea of levels and trends for most districts and subregions, but that improvements in data quality are essential to obtain more accurate monitoring.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06554-6.

The use of health facility data to assess the effects of armed conflicts on maternal and child health: experience from the Kivu, DR Congo

Background

In conflict-affected settings, data on reproductive, maternal, newborn and child health (RMNCH) are often lacking for priority setting and timely decision-making. We aimed to describe the levels and trends in RMNCH indicators within Kivu provinces between 2015 and 2018, by linking conflict data with health facility (HF) data from the District Health Information System 2 (DHIS2).

Methods

We used data from the DHIS2 for the period 2015–2018, the 2014 Demographic and Health Survey, the 2018 Multiple Indicators Cluster Survey and the Uppsala Conflict Data Program. Health zones were categorised in low, moderate and high conflict intensity level, based on an annual conflict death rate. We additionally defined a monthly conflict death rate and a conflict event-days rate as measures of conflict intensity and insecurity. Outcomes were completion of four antenatal care visits, health facility deliveries, caesarean sections and pentavalent vaccine coverage. We assessed data quality and analyzed coverage and trends in RMNCH indicators graphically, by conflict categories and using HF data aggregated annually. We used a series of fixed-effect regression models to examine the potential dose-response effect of varying conflict intensity and insecurity on RMNCH.

Results

The overall HF reporting was good, ranging between 83.3 and 93.2% and tending to be lower in health zones with high conflict intensity in 2016 and 2017 before converging in 2018. Despite the increasing number of conflict-affected health zones over time, more in North-Kivu than in South-Kivu, we could not identify any clear pattern of variation in RMNCH coverage both by conflict intensity and insecurity. North-Kivu province had consistently reported better RMNCH indicators than South-Kivu, despite being more affected by conflict. The Kivu as a whole recorded higher coverage than the national level. Coverage of RMNCH services calculated from HF data was consistent with population-based surveys, despite year-to-year fluctuation among health zones and across conflict-intensity categories.

Conclusions

Although good in general, the HF reporting rate in the Kivu was negatively impacted by conflict intensity especially at the beginning of the DHIS2’s rolling-up. Routine HF data appeared useful for assessing and monitoring trends in RMNCH service coverage, including in areas with high-intensity conflict.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06143-7.

Establishing evidence-based decision-making mechanism in a health eco-system and its linkages with health service coverage in 25 high-priority districts of Uttar Pradesh, India

Background

Achievement of successful health outcomes depends on evidence-based programming and implementation of effective health interventions. Routine Health Management Information System is one of the most valuable data sets to support evidence-based programming, however, evidence on systemic use of routine monitoring data for problem-solving and improving health outcomes remain negligible. We attempt to understand the effects of systematic evidence-based review mechanism on improving health outcomes in Uttar Pradesh, India.

Methods

Data comes from decision-tracking system and routine health management information system for period Nov-2017 to Mar-2019 covering 6963 health facilities across 25 high-priority districts of the state. Decision-tracking data captured pattern of decisions taken, actions planned and completed, while the latter one provided information on service coverage outcomes over time. Three service coverage indicators, namely, pregnant women receiving 4 or more times ANC and haemoglobin testing during pregnancy, delivered at the health facility, and receive post-partum care within 48 h of delivery were used as outcomes. Univariate and bivariate analyses were conducted.

Results

Total 412 decisions were taken during the study reference period and a majority were related to ante-natal care services (31%) followed by delivery (16%) and post-natal services (16%). About 21% decisions-taken were focused on improving data quality. By 1 year, 67% of actions planned based on these decisions were completed, 26% were in progress, and the remaining 7% were not completed. We found that, over a year, districts witnessing > 20 percentage-point increase in outcomes were also the districts with significantly higher action completion rates (> 80%) compared to the districts with < 10 percentage-point increase in outcomes having completion of action plans around 50–70%.

Conclusions

Findings revealed a significantly higher improvement in coverage outcomes among the districts which used routine health management data to conduct monthly review meetings and had high actions completion rates. A data-based review-mechanisms could specifically identify programmatic gaps in service delivery leading to strategic decision making by district authorities to bridge the programmatic gaps. Going forward, establishing systematic evidence-based review platforms can be an important strategy to improve health outcomes and promote the use of routine health monitoring system data in any setting.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06172-2.

Estimating malaria chemoprevention and vector control coverage using program and campaign data: A scoping review of current practices and opportunities

Background

Accurate estimation of intervention coverage is a vital component of malaria program monitoring and evaluation, both for process evaluation (how well program targets are achieved), and impact evaluation (whether intervention coverage had an impact on malaria burden). There is growing interest in maximizing the utility of program data to generate interim estimates of intervention coverage in the periods between large-scale cross-sectional surveys (the gold standard). As such, this study aimed to identify relevant concepts and themes that may guide future optimization of intervention coverage estimation using routinely collected data, or data collected during and following intervention campaigns, with a particular focus on strategies to define the denominator.

Methods

We conducted a scoping review of current practices to estimate malaria intervention coverage for insecticide-treated nets (ITNs); indoor residual spray (IRS); intermittent preventive treatment in pregnancy (IPTp); mass drug administration (MDA); and seasonal malaria chemoprevention (SMC) interventions; case management was excluded. Multiple databases were searched for relevant articles published from January 1, 2015 to June 1, 2018. Additionally, we identified and included other guidance relevant to estimating population denominators, with a focus on innovative techniques.

Results

While program data have the potential to provide intervention coverage data, there are still substantial challenges in selecting appropriate denominators. The review identified a lack of consistency in how coverage was defined and reported for each intervention type, with denominator estimation methods not clearly or consistently reported, and denominator estimates rarely triangulated with other data sources to present the feasible range of denominator values and consequently the range of likely coverage estimates.

Conclusions

Though household survey-based estimates of intervention coverage remain the gold standard, efforts should be made to further standardize practices for generating interim measurements of intervention coverage from program data, and for estimating and reporting population denominators. This includes fully describing any projections or adjustments made to existing census or population data, exploring opportunities to validate available data by comparing with other sources, and explaining how the denominator has been restricted (or not) to reflect exclusion criteria.

Development and testing of a composite index to monitor the continuum of maternal health service delivery at provincial and district level in South Africa

INTRODUCTION

The continuum of care is a recommended framework for comprehensive health service delivery for maternal health, and it integrates health system and social determinants of health. There is a current lack of knowledge on a measurement approach to monitor performance on the framework. In this study we aim to develop and test a composite index for assessing the maternal health continuum in a province in South Africa with the possibility of nationwide use.

MATERIALS AND METHODS

The composite index was computed as a geometric mean of four dimensions of adequacy of the continuum of care. Data was sourced from the district health information system, household surveys and the census. The index formula was tested for robustness when alternative inputs for indicators and standardization methods were used. The index was used to assess performance in service delivery in the North West province of South Africa, as well as its four districts over a five-year period (2013-2017). The index was validated by assessing associations with maternal health and other outcomes. And factor analysis was used to assess the statistical dimensions of the index.

RESULTS

The provincial level index score increased from 62.3 in 2013 to 74 in 2017, showing general improvement in service delivery over time. The district level scores also improved over time, and our analysis identified areas for performance improvement. These include social determinants of health in some districts, and access and linkages to care in others. The provincial index was correlated with institutional maternal mortality rates (rs = -0.90, 90% CI = (-1.00, -0.25)) and the Human Development Index (r = 0.97, 95% CI = (0.63, 0.99). It was robust to alternative approaches including z-score standardization of indicators. Factor analysis showed three groupings of indicators for the health system and social determinants of health.

CONCLUSIONS

This study demonstrated the development and testing of a composite index to monitor and assess service delivery on the continuum of care for maternal health. The index was shown to be robust and valid, and identified potential areas for service improvement. A contextualised version can be tested in other settings within and outside of South Africa.

Modelling geographical accessibility to urban centres in Kenya in 2019

BACKGROUND

Access to major services, often located in urban centres, is key to the realisation of numerous Sustainable Development Goals (SDGs). In Kenya, there are no up-to-date and localised estimates of spatial access to urban centres. We estimate the travel time to urban centres and identify marginalised populations for prioritisation and targeting.

METHODS

Urban centres were mapped from the 2019 Kenya population census and combined with spatial databases of road networks, elevation, land use and travel barriers within a cost-friction algorithm to compute travel time. Seven travel scenarios were considered: i) walking only (least optimistic), ii) bicycle only, iii) motorcycle only, iv) vehicle only (most optimistic), v) walking followed by motorcycle transport, vi) walking followed by vehicle transport, and vii) walking followed by motorcycle and then vehicle transport (most pragmatic). Mean travel time, and proportion of the population within 1-hour and 2-hours of the urban centres were summarized at sub-national units (counties) used for devolved planning. Inequities were explored and correlations between the proportion of the population within 1-hour of an urban centre and ten SDG indicators were computed.

RESULTS

A total of 307 urban centres were digitised. Nationally, the mean travel time was 4.5-hours for the walking-only scenario, 1.0-hours for the vehicle only (most optimistic) scenario and 1.5-hours for the walking-motorcycle-vehicle (most pragmatic) scenario. Forty-five per cent (21.3 million people) and 87% (41.6 million people) of Kenya's population resided within 1-hour of the nearest urban centre for the least optimistic and most pragmatic scenarios respectively. Over 3.2 million people were considered marginalised or living outside the 2-hour threshold in the pragmatic scenario, 16.0 million Kenyans for walking only, and 2.2 million for the most optimistic scenario. County-level spatial access was highly heterogeneous ranging between 8%-100% and 32%-100% of people within the 1-hour threshold for the least and most optimistic scenarios, respectively. Counties in northern and eastern parts of Kenya were generally most marginalised. The correlation coefficients for nine SDG indicators ranged between 0.45 to 0.78 and were statistically significant.

CONCLUSION

Travel time to urban centres in Kenya is heterogeneous. Therefore, marginalised populations should be prioritised during resource allocation and policies should be formulated to enhance equitable access to public services and opportunities in urban areas.

Subnational estimates of factors associated with under-five mortality in Kenya: a spatio-temporal analysis, 1993-2014

BACKGROUND

To improve child survival, it is necessary to describe and understand the spatial and temporal variation of factors associated with child survival beyond national aggregates, anchored at decentralised health planning units. Therefore, we aimed to provide subnational estimates of factors associated with child survival while elucidating areas of progress, stagnation and decline in Kenya.

METHODS

Twenty household surveys and three population censuses conducted since 1989 were assembled and spatially aligned to 47 subnational Kenyan county boundaries. Bayesian spatio-temporal Gaussian process regression models accounting for inadequate sample size and spatio-temporal relatedness were fitted for 43 factors at county level between 1993 and 2014.

RESULTS

Nationally, the coverage and prevalence were highly variable with 38 factors recording an improvement. The absolute percentage change (1993-2014) was heterogeneous ranging between 1% and 898%. At the county level, the estimates varied across space and over time with a majority showing improvements after 2008 which was preceded by a period of deterioration (late-1990 to early-2000). Counties in Northern Kenya were consistently observed to have lower coverage of interventions and remained disadvantaged in 2014 while areas around Central Kenya had and historically have had higher coverage across all intervention domains. Most factors in Western and South-East Kenya recorded moderate intervention coverage although having a high infection prevalence of both HIV and malaria.

CONCLUSION

The heterogeneous estimates necessitates prioritisation of the marginalised counties to achieve health equity and improve child survival uniformly across the country. Efforts are required to narrow the gap between counties across all the drivers of child survival. The generated estimates will facilitate improved benchmarking and establish a baseline for monitoring child development goals at subnational level.

Barriers and enablers to routine register data collection for newborns and mothers: EN-BIRTH multi-country validation study

BACKGROUND

Policymakers need regular high-quality coverage data on care around the time of birth to accelerate progress for ending preventable maternal and newborn deaths and stillbirths. With increasing facility births, routine Health Management Information System (HMIS) data have potential to track coverage. Identifying barriers and enablers faced by frontline health workers recording HMIS source data in registers is important to improve data for use.

METHODS

The EN-BIRTH study was a mixed-methods observational study in five hospitals in Bangladesh, Nepal and Tanzania to assess measurement validity for selected Every Newborn coverage indicators. We described data elements required in labour ward registers to track these indicators. To evaluate barriers and enablers for correct recording of data in registers, we designed three interview tools: a) semi-structured in-depth interview (IDI) guide b) semi-structured focus group discussion (FGD) guide, and c) checklist assessing care-to-documentation. We interviewed two groups of respondents (January 2018-March 2019): hospital nurse-midwives and doctors who fill ward registers after birth (n = 40 IDI and n = 5 FGD); and data collectors (n = 65). Qualitative data were analysed thematically by categorising pre-identified codes. Common emerging themes of barriers or enablers across all five hospitals were identified relating to three conceptual framework categories.

RESULTS

Similar themes emerged as both barriers and enablers. First, register design was recognised as crucial, yet perceived as complex, and not always standardised for necessary data elements. Second, register filling was performed by over-stretched nurse-midwives with variable training, limited supervision, and availability of logistical resources. Documentation complexity across parallel documents was time-consuming and delayed because of low staff numbers. Complete data were valued more than correct data. Third, use of register data included clinical handover and monthly reporting, but little feedback was given from data users.

CONCLUSION

Health workers invest major time recording register data for maternal and newborn core health indicators. Improving data quality requires standardised register designs streamlined to capture only necessary data elements. Consistent implementation processes are also needed. Two-way feedback between HMIS levels is critical to improve performance and accurately track progress towards agreed health goals.

Uterotonics for prevention of postpartum haemorrhage: EN-BIRTH multi-country validation study

BACKGROUND

Postpartum haemorrhage (PPH) is a leading cause of preventable maternal mortality worldwide. The World Health Organization (WHO) recommends uterotonic administration for every woman after birth to prevent PPH. There are no standardised data collected in large-scale measurement platforms. The Every Newborn Birth Indicators Research Tracking in Hospitals (EN-BIRTH) is an observational study to assess the validity of measurement of maternal and newborn indicators, and this paper reports findings regarding measurement of coverage and quality for uterotonics.

METHODS

The EN-BIRTH study took place in five hospitals in Bangladesh, Nepal and Tanzania, from July 2017 to July 2018. Clinical observers collected tablet-based, time-stamped data. We compared observation data for uterotonics to routine hospital register-records and women's report at exit-interview survey. We analysed the coverage and quality gap for timing and dose of administration. The register design was evaluated against gap analyses and qualitative interview data assessing the barriers and enablers to data recording and use.

RESULTS

Observed uterotonic coverage was high in all five hospitals (> 99%, 95% CI 98.7-99.8%). Survey-report underestimated coverage (79.5 to 91.7%). "Don't know" replies varied (2.1 to 14.4%) and were higher after caesarean (3.7 to 59.3%). Overall, there was low accuracy in survey data for details of uterotonic administration (type and timing). Register-recorded coverage varied in four hospitals capturing uterotonics in a specific column (21.6, 64.5, 97.6, 99.4%). The average coverage measurement gap was 18.1% for register-recorded and 6.0% for survey-reported coverage. Uterotonics were given to 15.9% of women within the "right time" (1 min) and 69.8% within 3 min. Women's report of knowing the purpose of uterotonics after birth ranged from 0.4 to 64.9% between hospitals. Enabling register design and adequate staffing were reported to improve routine recording.

CONCLUSIONS

Routine registers have potential to track uterotonic coverage - register data were highly accurate in two EN-BIRTH hospitals, compared to consistently underestimated coverage by survey-report. Although uterotonic coverage was high, there were gaps in observed quality for timing and dose. Standardisation of register design and implementation could improve data quality and data flow from registers into health management information reporting systems, and requires further assessment.

Neonatal mortality burden and trends in UNHCR refugee camps, 2006-2017: a retrospective analysis

Background

More than 100 million people were forcibly displaced over the last decade, including millions of refugees displaced across international borders. Although refugee health and well-being has gained increasing attention from researchers in recent years, few studies have examined refugee birth outcomes or newborn health on a regional or global scale. This study uses routine health information system data to examine neonatal mortality burden and trends in refugee camps between 2006 and 2017.

Methods

Refugee population and mortality data were exported from the United Nations High Commissioner for Refugees (UNHCR) Health Information System (HIS) database. Tableau was used to export the data. Stata was used for data cleaning and statistical analysis. Neonatal mortality burdens and trends in refugee camps were analyzed and compared to national and subnational neonatal mortality rates captured by household surveys.

Findings

One hundred fifty refugee camps in 21 countries were included in this study, with an average population of 1,725,433 between 2006 and 2017. A total of 663,892 live births and 3382 neonatal deaths were captured during this period. Annual country-level refugee camp neonatal mortality rates (NMR) ranged from 12 to 56 neonatal deaths per 1000 live births. In most countries and years where national population-based surveys are available, refugee camp NMR as reported in the UNHCR HIS was lower than that of the immediate host community.

Conclusion

The UNHCR HIS provides insights into the neonatal mortality burden among refugees in camp settings and issues to consider in design and use of routine health information systems to monitor neonatal health in sub-national populations. Increased visibility of neonatal deaths and stillbirths among displaced populations can drive advocacy and inform decisions needed to strengthen health systems. Efforts to count every stillbirth and neonatal death are critical, as well as improvements to reporting systems and mechanisms for data review within broader efforts to improve the quality of neonatal care practices within and outside of health facilities.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-021-10343-5.

WHO digital health packages for disseminating data standards and data use practices

OBJECTIVE

The aim of the study is to analyse an initiative from the World Health Organisation (WHO) to facilitate the dissemination of global standards and guidelines through digital health packages, which can strengthen the capacity of countries to monitor the SDGs. Digital health packages include data standards, guidance on data analysis, specifications for analytical dashboards for a specific health area, including a configuration of these standards for a widely used software platform.

METHODS

This is a multi-year case study, in which the authors have actively participated in the various aspects of development, implementation and evaluation of the digital health packages initiative.

RESULTS

We discuss the key innovations of the digital health package approach: First, the development process, which is based on flexible standards and an integrated approach across health programmes. Second, how the digital health packages combine several related types of standards into one package, including configurations for a widely used software platform supported by strong global and regional technical teams.

OUTCOME

This study shows a novel approach to dissemination of standards, through a digital platform. Currently, 40 countries have adopted one or more of the digital health packages in their national health information system.

Improving the quality of routine maternal and newborn data captured in primary health facilities in Gombe State, Northeastern Nigeria: a before-and-after study

OBJECTIVES

Primary objective: to assess nine data quality metrics for 14 maternal and newborn health data elements, following implementation of an integrated, district-focused data quality intervention.

SECONDARY OBJECTIVE

to consider whether assessing the data quality metrics beyond completeness and accuracy of facility reporting offered new insight into reviewing routine data quality.

DESIGN

Before-and-after study design.

SETTING

Primary health facilities in Gombe State, Northeastern Nigeria.

PARTICIPANTS

Monitoring and evaluation officers and maternal, newborn and child health coordinators for state-level and all 11 local government areas (district-equivalent) overseeing 492 primary care facilities offering maternal and newborn care services.

INTERVENTION

Between April 2017 and December 2018, we implemented an integrated data quality intervention which included: introduction of job aids and regular self-assessment of data quality, peer-review and feedback, learning workshops, work planning for improvement, and ongoing support through social media.

OUTCOME MEASURES

9 metrics for the data quality dimensions of completeness and timeliness, internal consistency of reported data, and external consistency.

RESULTS

The data quality intervention was associated with improvements in seven of nine data quality metrics assessed including availability and timeliness of reporting, completeness of data elements, accuracy of facility reporting, consistency between related data elements, and frequency of outliers reported. Improvement differed by data element type, with content of care and commodity-related data improving more than contact-related data. Increases in the consistency between related data elements demonstrated improved internal consistency within and across facility documentation.

CONCLUSIONS

An integrated district-focused data quality intervention-including regular self-assessment of data quality, peer-review and feedback, learning workshops, work planning for improvement, and ongoing support through social media-can increase the completeness, accuracy and internal consistency of facility-based routine data.

Mapping of anaemia prevalence among pregnant women in Kenya (2016-2019)

BACKGROUND

Reducing the burden of anaemia is a critical global health priority that could improve maternal outcomes amongst pregnant women and their neonates. As more counties in Kenya commit to universal health coverage, there is a growing need for optimal allocation of the limited resources to sustain the gains achieved with the devolution of healthcare services. This study aimed to describe the spatio-temporal patterns of maternal anaemia prevalence in Kenya from 2016 to 2019.

METHODS

Quarterly reported sub-county level maternal anaemia cases from January 2016 - December 2019 were obtained from the Kenyan District Health Information System. A Bayesian hierarchical negative binomial spatio-temporal conditional autoregressive (CAR) model was used to estimate maternal anaemia prevalence by sub-county and quarter. Spatial and temporal correlations were considered by assuming a conditional autoregressive and a first-order autoregressive process on sub-county and seasonal specific random effects, respectively.

RESULTS

The overall estimated number of pregnant women with anaemia increased by 90.1% (95% uncertainty interval [95% UI], 89.9-90.2) from 155,539 cases in 2016 to 295,642 cases 2019. Based on the WHO classification criteria, the proportion of sub-counties with normal prevalence decreased from 28.0% (95% UI, 25.4-30.7) in 2016 to 5.4% (95% UI, 4.1-6.7) in 2019, whereas moderate anaemia prevalence increased from 16.8% (95% UI, 14.7-19.1) in 2016 to 30.1% (95% UI, 27.5-32.8) in 2019 and severe anaemia prevalence increased from 7.0% (95% UI, 5.6-8.6) in 2016 to 16.6% (95% UI, 14.5-18.9) in 2019. Overall, 45.1% (95% UI: 45.0-45.2) of the estimated cases were in malaria-endemic sub-counties, with the coastal endemic zone having the highest proportion 72.8% (95% UI: 68.3-77.4) of sub-counties with severe prevalence.

CONCLUSION

As the number of women of reproductive age continues to grow in Kenya, the use of routinely collected data for accurate mapping of poor maternal outcomes remains an integral component of a functional maternal health strategy. By unmasking the sub-county disparities often concealed by national and county estimates, our study findings reiterate the importance of maternal anaemia prevalence as a metric for estimating malaria burden and offers compelling policy implications for achieving national nutritional targets.

Data cleaning process for HIV-indicator data extracted from DHIS2 national reporting system: a case study of Kenya

BACKGROUND

The District Health Information Software-2 (DHIS2) is widely used by countries for national-level aggregate reporting of health-data. To best leverage DHIS2 data for decision-making, countries need to ensure that data within their systems are of the highest quality. Comprehensive, systematic, and transparent data cleaning approaches form a core component of preparing DHIS2 data for analyses. Unfortunately, there is paucity of exhaustive and systematic descriptions of data cleaning processes employed on DHIS2-based data. The aim of this study was to report on methods and results of a systematic and replicable data cleaning approach applied on HIV-data gathered within DHIS2 from 2011 to 2018 in Kenya, for secondary analyses.

METHODS

Six programmatic area reports containing HIV-indicators were extracted from DHIS2 for all care facilities in all counties in Kenya from 2011 to 2018. Data variables extracted included reporting rate, reporting timeliness, and HIV-indicator data elements per facility per year. 93,179 facility-records from 11,446 health facilities were extracted from year 2011 to 2018. Van den Broeck et al.'s framework, involving repeated cycles of a three-phase process (data screening, data diagnosis and data treatment), was employed semi-automatically within a generic five-step data-cleaning sequence, which was developed and applied in cleaning the extracted data. Various quality issues were identified, and Friedman analysis of variance conducted to examine differences in distribution of records with selected issues across eight years.

RESULTS

Facility-records with no data accounted for 50.23% and were removed. Of the remaining, 0.03% had over 100% in reporting rates. Of facility-records with reporting data, 0.66% and 0.46% were retained for voluntary medical male circumcision and blood safety programmatic area reports respectively, given that few facilities submitted data or offered these services. Distribution of facility-records with selected quality issues varied significantly by programmatic area (p < 0.001). The final clean dataset obtained was suitable to be used for subsequent secondary analyses.

CONCLUSIONS

Comprehensive, systematic, and transparent reporting of cleaning-process is important for validity of the research studies as well as data utilization. The semi-automatic procedures used resulted in improved data quality for use in secondary analyses, which could not be secured by automated procedures solemnly.

Improving the quality and use of immunization and surveillance data: Summary report of the Working Group of the Strategic Advisory Group of Experts on Immunization

Concerns about the quality and use of immunization and vaccine-preventable disease (VPD) surveillance data have been highlighted on the global agenda for over two decades. In August 2017, the Strategic Advisory Group of Experts (SAGE) established a Working Group (WG) onthe Quality and Use of Global Immunization and Surveillance Data to review the current status and evidence to make recommendations, which were presented to SAGE in October 2019. The WG synthesized evidence from landscape analyses, literature reviews, country case-studies, a data triangulation analysis, as well as surveys of experts. Data quality (DQ) was defined as data that are accurate, precise, relevant, complete, and timely enough for the intended purpose (fit-for-purpose), and data use as the degree to which data are actually used for defined purposes, e.g., immunization programme management, performance monitoring, decision-making. The WG outlined roles and responsibilities for immunization and surveillance DQ and use by programme level. The WG found that while DQ is dependent on quality data collection at health facilities, many interventions have targeted national and subnational levels, or have focused on new technologies, rather than the people and enabling environments required for functional information systems. The WG concluded that sustainable improvements in immunization and surveillance DQ and use will require efforts across the health system - governance, people, tools, and processes, including use of data for continuous quality improvement (CQI) - and that the approaches need to be context-specific, country-owned and driven from the frontline up. At the country level, major efforts are needed to: (1) embed monitoring DQ and use alongside monitoring of immunization and surveillance performance, (2) increase workforce capacity and capability for DQ and use, starting at the facility level, (3) improve the accuracy of immunization programme targets (denominators), (4) enhance use of existing data for tailored programme action (e.g., immunization programme planning, management and policy-change), (5) adopt a data-driven CQI approach as part of health system strengthening, (6) strengthen governance around piloting and implementation of new information and communication technology tools, and (7) improve data sharing and knowledge management across areas and organizations for improved transparency and efficiency. Global and regional partners are requested to support countries in adopting relevant recommendations for their setting and to continue strengthening the reporting and monitoring of immunization and VPD surveillance data through processes periodic needs assessment and revision processes. This summary of the WG's findings and recommendations can support "data-guided" implementation of the new Immunization Agenda 2030.

Assessing coverage of essential maternal and child health interventions using health-facility data in Uganda

BACKGROUND

Nationally representative household surveys are the gold standard for tracking progress in coverage of life-saving maternal and child interventions, but often do not provide timely information on coverage at the local and health facility level. Electronic routine health information system (RHIS) data could help provide this information, but there are currently concerns about data quality. This analysis seeks to improve the usability of and confidence in electronic RHIS data by using adjustments to calculate more accurate numerators and denominators for essential interventions.

METHODS

Data from three sources (Ugandan Demographic and Health (UDHS) survey, electronic RHIS, and census) were used to provide estimates of essential maternal (> 4 antenatal care visits (ANC), skilled delivery, and postnatal care visit (PNC)) and child health interventions (diphtheria, pertussis, tetanus, and hepatitis B and Haemophilus influenzae type b and polio vaccination series, measles vaccination, and vitamin A). Electronic RHIS data was checked for quality and both numerators and denominators were adjusted to improve accuracy. Estimates were compared between the three sources.

RESULTS

Estimates of maternal health interventions from adjusted electronic RHIS data were lower than those of the UDHS, while child intervention estimates were typically higher. Adjustment of electronic RHIS data generally improved accuracy compared with no adjustment. There was considerable agreement between estimates from adjusted, electronic RHIS data, and UDHS for skilled delivery and first dose of childhood vaccination series, but lesser agreement for ANC visits and second and third doses of childhood vaccinations.

CONCLUSIONS

Nationally representative household surveys will likely continue being the gold standard of coverage estimates of maternal and child health interventions, but this analysis shows that current approaches to adjusting health facility estimate works better for some indications than others. Further efforts to improve accuracy of estimates from RHIS sources are needed.

Bayesian spatio-temporal modeling of malaria risk in Rwanda

Every year, 435,000 people worldwide die from Malaria, mainly in Africa and Asia. However, malaria is a curable and preventable disease. Most countries are developing malaria elimination plans to meet sustainable development goal three, target 3.3, which includes ending the epidemic of malaria by 2030. Rwanda, through the malaria strategic plan 2012-2018 set a target to reduce malaria incidence by 42% from 2012 to 2018. Assessing the health policy and taking a decision using the incidence rate approach is becoming more challenging. We are proposing suitable statistical methods that handle spatial structure and uncertainty on the relative risk that is relevant to National Malaria Control Program. We used a spatio-temporal model to estimate the excess probability for decision making at a small area on evaluating reduction of incidence. SIR and BYM models were developed using routine data from Health facilities for the period from 2012 to 2018 in Rwanda. The fitted model was used to generate relative risk (RR) estimates comparing the risk with the malaria risk in 2012, and to assess the probability of attaining the set target goal per area. The results showed an overall increase in malaria in 2013 to 2018 as compared to 2012. Ofall sectors in Rwanda, 47.36% failed to meet targeted reduction in incidence from 2012 to 2018. Our approach of using excess probability method to evaluate attainment of target or identifying threshold is a relevant statistical method, which will enable the Rwandan Government to sustain malaria control and monitor the effectiveness of targeted interventions.

Performance monitoring and accountability: The Agile Project's protocol, record and experience

The Performance Monitoring and Accountability 2020 (PMA2020) project implemented a multi-country sub-project called PMA Agile, a system of continuous data collection for a probability sample of urban public and private health facilities and their clients that began November 2017 and concluded December 2019.  The objective was to monitor the supply, quality and consumption of family planning services.  In total, across 14 urban settings, nearly 2300 health facilities were surveyed three to six times in two years and a total sample of 48,610 female and male clients of childbearing age were interviewed in Burkina Faso, Democratic Republic of Congo, India, Kenya, Niger and Nigeria.  Consenting female clients with access to a cellphone were re-interviewed by telephone after four months; two rounds of the client exit, and follow-up interviews were conducted in nearly all settings.  This paper reports on the PMA Agile data system protocols, coverage and early experiences.  An online dashboard is publicly accessible, analyses of measured trends are underway, and the data are publicly available.

Determinants of subnational disparities in antenatal care utilisation: a spatial analysis of demographic and health survey data in Kenya

BACKGROUND

The spatial variation in antenatal care (ANC) utilisation is likely associated with disparities observed in maternal and neonatal deaths. Most maternal deaths are preventable through services offered during ANC; however, estimates of ANC coverage at lower decision-making units (sub-county) is mostly lacking. In this study, we aimed to estimate the coverage of at least four ANC (ANC4) visits at the sub-county level using the 2014 Kenya Demographic and Health Survey (KDHS 2014) and identify factors associated with ANC utilisation in Kenya.

METHODS

Data from the KDHS 2014 was used to compute sub-county estimates of ANC4 using small area estimation (SAE) techniques which relied on spatial relatedness to yield precise and reliable estimates at each of the 295 sub-counties. Hierarchical mixed-effect logistic regression was used to identify factors influencing ANC4 utilisation. Sub-county estimates of factors significantly associated with ANC utilisation were produced using SAE techniques and mapped to visualise disparities.

RESULTS

The coverage of ANC4 across sub-counties was heterogeneous, ranging from a low of 17% in Mandera West sub-county to over 77% in Nakuru Town West and Ruiru sub-counties. Thirty-one per cent of the 295 sub-counties had coverage of less than 50%. Maternal education, household wealth, place of delivery, marital status, age at first marriage, and birth order were all associated with ANC utilisation. The areas with low ANC4 utilisation rates corresponded to areas of low socioeconomic status, fewer educated women and a small number of health facility deliveries.

CONCLUSION

Suboptimal coverage of ANC4 and its heterogeneity at sub-county level calls for urgent, focused and localised approaches to improve access to antenatal care services. Policy formulation and resources allocation should rely on data-driven strategies to guide national and county governments achieve equity in access and utilisation of health interventions.

Effective coverage measurement in maternal, newborn, child, and adolescent health and nutrition: progress, future prospects, and implications for quality health systems

Intervention coverage-the proportion of the population with a health-care need who receive care-does not account for intervention quality and potentially overestimates health benefits of services provided to populations. Effective coverage introduces the dimension of quality of care to the measurement of intervention coverage. Many definitions and methodological approaches to measuring effective coverage have been developed, resulting in confusion over definition, calculation, interpretation, and monitoring of these measures. To develop a consensus on the definition and measurement of effective coverage for maternal, newborn, child, and adolescent health and nutrition (MNCAHN), WHO and UNICEF convened a group of experts, the Effective Coverage Think Tank Group, to make recommendations for standardising the definition of effective coverage, measurement approaches for effective coverage, indicators of effective coverage in MNCAHN, and to develop future effective coverage research priorities. Via a series of consultations, the group recommended that effective coverage be defined as the proportion of a population in need of a service that resulted in a positive health outcome from the service. The proposed effective coverage measures and care cascade steps can be applied to further develop effective coverage measures across a broad range of MNCAHN services. Furthermore, advances in measurement of effective coverage could improve monitoring efforts towards the achievement of universal health coverage.

Performance monitoring and accountability: The Agile Project’s protocol, record and experience

The Performance Monitoring and Accountability 2020 (PMA2020) project implemented a multi-country sub-project called PMA Agile, a system of continuous data collection for a probability sample of urban public and private health facilities and their clients that began November 2017 and concluded December 2019.  The objective was to monitor the supply, quality and consumption of family planning services.  In total, across 14 urban settings, nearly 2300 health facilities were surveyed three to six times in two years and a total sample of 48,610 female and male clients of childbearing age were interviewed in Burkina Faso, Democratic Republic of Congo, India, Kenya, Niger and Nigeria.  Consenting female clients with access to a cellphone were re-interviewed by telephone after four months; two rounds of the client exit, and follow-up interviews were conducted in nearly all settings.  This paper reports on the PMA Agile data system protocols, coverage and early experiences.  An online dashboard is publicly accessible, analyses of measured trends are underway, and the data are publicly available.

Combining national survey with facility-based HIV testing data to obtain more accurate estimate of HIV prevalence in districts in Uganda

Background

National or regional population-based HIV prevalence surveys have small sample sizes at district or sub-district levels; this leads to wide confidence intervals when estimating HIV prevalence at district level for programme monitoring and decision making. Health facility programme data, collected during service delivery is widely available, but since people self-select for HIV testing, HIV prevalence estimates based on it, is subject to selection bias. We present a statistical annealing technique, Hybrid Prevalence Estimation (HPE), that combines a small population-based survey sample with a facility-based sample to generate district level HIV prevalence estimates with associated confidence intervals.

Methods

We apply the HPE methodology to combine the 2011 Uganda AIDS indicator survey with the 2011 health facility HIV testing data to obtain HIV prevalence estimates for districts in Uganda. Multilevel logistic regression was used to obtain the propensity of testing for HIV in a health facility, and the propensity to test was used to combine the population survey and health facility HIV testing data to obtain the HPEs. We assessed comparability of the HPEs and survey-based estimates using Bland Altman analysis.

Results

The estimates ranged from 0.012 to 0.178 and had narrower confidence intervals compared to survey-based estimates. The average difference between HPEs and population survey estimates was 0.00 (95% CI: − 0.04, 0.04). The HPE standard errors were 28.9% (95% CI: 23.4–34.4) reduced, compared to survey-based standard errors. Overall reduction in HPE standard errors compared survey-based standard errors ranged from 5.4 to 95%.

Conclusions

Facility data can be combined with population survey data to obtain more accurate HIV prevalence estimates for geographical areas with small population survey sample sizes. We recommend use of the methodology by district level managers to obtain more accurate HIV prevalence estimates to guide decision making without incurring additional data collection costs.

Consultative meeting that examined alignment and discrepancies between health facility and household survey data in Malawi

Malawi is midway through its current Malaria Strategic Plan 2017–2022, which aims to reduce malaria incidence and deaths by at least 50% by 2022. Malariometric data are available with health surveillance data housed in District Health Information Software 2 (DHIS2) and household survey data from two recent Malaria Indicator Surveys (MIS) and a Demographic and Health Survey (DHS). Strengths and weaknesses of the data were discussed during a consultative meeting in Lilongwe, Malawi in July 2019. The first 3 days included in-depth exploration and analysis of surveillance and survey data by 13 participants from the National Malaria Control Programme, district health offices, and partner organizations. Key indicators derived from both DHIS2 and MIS/DHS sources were analysed with three case studies, and presented to stakeholders on the fourth day of the meeting. Applications of the findings to programmatic decision-making and strategic plan evaluation were critiqued and discussed.

The costs of developing, deploying and maintaining electronic immunisation registries in Tanzania and Zambia

Objective

To determine the costs to develop, roll out and maintain electronic immunisation registries (EIRs) and a related suite of data use interventions.

Methods

The Better Immunisation Data (BID) Initiative conducted the activities from 2013 to 2018 in three regions in Tanzania and one province in Zambia. The Initiative’s financial records were used to account for the financial costs of designing and developing the EIRs, BID staff time, expenditures for rolling out the EIR systems and the related suite of interventions to health facilities, and recurrent costs. Total financial costs, cost per facility and cost per child were calculated in 2018 US$.

Findings

Total expenditures were ~US$4.2 million in Tanzania and US$3.6 million in Zambia. System design and development costs accounted for ~33% and 26% of the expenditures in each country, respectively, while BID staff costs accounted for 39% and 52%, respectively. Average expenditures per health facility for rolling out the EIR system were between US$709 and US$1320 for the Tanzania regions and US$2591 for Zambia. The annualised average expenditure per child was estimated to be between US$3.30 and US$3.81 for the regions in Tanzania and US$8.46 in Zambia. Expenditures per child were higher in Zambia partly because of a much smaller birth cohort compared with Tanzania.

Conclusion

Other countries may benefit from the investments made and lessons learnt in Tanzania and Zambia by leveraging these now existing EIR platforms and rollout strategies, and hence may be able to implement EIRs at lower costs than reported here.

Generating statistics from health facility data: the state of routine health information systems in Eastern and Southern Africa

Health facility data are a critical source of local and continuous health statistics. Countries have introduced web-based information systems that facilitate data management, analysis, use and visualisation of health facility data. Working with teams of Ministry of Health and country public health institutions analysts from 14 countries in Eastern and Southern Africa, we explored data quality using national-level and subnational-level (mostly district) data for the period 2013-2017. The focus was on endline analysis where reported health facility and other data are compiled, assessed and adjusted for data quality, primarily to inform planning and assessments of progress and performance. The analyses showed that although completeness of reporting was generally high, there were persistent data quality issues that were common across the 14 countries, especially at the subnational level. These included the presence of extreme outliers, lack of consistency of the reported data over time and between indicators (such as vaccination and antenatal care), and challenges related to projected target populations, which are used as denominators in the computation of coverage statistics. Continuous efforts to improve recording and reporting of events by health facilities, systematic examination and reporting of data quality issues, feedback and communication mechanisms between programme managers, care providers and data officers, and transparent corrections and adjustments will be critical to improve the quality of health statistics generated from health facility data.

"Every Newborn-BIRTH" protocol: observational study validating indicators for coverage and quality of maternal and newborn health care in Bangladesh, Nepal and Tanzania

Background

To achieve Sustainable Development Goals and Universal Health Coverage, programmatic data are essential. The Every Newborn Action Plan, agreed by all United Nations member states and >80 development partners, includes an ambitious Measurement Improvement Roadmap. Quality of care at birth is prioritised by both Every Newborn and Ending Preventable Maternal Mortality strategies, hence metrics need to advance from health service contact alone, to content of care. As facility births increase, monitoring using routine facility data in DHIS2 has potential, yet validation research has mainly focussed on maternal recall surveys. The Every Newborn – Birth Indicators Research Tracking in Hospitals (EN-BIRTH) study aims to validate selected newborn and maternal indicators for routine tracking of coverage and quality of facility-based care for use at district, national and global levels.

Methods

EN-BIRTH is an observational study including >20 000 facility births in three countries (Tanzania, Bangladesh and Nepal) to validate selected indicators. Direct clinical observation will be compared with facility register data and a pre-discharge maternal recall survey for indicators including: uterotonic administration, immediate newborn care, neonatal resuscitation and Kangaroo mother care. Indicators including neonatal infection management and antenatal corticosteroid administration, which cannot be easily observed, will be validated using inpatient records. Trained clinical observers in Labour/Delivery ward, Operation theatre, and Kangaroo mother care ward/areas will collect data using a tablet-based customised data capturing application. Sensitivity will be calculated for numerators of all indicators and specificity for those numerators with adequate information. Other objectives include comparison of denominator options (ie, true target population or surrogates) and quality of care analyses, especially regarding intervention timing. Barriers and enablers to routine recording and data usage will be assessed by data flow assessments, quantitative and qualitative analyses.

Conclusions

To our knowledge, this is the first large, multi-country study validating facility-based routine data compared to direct observation for maternal and newborn care, designed to provide evidence to inform selection of a core list of indicators recommended for inclusion in national DHIS2. Availability and use of such data are fundamental to drive progress towards ending the annual 5.5 million preventable stillbirths, maternal and newborn deaths.

Quality of routine facility data for monitoring priority maternal and newborn indicators in DHIS2: A case study from Gombe State, Nigeria

INTRODUCTION

Routine health information systems are critical for monitoring service delivery. District Heath Information System, version 2 (DHIS2) is an open source software platform used in more than 60 countries, on which global initiatives increasingly rely for such monitoring. We used facility-reported data in DHIS2 for Gombe State, north-eastern Nigeria, to present a case study of data quality to monitor priority maternal and neonatal health indicators.

METHODS

For all health facilities in DHIS2 offering antenatal and postnatal care services (n = 497) and labor and delivery services (n = 486), we assessed the quality of data for July 2016-June 2017 according to the World Health Organization data quality review guidance. Using data from DHIS2 as well as external facility-level and population-level household surveys, we reviewed three data quality dimensions-completeness and timeliness, internal consistency, and external consistency-and considered the opportunities for improvement.

RESULTS

Of 14 priority maternal and neonatal health indicators that could be tracked through facility-based data, 12 were included in Gombe's DHIS2. During July 2016-June 2017, facility-reported data in DHIS2 were incomplete at least 40% of the time, under-reported 10%-60% of the events documented in facility registers, and showed inconsistencies over time, between related indicators, and with an external data source. The best quality data elements were those that aligned with Gombe's health program priorities, particularly older health programs, and those that reflected contact indicators rather than indicators related to the provision of commodities or content of care.

CONCLUSION

This case study from Gombe State, Nigeria, demonstrates the high potential for effective monitoring of maternal and neonatal health using DHIS2. However, coordinated action at multiple levels of the health system is needed to maximize reporting of existing data; rationalize data flow; routinize data quality review, feedback, and supervision; and ensure ongoing maintenance of DHIS2.

Hybrid prevalence estimation: Method to improve intervention coverage estimations

Delivering excellent health services requires accurate health information systems (HIS) data. Poor-quality data can lead to poor judgments and outcomes. Unlike probability surveys, which are representative of the population and carry accuracy estimates, HIS do not, but in many countries the HIS is the primary source of data used for administrative estimates. However, HIS are not structured to detect gaps in service coverage and leave communities exposed to unnecessary health risks. Here we propose a method to improve informatics by combining HIS and probability survey data to construct a hybrid estimator. This technique provides a more accurate estimator than either data source alone and facilitates informed decision-making. We use data from vitamin A and polio vaccination campaigns in children from Madagascar and Benin to demonstrate the effect. The hybrid estimator is a weighted average of two measurements and produces SEs and 95% confidence intervals (CIs) for the hybrid and HIS estimators. The estimates of coverage proportions using the combined data and the survey estimates differ by no more than 3%, while decreasing the SE by 1-6%; the administrative estimates from the HIS and combined data estimates are very different, with 3-25 times larger CI, questioning the value of administrative estimates. Estimators of unknown accuracy may lead to poorly formulated policies and wasted resources. The hybrid estimator technique can be applied to disease prevention services for which population coverages are measured. This methodology creates more accurate estimators, alongside measured HIS errors, to improve tracking the public's health.