Monitoring child survival in 'real time' using routine health facility records: results from Malawi

The Impact of Health Information System Interventions on Maternal and Child Health Service Utilizations in Ethiopia: A Quasi-Experimental Study

BACKGROUND

Health information systems (HIS) are vital in supporting all aspects of managing health systems, financing, policymaking, and service delivery. A package of priority HIS interventions was piloted in selected woredas across all regions in Ethiopia. This study examined the impact of HIS interventions on maternal and child health (MCH) service utilization.

METHODS

A 2-arm quasi-experimental study was implemented in intervention and control woredas. Baseline and endline household and health facility surveys were conducted for both arms in 2020 and 2022, respectively. At baseline, 3,016 mothers and 167 health facilities were surveyed. At endline, 3,076 mothers and 160 health facilities were surveyed. The study used modified Performance of Routine Information System Management tools for the facility survey and a structured questionnaire for the household survey. Difference-in-difference (DID) analysis using mixed effect modeling was employed to measure changes and to account for clustering and control for likely confounders.

RESULTS

Intervention sites showed greater improvements in 75% of key HIS performance indicators. The changes in 90% of the MCH service utilization indicators were higher in the intervention sites. Significant (DID: P<.05) changes were observed in indicators including quality of antenatal care, skilled birth attendance, delivery at a health facility, family planning met need and unmet need, measles and second dose of rotavirus vaccination, and Vitamin A supplementation. BCG vaccination showed significantly higher improvement in the control sites. Other key indicators did not show significant changes.

CONCLUSIONS

In many of the MCH service utilization indicators, the changes in the intervention sites were significantly higher compared to the control sites, but it was not universal. Scale-up of performance monitoring teams is crucial because it is one of the key pathways that links HIS performance with MCH service utilization. Outcome indicators that showed no or lower improvement require in-depth investigation.

Effectiveness of in-service training plus the collaborative improvement strategy on the quality of routine malaria surveillance data: results of a pilot study in Kayunga District, Uganda

BACKGROUND

Surveillance data are essential for malaria control, but quality is often poor. The aim of the study was to evaluate the effectiveness of the novel combination of training plus an innovative quality improvement method-collaborative improvement (CI)-on the quality of malaria surveillance data in Uganda.

METHODS

The intervention (training plus CI, or TCI), including brief in-service training and CI, was delivered in 5 health facilities (HFs) in Kayunga District from November 2015 to August 2016. HF teams monitored data quality, conducted plan-do-study-act cycles to test changes, attended periodic learning sessions, and received CI coaching. An independent evaluation was conducted to assess data completeness, accuracy, and timeliness. Using an interrupted time series design without a separate control group, data were abstracted from 156,707 outpatient department (OPD) records, laboratory registers, and aggregated monthly reports (MR) for 4 time periods: baseline-12 months, TCI scale-up-5 months; CI implementation-9 months; post-intervention-4 months. Monthly OPD register completeness was measured as the proportion of patient records with a malaria diagnosis with: (1) all data fields completed, and (2) all clinically-relevant fields completed. Accuracy was the relative difference between: (1) number of monthly malaria patients reported in OPD register versus MR, and (2) proportion of positive malaria tests reported in the laboratory register versus MR. Data were analysed with segmented linear regression modelling.

RESULTS

Data completeness increased substantially following TCI. Compared to baseline, all-field completeness increased by 60.1%-points (95% confidence interval [CI]: 46.9-73.2%) at mid-point, and clinically-relevant completeness increased by 61.6%-points (95% CI: 56.6-66.7%). A relative - 57.4%-point (95% confidence interval: - 105.5, - 9.3%) change, indicating an improvement in accuracy of malaria test positivity reporting, but no effect on data accuracy for monthly malaria patients, were observed. Cost per additional malaria patient, for whom complete clinically-relevant data were recorded in the OPD register, was $3.53 (95% confidence interval: $3.03, $4.15).

CONCLUSIONS

TCI improved malaria surveillance completeness considerably, with limited impact on accuracy. Although these results are promising, the intervention's effectiveness should be evaluated in more HFs, with longer follow-up, ideally in a randomized trial, before recommending CI for wide-scale use.

Lessons learned from the development and implementation of an electronic paediatric emergency and acute care database in Lilongwe, Malawi

As the field of global child health increasingly focuses on inpatient and emergency care, there is broad recognition of the need for comprehensive, accurate data to guide decision-making at both patient and system levels. Limited financial and human resources present barriers to reliable and detailed clinical documentation at hospitals in low-and-middle-income countries (LMICs). Kamuzu Central Hospital (KCH) is a tertiary referral hospital in Malawi where the paediatric ward admits up to 3000 children per month. To improve availability of robust inpatient data, we collaboratively designed an acute care database on behalf of PACHIMAKE, a consortium of Malawi and US-based institutions formed to improve paediatric care at KCH. We assessed the existing health information systems at KCH, reviewed quality care metrics, engaged clinical providers and interviewed local stakeholders who would directly use the database or be involved in its collection. Based on the information gathered, we developed electronic forms collecting data at admission, follow-up and discharge for children admitted to the KCH paediatric wards. The forms record demographic information, basic medical history, clinical condition and pre-referral management; track diagnostic processes, including laboratory studies, imaging modalities and consults; and document the final diagnoses and disposition obtained from clinical files and corroborated through review of existing admission and death registries. Our experience with the creation of this database underscores the importance of fully assessing existing health information systems and involving all stakeholders early in the planning process to ensure meaningful and sustainable implementation.

Using routine health information data for research in low- and middle-income countries: a systematic review

BACKGROUND

Routine health information systems (RHISs) support resource allocation and management decisions at all levels of the health system, as well as strategy development and policy-making in many low- and middle-income countries (LMICs). Although RHIS data represent a rich source of information, such data are currently underused for research purposes, largely due to concerns over data quality. Given that substantial investments have been made in strengthening RHISs in LMICs in recent years, and that there is a growing demand for more real-time data from researchers, this systematic review builds upon the existing literature to summarize the extent to which RHIS data have been used in peer-reviewed research publications.

METHODS

Using terms 'routine health information system', 'health information system', or 'health management information system' and a list of LMICs, four electronic peer-review literature databases were searched from inception to February 202,019: PubMed, Scopus, EMBASE, and EconLit. Articles were assessed for inclusion based on pre-determined eligibility criteria and study characteristics were extracted from included articles using a piloted data extraction form.

RESULTS

We identified 132 studies that met our inclusion criteria, originating in 37 different countries. Overall, the majority of the studies identified were from Sub-Saharan Africa and were published within the last 5 years. Malaria and maternal health were the most commonly studied health conditions, although a number of other health conditions and health services were also explored.

CONCLUSIONS

Our study identified an increasing use of RHIS data for research purposes, with many studies applying rigorous study designs and analytic methods to advance program evaluation, monitoring and assessing services, and epidemiological studies in LMICs. RHIS data represent an underused source of data and should be made more available and further embraced by the research community in LMIC health systems.

Childhood malaria case incidence in Malawi between 2004 and 2017: spatio-temporal modelling of climate and non-climate factors

BACKGROUND

Malaria transmission is influenced by a complex interplay of factors including climate, socio-economic, environmental factors and interventions. Malaria control efforts across Africa have shown a mixed impact. Climate driven factors may play an increasing role with climate change. Efforts to strengthen routine facility-based monthly malaria data collection across Africa create an increasingly valuable data source to interpret burden trends and monitor control programme progress. A better understanding of the association with other climatic and non-climatic drivers of malaria incidence over time and space may help guide and interpret the impact of interventions.

METHODS

Routine monthly paediatric outpatient clinical malaria case data were compiled from 27 districts in Malawi between 2004 and 2017, and analysed in combination with data on climatic, environmental, socio-economic and interventional factors and district level population estimates. A spatio-temporal generalized linear mixed model was fitted using Bayesian inference, in order to quantify the strength of association of the various risk factors with district-level variation in clinical malaria rates in Malawi, and visualized using maps.

RESULTS

Between 2004 and 2017 reported childhood clinical malaria case rates showed a slight increase, from 50 to 53 cases per 1000 population, with considerable variation across the country between climatic zones. Climatic and environmental factors, including average monthly air temperature and rainfall anomalies, normalized difference vegetative index (NDVI) and RDT use for diagnosis showed a significant relationship with malaria incidence. Temperature in the current month and in each of the 3 months prior showed a significant relationship with the disease incidence unlike rainfall anomaly which was associated with malaria incidence at only three months prior. Estimated risk maps show relatively high risk along the lake and Shire valley regions of Malawi.

CONCLUSION

The modelling approach can identify locations likely to have unusually high or low risk of malaria incidence across Malawi, and distinguishes between contributions to risk that can be explained by measured risk-factors and unexplained residual spatial variation. Also, spatial statistical methods applied to readily available routine data provides an alternative information source that can supplement survey data in policy development and implementation to direct surveillance and intervention efforts.

Targeting Pregnant Women for Malaria Surveillance

Women attending antenatal care (ANC) are a generally healthy, easy-access population, contributing valuable data for infectious disease surveillance at the community level. ANC-based malaria surveillance would provide a routine measure of the malaria burden in pregnancy, which countries lack, whilst potentially improving pregnancy outcomes. It could also offer contemporary information on temporal trends and the geographic distribution of malaria burden as well as intervention coverage in the population to guide resource allocation and to assess progress towards elimination. Here, we review the factors underlying the relationship between Plasmodium falciparum in pregnancy and in the community, and outline strengths and limitations of an ANC-based surveillance in sub-Saharan Africa, its potential role within wider malaria surveillance systems, and subsequent programmatic applications.

On the estimation of population cause-specific mortality fractions from in-hospital deaths

BACKGROUND

Almost all countries without complete vital registration systems have data on deaths collected by hospitals. However, these data have not been widely used to estimate cause of death (COD) patterns in populations because only a non-representative fraction of people in these countries die in health facilities. Methods that can exploit hospital mortality statistics to reliably estimate community COD patterns are required to strengthen the evidence base for disease and injury control programs. We propose a method that weights hospital-certified causes by the probability of death to estimate population cause-specific mortality fractions (CSMFs).

METHODS

We used an established verbal autopsy instrument (VAI) to collect data from hospital catchment areas in Chandpur and Comilla Districts, Bangladesh, and Bohol province, the Philippines, between 2011 and 2014, along with demographic covariates for each death. Hospital medical certificates of cause of death (death certificates) were collected and mapped to the corresponding cause categories of the VAI. Tariff 2.0 was used to assign a COD for community deaths. Logistic regression models were created for broad causes in each country to calculate the probability of in-hospital death, given a set of covariate values. The reweighted CSMFs for deaths in the hospital catchment population, represented by each hospital death, were calculated from the corresponding regression models.

RESULTS

We collected data on 4228 adult deaths in the Philippines and 3725 deaths in Bangladesh. Short time to hospital and education were consistently associated with in-hospital death in the Philippines and absence of a disability was consistently associated with in-hospital death in Bangladesh. Non-communicable diseases (excluding stroke) and stroke were the leading causes of death in both the Philippines (33.9%, 19.1%) and Bangladesh (46.1%, 21.1%) according to the reweighted method. The reweighted method generally estimated CSMFs that fell between those derived from hospitals and those diagnosed by Tariff 2.0.

CONCLUSIONS

Statistical methods can be used to derive estimates of cause-specific probability of death in-hospital for Bangladesh and the Philippines to generate population CSMFs. In regions where hospital death certification is of reasonable quality and routine verbal autopsy is not applied, these estimates could be applied to generate cost-effective and robust CSMFs for the population.

Quality of routine facility data for monitoring priority maternal and newborn indicators in DHIS2: A case study from Gombe State, Nigeria

INTRODUCTION

Routine health information systems are critical for monitoring service delivery. District Heath Information System, version 2 (DHIS2) is an open source software platform used in more than 60 countries, on which global initiatives increasingly rely for such monitoring. We used facility-reported data in DHIS2 for Gombe State, north-eastern Nigeria, to present a case study of data quality to monitor priority maternal and neonatal health indicators.

METHODS

For all health facilities in DHIS2 offering antenatal and postnatal care services (n = 497) and labor and delivery services (n = 486), we assessed the quality of data for July 2016-June 2017 according to the World Health Organization data quality review guidance. Using data from DHIS2 as well as external facility-level and population-level household surveys, we reviewed three data quality dimensions-completeness and timeliness, internal consistency, and external consistency-and considered the opportunities for improvement.

RESULTS

Of 14 priority maternal and neonatal health indicators that could be tracked through facility-based data, 12 were included in Gombe's DHIS2. During July 2016-June 2017, facility-reported data in DHIS2 were incomplete at least 40% of the time, under-reported 10%-60% of the events documented in facility registers, and showed inconsistencies over time, between related indicators, and with an external data source. The best quality data elements were those that aligned with Gombe's health program priorities, particularly older health programs, and those that reflected contact indicators rather than indicators related to the provision of commodities or content of care.

CONCLUSION

This case study from Gombe State, Nigeria, demonstrates the high potential for effective monitoring of maternal and neonatal health using DHIS2. However, coordinated action at multiple levels of the health system is needed to maximize reporting of existing data; rationalize data flow; routinize data quality review, feedback, and supervision; and ensure ongoing maintenance of DHIS2.

Using community-based reporting of vital events to monitor child mortality: Lessons from rural Ghana

Background

Reducing neonatal and child mortality is a key component of the health-related sustainable development goal (SDG), but most low and middle income countries lack data to monitor child mortality on an annual basis. We tested a mortality monitoring system based on the continuous recording of pregnancies, births and deaths by trained community-based volunteers (CBV).

Methods and findings

This project was implemented in 96 clusters located in three districts of the Northern Region of Ghana. Community-based volunteers (CBVs) were selected from these clusters and were trained in recording all pregnancies, births, and deaths among children under 5 in their catchment areas. Data collection lasted from January 2012 through September 2013. All CBVs transmitted tallies of recorded births and deaths to the Ghana Birth and deaths registry each month, except in one of the study districts (approximately 80% reporting). Some events were reported only several months after they had occurred. We assessed the completeness and accuracy of CBV data by comparing them to retrospective full pregnancy histories (FPH) collected during a census of the same clusters conducted in October-December 2013. We conducted all analyses separately by district, as well as for the combined sample of all districts. During the 21-month implementation period, the CBVs reported a total of 2,819 births and 137 under-five deaths. Among the latter, there were 84 infant deaths (55 neonatal deaths and 29 post-neonatal deaths). Comparison of the CBV data with FPH data suggested that CBVs significantly under-estimated child mortality: the estimated under-5 mortality rate according to CBV data was only 2/3 of the rate estimated from FPH data (95% Confidence Interval for the ratio of the two rates = 51.7 to 81.4). The discrepancies between the CBV and FPH estimates of infant and neonatal mortality were more limited, but varied significantly across districts.

Conclusions

In northern Ghana, a community-based data collection systems relying on volunteers did not yield accurate estimates of child mortality rates. Additional implementation research is needed to improve the timeliness, completeness and accuracy of such systems. Enhancing pregnancy monitoring, in particular, may be an essential step to improve the measurement of neonatal mortality.

National Assessment of Data Quality and Associated Systems-Level Factors in Malawi

BACKGROUND

Routine health data can guide health systems improvements, but poor quality of these data hinders use. To address concerns about data quality in Malawi, the Ministry of Health and National Statistical Office conducted a data quality assessment (DQA) in July 2016 to identify systems-level factors that could be improved.

METHODS

We used 2-stage stratified random sampling methods to select health centers and hospitals under Ministry of Health auspices, included those managed by faith-based entities, for this DQA. Dispensaries, village clinics, police and military facilities, tertiary-level hospitals, and private facilities were excluded. We reviewed client registers and monthly reports to verify availability, completeness, and accuracy of data in 4 service areas: antenatal care (ANC), family planning, HIV testing and counseling, and acute respiratory infection (ARI). We also conducted interviews with facility and district personnel to assess health management information system (HMIS) functioning and systems-level factors that may be associated with data quality. We compared systems and quality factors by facility characteristics using 2-sample t tests with Welch's approximation, and calculated verification ratios comparing total entries in registers to totals from summarized reports.

RESULTS

We selected 16 hospitals (of 113 total in Malawi), 90 health centers (of 466), and 16 district health offices (of 28) in 16 of Malawi's 28 districts. Nearly all registers were available and complete in health centers and district hospitals, but data quality varied across service areas; median verification ratios comparing register and report totals at health centers ranged from 0.78 (interquartile range [IQR]: 0.25, 1.07) for ARI and 0.99 (IQR: 0.82, 1.36) for family planning to 1.00 (IQR: 0.96, 1.00) for HIV testing and counseling and 1.00 (IQR: 0.80, 1.23) for ANC. More than half (60%) of facilities reported receiving a documented supervisory visit for HMIS in the prior 6 months. A recent supervision visit was associated with better availability of data (P=.05), but regular district- or central-level supervision was not. Use of data by the facility to track performance toward targets was associated with both improved availability (P=.04) and completeness of data (P=.02). Half of facilities had a full-time statistical clerk, but their presence did not improve the availability or completeness of data (P=.39 and P=.69, respectively).

CONCLUSION

Findings indicate both strengths and weaknesses in Malawi's HMIS performance, with key weaknesses including infrequent data quality checks and unreliable supervision. Efforts to strengthen HMIS in low- and middle-income countries should be informed by similar assessments.

Methodological Considerations for Use of Routine Health Information System Data to Evaluate Malaria Program Impact in an Era of Declining Malaria Transmission

Coverage of malaria control interventions is increasing dramatically across endemic countries. Evaluating the impact of malaria control programs and specific interventions on health indicators is essential to enable countries to select the most effective and appropriate combination of tools to accelerate progress or proceed toward malaria elimination. When key malaria interventions have been proven effective under controlled settings, further evaluations of the impact of the intervention using randomized approaches may not be appropriate or ethical. Alternatives to randomized controlled trials are therefore required for rigorous evaluation under conditions of routine program delivery. Routine health management information system (HMIS) data are a potentially rich source of data for impact evaluation, but have been underused in impact evaluation due to concerns over internal validity, completeness, and potential bias in estimates of program or intervention impact. A range of methodologies were identified that have been used for impact evaluations with malaria outcome indicators generated from HMIS data. Methods used to maximize internal validity of HMIS data are presented, together with recommendations on reducing bias in impact estimates. Interrupted time series and dose-response analyses are proposed as the strongest quasi-experimental impact evaluation designs for analysis of malaria outcome indicators from routine HMIS data. Interrupted time series analysis compares the outcome trend and level before and after the introduction of an intervention, set of interventions or program. The dose-response national platform approach explores associations between intervention coverage or program intensity and the outcome at a subnational (district or health facility catchment) level.

Completeness of malaria indicator data reporting via the District Health Information Software 2 in Kenya, 2011-2015

Background

Health facility-based data reported through routine health information systems form the primary data source for programmatic monitoring and evaluation in most developing countries. The adoption of District Health Information Software (DHIS2) has contributed to improved availability of routine health facility-based data in many low-income countries. An assessment of malaria indicators data reported by health facilities in Kenya during the first 5 years of implementation of DHIS2, from January 2011 to December 2015, was conducted.

Methods

Data on 19 malaria indicators reported monthly by health facilities were extracted from the online Kenya DHIS2 database. Completeness of reporting was analysed for each of the 19 malaria indicators and expressed as the percentage of data values actually reported over the expected number; all health facilities were expected to report data for each indicator for all 12 months in a year.

Results

Malaria indicators data were analysed for 6235 public and 3143 private health facilities. Between 2011 and 2015, completeness of reporting in the public sector increased significantly for confirmed malaria cases across all age categories (26.5–41.9%, p < 0.0001, in children aged <5 years; 30.6–51.4%, p < 0.0001, in persons aged ≥5 years). Completeness of reporting of new antenatal care (ANC) clients increased from 53.7 to 70.5%, p < 0.0001). Completeness of reporting of intermittent preventive treatment in pregnancy (IPTp) decreased from 64.8 to 53.7%, p < 0.0001 for dose 1 and from 64.6 to 53.4%, p < 0.0001 for dose 2. Data on malaria tests performed and test results were not available in DHIS2 from 2011 to 2014. In 2015, sparse data on microscopy (11.5% for children aged <5 years; 11.8% for persons aged ≥5 years) and malaria rapid diagnostic tests (RDTs) (8.1% for all ages) were reported. In the private sector, completeness of reporting increased significantly for confirmed malaria cases across all age categories (16.7–23.1%, p < 0.0001, in children aged <5 years; 19.4–28.6%, p < 0.0001, in persons aged ≥5 years). Completeness of reporting also improved for new ANC clients (16.2–23.6%, p < 0.0001), and for IPTp doses 1 and 2 (16.6–20.2%, p < 0.0001 and 15.5–20.5%, p < 0.0001, respectively). In 2015, less than 3% of data values for malaria tests performed were reported in DHIS2 from the private sector.

Conclusions

There have been sustained improvements in the completeness of data reported for most key malaria indicators since the adoption of DHIS2 in Kenya in 2011. However, major data gaps were identified for the malaria-test indicator and overall low reporting across all indicators from private health facilities. A package of proven DHIS2 implementation interventions and performance-based incentives should be considered to improve private-sector data reporting.

Electronic supplementary material

The online version of this article (doi:10.1186/s12936-017-1973-y) contains supplementary material, which is available to authorized users.

Social autopsy of neonatal mortality suggests needed improvements in maternal and neonatal interventions in Balaka and Salima districts of Malawi

Background

The Every Newborn Action Plan calls for reducing the neonatal mortality rates to fewer than 10 deaths per 1000 live births in all countries by 2035. The current study aims to increase our understanding of the social and modifiable factors that can be addressed or reinforced to improve and accelerate the decline in neonatal mortality in Malawi.

Methods

The data come from the 2013 Verbal and Social Autopsy (VASA) study that collected data in order to describe the biological causes and the social determinants of deaths of children under 5 years of age in Balaka and Salima districts of Malawi. This paper analyses the social autopsy data of the neonatal deaths and presents results of a review of the coverage of key interventions along the continuum of normal maternal and newborn care and the description of breakdowns in the care provided for neonatal illnesses within the Pathway to Survival framework.

Results

A total of 320 neonatal deaths were confirmed from the VASA survey. While one antenatal care (ANC) visit was high at 94%, the recommended four ANC visits was much lower at 41% and just 17% of the mothers had their urines tested during the pregnancy. 173 (54%) mothers of the deceased newborns had at least one labor/delivery complication that began at home. The caregivers of 65% (n = 75) of the 180 newborns that were born at home or born and left a health facility alive perceived them to be severely ill at the onset of their illness, yet only 44% (n = 80) attempted and 36% (n = 65)could reach the first health provider after an average of 91 minutes travel time. Distance, lack of transport and cost emerged as the most important constraints to formal care–seeking during delivery and during the newborn fatal illness.

Conclusions

This study suggests that maternal and neonatal health organizations and the local government of Malawi should increase the demand for key maternal and child health interventions, including the recommended 4 ANC visits, and ensure urine screening for all pregnant women. Early recognition and referrals of women with obstetric complications and interventions to promote maternal recognition of neonatal illnesses and care–seeking before the child becomes severely ill are also needed to improve newborn survival in Balaka and Salima districts of Malawi.

Understanding Misclassification between Neonatal Deaths and Stillbirths: Empirical Evidence from Malawi

Improving the counting of stillbirths and neonatal deaths is important to tracking Sustainable Development Goal 3.2 and improving vital statistics in low- and middle-income countries (LMICs). However, the validity of self-reported stillbirths and neonatal deaths in surveys is often threatened by misclassification errors between the two birth outcomes. We assessed the extent and correlates of stillbirths being misclassified as neonatal deaths by comparing two recent and linked population surveys conducted in Malawi, one being a full birth history (FBH) survey, and the other a follow-up verbal/social autopsy (VASA) survey. We found that one-fifth of 365 neonatal deaths identified in the FBH survey were classified as stillbirths in the VASA survey. Neonatal deaths with signs of movements in the last few days before delivery reported were less likely to be misclassified stillbirths (OR = 0.08, p<0.05). Having signs of birth injury was found to be associated with higher odds of misclassification (OR = 6.17, p<0.05). We recommend replicating our study with larger sample size in other settings. Additionally, we recommend conducting validation studies to confirm accuracy and completeness of live births and neonatal deaths reported in household surveys with events reported in a full birth history and the extent of underestimation of neonatal mortality resulting from misclassifications. Questions on fetal movement, signs of life at delivery and improved probing among older mother may be useful to improve accuracy of reported events.

Potential use cases for the development of an electronic health facility registry in Nigeria: Key informant's perspectives

BACKGROUND

Master facility lists (MFL) maintain an important standard (unique identifier) in country health information systems that will aid integration and interoperability of multiple health facility based data sources. However, this standard is not readily available in several low and middle income countries where reliable data is most needed for efficient planning. The World Health Organization in 2012 drew up guidelines for the creation of MFLs in countries but this guideline still requires domestication and process modeling for each country adopting it. Nigeria in 2013 published a paper-based MFL directory which it hopes to migrate to an electronic MFL registry for use across the country.

OBJECTIVE

To identify the use cases of importance in the development of an electronic health facility registry to manage the MFL compiled in Nigeria.

METHODS

Potential use cases for the health facility registry were identified through consultations with key informants at the Federal Ministry of Health. These will serve as input into an electronic MFL registry development effort.

RESULTS

The use cases identified include: new health facility is created, update of status of health facility, close-out, relocation, new information available, delete and management of multi-branch health facility.

CONCLUSION

Development of an application for the management of MFLs requires proper architectural analysis of the manifestations that can befall a health facility through its lifecycle. A MFL electronic registry will be invaluable to manage health facility data and will aid the integration and interoperability of health facility information systems.

Strengthening Community-Based Vital Events Reporting for Real-Time Monitoring of Under-Five Mortality: Lessons Learned from the Balaka and Salima Districts in Malawi

Background

Malawi ratified a compulsory birth and death registration system in 2009. Until it captures complete coverage of vital events, Malawi relies on other data sources to calculate mortality estimates. We tested a community-based method to estimate annual under-five mortality rates (U5MR) through the Real-Time Monitoring of Under-Five Mortality (RMM) project in Malawi. We implemented RMM in two phases, and conducted an independent evaluation of phase one after 21 months of implementation. We present results of the phase two validation that covers the full project time span, and compare the results to those of the phase one validation.

Methods and Findings

We assessed the completeness of the counts of births and deaths and the accuracy of disaggregated U5MR from the community-based method against a retrospective full pregnancy history for rolling twelve-month periods after the independent evaluation. We used full pregnancy histories collected through household interviews carried out between November 2013 and January 2014 as the validation data source. Health Surveillance Agents (HSAs) across the 160 catchment areas submitted routine reports on pregnancies, births, and deaths consistently. However, for the 15-month implementation period post-evaluation, average completeness of birth event reporting was 76%, whereas average completeness of death event reporting was 67% relative to that expected from a comparable pregnancy history. HSAs underestimated the U5MR by an average of 21% relative to that estimated from a comparable pregnancy history.

Conclusions

On a medium scale, the community-based RMM method in Malawi produced substantial underestimates of annualized U5MR relative to those obtained from a full pregnancy history, despite the additional incentives and quality-control activities. We were not able to achieve an optimum level of incentive and support to make the system work while ensuring sustainability. Lessons learned from the implementation of RMM can inform programs supporting community-based interventions through HSAs in Malawi.

Evaluation of a mHealth Data Quality Intervention to Improve Documentation of Pregnancy Outcomes by Health Surveillance Assistants in Malawi: A Cluster Randomized Trial

BACKGROUND

While community health workers are being recognized as an integral work force with growing responsibilities, increased demands can potentially affect motivation and performance. The ubiquity of mobile phones, even in hard-to-reach communities, has facilitated the pursuit of novel approaches to support community health workers beyond traditional modes of supervision, job aids, in-service training, and material compensation. We tested whether supportive short message services (SMS) could improve reporting of pregnancies and pregnancy outcomes among community health workers (Health Surveillance Assistants, or HSAs) in Malawi.

METHODS AND FINDINGS

We designed a set of one-way SMS that were sent to HSAs on a regular basis during a 12-month period. We tested the effectiveness of the cluster-randomized intervention in improving the complete documentation of a pregnancy. We defined complete documentation as a pregnancy for which a specific outcome was recorded. HSAs in the treatment group received motivational and data quality SMS. HSAs in the control group received only motivational SMS. During baseline and intervention periods, we matched reported pregnancies to reported outcomes to determine if reporting of matched pregnancies differed between groups and by period. The trial is registered as ISCTRN24785657.

CONCLUSIONS

Study results show that the mHealth intervention improved the documentation of matched pregnancies in both the treatment (OR 1.31, 95% CI: 1.10-1.55, p<0.01) and control (OR 1.46, 95% CI: 1.11-1.91, p = 0.01) groups relative to the baseline period, despite differences in SMS content between groups. The results should be interpreted with caution given that the study was underpowered. We did not find a statistically significant difference in matched pregnancy documentation between groups during the intervention period (OR 0.94, 95% CI: 0.63-1.38, p = 0.74). mHealth applications have the potential to improve the tracking and data quality of pregnancies and pregnancy outcomes, particularly in low-resource settings.

Can Community Health Workers Report Accurately on Births and Deaths? Results of Field Assessments in Ethiopia, Malawi and Mali

Introduction

Most low-income countries lack complete and accurate vital registration systems. As a result, measures of under-five mortality rates rely mostly on household surveys. In collaboration with partners in Ethiopia, Ghana, Malawi, and Mali, we assessed the completeness and accuracy of reporting of births and deaths by community-based health workers, and the accuracy of annualized under-five mortality rate estimates derived from these data. Here we report on results from Ethiopia, Malawi and Mali.

Method

In all three countries, community health workers (CHWs) were trained, equipped and supported to report pregnancies, births and deaths within defined geographic areas over a period of at least fifteen months. In-country institutions collected these data every month. At each study site, we administered a full birth history (FBH) or full pregnancy history (FPH), to women of reproductive age via a census of households in Mali and via household surveys in Ethiopia and Malawi. Using these FBHs/FPHs as a validation data source, we assessed the completeness of the counts of births and deaths and the accuracy of under-five, infant, and neonatal mortality rates from the community-based method against the retrospective FBH/FPH for rolling twelve-month periods. For each method we calculated total cost, average annual cost per 1,000 population, and average cost per vital event reported.

Results

On average, CHWs submitted monthly vital event reports for over 95 percent of catchment areas in Ethiopia and Malawi, and for 100 percent of catchment areas in Mali. The completeness of vital events reporting by CHWs varied: we estimated that 30%-90% of annualized expected births (i.e. the number of births estimated using a FPH) were documented by CHWs and 22%-91% of annualized expected under-five deaths were documented by CHWs. Resulting annualized under-five mortality rates based on the CHW vital events reporting were, on average, under-estimated by 28% in Ethiopia, 32% in Malawi, and 9% in Mali relative to comparable FPHs. Costs per vital event reported ranged from $21 in Malawi to $149 in Mali.

Discussion

Our findings in Mali suggest that CHWs can collect complete and high-quality vital events data useful for monitoring annual changes in under-five mortality rates. Both the supervision of CHWs in Mali and the rigor of the associated field-based data quality checks were of a high standard, and the size of the pilot area in Mali was small (comprising of approximately 53,205 residents in 4,200 households). Hence, there are remaining questions about whether this level of vital events reporting completeness and data quality could be maintained if the approach was implemented at scale. Our experience in Malawi and Ethiopia suggests that, in some settings, establishing and maintaining the completeness and quality of vital events reporting by CHWs over time is challenging. In this sense, our evaluation in Mali falls closer to that of an efficacy study, whereas our evaluations in Ethiopia and Malawi are more akin to an effectiveness study. Our overall findings suggest that no one-size-fits-all approach will be successful in guaranteeing complete and accurate reporting of vital events by CHWs.

"Real-Time" Monitoring of Under-Five Mortality: Lessons for Strengthened Vital Statistics Systems

Bryce and colleagues, reflect on lessons that can be learned from the Real-Time Monitoring of Under-Five Mortality Collection.

Using Health Extension Workers for Monitoring Child Mortality in Real-Time: Validation against Household Survey Data in Rural Ethiopia

BACKGROUND

Ethiopia has scaled up its community-based programs over the past decade by training and deploying health extension workers (HEWs) in rural communities throughout the country. Consequently, child mortality has declined substantially, placing Ethiopia among the few countries that have achieved the United Nations' fourth Millennium Development Goal. As Ethiopia continues its efforts, results must be assessed regularly to provide timely feedback for improvement and to generate further support for programs. More specifically the expansion of HEWs at the community level provides a unique opportunity to build a system for real-time monitoring of births and deaths, linked to a civil registration and vital statistics system that Ethiopia is also developing. We tested the accuracy and completeness of births and deaths reported by trained HEWs for monitoring child mortality over 15 -month periods.

METHODS AND FINDINGS

HEWs were trained in 93 randomly selected rural kebeles in Jimma and West Hararghe zones of the Oromia region to report births and deaths over a 15-month period from January, 2012 to March, 2013. Completeness of number of births and deaths, age distribution of deaths, and accuracy of resulting under-five, infant, and neonatal mortality rates were assessed against data from a large household survey with full birth history from women aged 15-49. Although, in general HEWs, were able to accurately report events that they identified, the completeness of number of births and deaths reported over twelve-month periods was very low and variable across the two zones. Compared to household survey estimates, HEWs reported only about 30% of births and 21% of under-five deaths occurring in their communities over a twelve-month period. The under-five mortality rate was under-estimated by around 30%, infant mortality rate by 23% and neonatal mortality by 17%. HEWs reported disproportionately higher number of deaths among the very young infants than among the older children.

CONCLUSION

Birth and death data reported by HEWs are not complete enough to support the monitoring of changes in childhood mortality. HEWs can significantly contribute to the success of a CRVS in Ethiopia, but cannot be relied upon as the sole source for identification of vital events. Further studies are needed to understand how to increase the level of completeness.

Using routine health information systems for well-designed health evaluations in low- and middle-income countries

Routine health information systems (RHISs) are in place in nearly every country and provide routinely collected full-coverage records on all levels of health system service delivery. However, these rich sources of data are regularly overlooked for evaluating causal effects of health programmes due to concerns regarding completeness, timeliness, representativeness and accuracy. Using Mozambique's national RHIS (Módulo Básico) as an illustrative example, we urge renewed attention to the use of RHIS data for health evaluations. Interventions to improve data quality exist and have been tested in low-and middle-income countries (LMICs). Intrinsic features of RHIS data (numerous repeated observations over extended periods of time, full coverage of health facilities, and numerous real-time indicators of service coverage and utilization) provide for very robust quasi-experimental designs, such as controlled interrupted time-series (cITS), which are not possible with intermittent community sample surveys. In addition, cITS analyses are well suited for continuously evolving development contexts in LMICs by: (1) allowing for measurement and controlling for trends and other patterns before, during and after intervention implementation; (2) facilitating the use of numerous simultaneous control groups and non-equivalent dependent variables at multiple nested levels to increase validity and strength of causal inference; and (3) allowing the integration of continuous 'effective dose received' implementation measures. With expanded use of RHIS data for the evaluation of health programmes, investments in data systems, health worker interest in and utilization of RHIS data, as well as data quality will further increase over time. Because RHIS data are ministry-owned and operated, relying upon these data will contribute to sustainable national capacity over time.

Effects of a health information system data quality intervention on concordance in Mozambique: time-series analyses from 2009-2012

Background

We assessed the effects of a three-year national-level, ministry-led health information system (HIS) data quality intervention and identified associated health facility factors.

Methods

Monthly summary HIS data concordance between a gold standard data quality audit and routine HIS data was assessed in 26 health facilities in Sofala Province, Mozambique across four indicators (outpatient consults, institutional births, first antenatal care visits, and third dose of diphtheria, pertussis, and tetanus vaccination) and five levels of health system data aggregation (daily facility paper registers, monthly paper facility reports, monthly paper district reports, monthly electronic district reports, and monthly electronic provincial reports) through retrospective yearly audits conducted July-August 2010–2013. We used mixed-effects linear models to quantify changes in data quality over time and associated health system determinants.

Results

Median concordance increased from 56.3% during the baseline period (2009–2010) to 87.5% during 2012–2013. Concordance improved by 1.0% (confidence interval [CI]: 0.60, 1.5) per month during the intervention period of 2010–2011 and 1.6% (CI: 0.89, 2.2) per month from 2011–2012. No significant improvements were observed from 2009–2010 (during baseline period) or 2012–2013. Facilities with more technical staff (aβ: 0.71; CI: 0.14, 1.3), more first antenatal care visits (aβ: 3.3; CI: 0.43, 6.2), and fewer clinic beds (aβ: -0.94; CI: −1.7, −0.20) showed more improvements. Compared to facilities with no stock-outs, facilities with five essential drugs stocked out had 51.7% (CI: −64.8 -38.6) lower data concordance.

Conclusions

A data quality intervention was associated with significant improvements in health information system data concordance across public-sector health facilities in rural and urban Mozambique. Concordance was higher at those facilities with more human resources for health and was associated with fewer clinic-level stock-outs of essential medicines. Increased investments should be made in data audit and feedback activities alongside targeted efforts to improve HIS data in low- and middle-income countries.

The unfinished agenda in child survival

10 years ago, The Lancet published a Series about child survival. In this Review, we examine progress in the past decade in child survival, with a focus on epidemiology, interventions and intervention coverage, strategies of health programmes, equity, evidence, accountability, and global leadership. Knowledge of child health epidemiology has greatly increased, and although more and better interventions are available, they still do not reach large numbers of mothers and children. Child survival should remain at the heart of global goals in the post-2015 era. Many countries are now making good progress and need the time and support required to finish the task. The global health community should show its steadfast commitment to child survival by amassing knowledge and experience as a basis for ever more effective programmes. Leadership and accountability for child survival should be strengthened and shared among the UN system; governments in high-income, middle-income, and low-income countries; and non-governmental organisations.