Methodological Considerations for Use of Routine Health Information System Data to Evaluate Malaria Program Impact in an Era of Declining Malaria Transmission

Impact of seasonal malaria chemoprevention based on the number of medicines doses received on malaria burden among children aged 3-59 months in Nigeria: A propensity score-matched analysis

BACKGROUND

Seasonal malaria chemoprevention using sulfadoxine-pyrimethamine plus amodiaquine (sulfadoxine-pyrimethamine plus amodiaquine on Day 1 and amodiaquine on both Day 2 and Day 3) is delivered to children aged 3-59 months in areas of highly season malaria transmission. While the overall population-level impact of seasonal malaria chemoprevention on malaria control has been documented in various countries and time periods, there is no clear evidence regarding seasonal malaria chemoprevention impact based on the number of medicine doses children receive in one cycle in routine programmatic conditions.

METHODS

Data were extracted from Nigeria's routinely collected seasonal malaria chemoprevention end-of-round coverage surveys (2021, 2022). We matched seasonal malaria chemoprevention-targeted children who received specific numbers of seasonal malaria chemoprevention medicines with those who did not receive any doses of seasonal malaria chemoprevention medicines (non-sulfadoxine-pyrimethamine plus amodiaquine) using multiple sets of propensity score matches. We performed multilevel logistic regression for each matched group to evaluate the association between the number of doses of seasonal malaria chemoprevention medicines and monthly confirmed malaria cases (caregiver-reported malaria infection diagnosed by rapid diagnostic test at a health facility following the penultimate cycle of seasonal malaria chemoprevention).

RESULTS

Among 21,621 SMC-targeted children, 9.7% received non-sulfadoxine-pyrimethamine plus amodiaquine, 0.5% received only Day 1 sulfadoxine-pyrimethamine plus amodiaquine, 1.0% received Day 1 sulfadoxine-pyrimethamine plus amodiaquine and either Day 2 amodiaquine or Day 3 amodiaquine (sulfadoxine-pyrimethamine plus amodiaquine + amodiaquine), and 88.8% received Day 1 sulfadoxine-pyrimethamine plus amodiaquine and both Day 2 and Day 3 amodiaquine (sulfadoxine-pyrimethamine plus amodiaquine + amodiaquine + amodiaquine). Children receiving only Day 1 sulfadoxine-pyrimethamine plus amodiaquine did not have significant lower odds of rapid diagnostic tests-confirmed malaria than those receiving non-sulfadoxine-pyrimethamine plus amodiaquine (OR 0.77, 0.42-1.42). However, children receiving sulfadoxine-pyrimethamine plus amodiaquine + amodiaquine had significantly lower odds of rapid diagnostic tests-confirmed malaria than those receiving non-sulfadoxine-pyrimethamine plus amodiaquine (OR 0.42, 95% CI 0.28-0.63). Similarly, children receiving sulfadoxine-pyrimethamine plus amodiaquine + amodiaquine + amodiaquine also had significantly lower odds of rapid diagnostic test-confirmed malaria than those receiving non-sulfadoxine-pyrimethamine plus amodiaquine (OR 0.54, 95% CI 0.47-0.62).

CONCLUSION

Adherence to at least one daily dose of amodiaquine administration following receipt of Day 1 sulfadoxine-pyrimethamine plus amodiaquine by eligible children is crucial to ensure the effectiveness of seasonal malaria chemoprevention. This demonstrates the importance of enhancing caregiver awareness regarding the importance of amodiaquine and identifying barriers toward amodiaquine administration at the community level.

Reduction of malaria case incidence following the introduction of clothianidin-based indoor residual spraying in previously unsprayed districts: an observational analysis using health facility register data from Côte d'Ivoire, 2018-2022

BACKGROUND

Indoor residual spraying (IRS) using neonicotinoid-based insecticides (clothianidin and combined clothianidin with deltamethrin) was deployed in two previously unsprayed districts of Côte d'Ivoire in 2020 and 2021 to complement standard pyrethroid insecticide-treated nets. This retrospective observational study uses health facility register data to assess the impact of IRS on clinically reported malaria case incidence.

METHODS

Health facility data were abstracted from consultation registers for the period September 2018 to April 2022 in two IRS districts and two control districts that did not receive IRS. Malaria cases reported by community health workers (CHWs) were obtained from district reports and District Health Information Systems 2. Facilities missing complete data were excluded. Controlled interrupted time series models were used to estimate the effect of IRS on monthly all-ages population-adjusted confirmed malaria cases and cases averted by IRS. Models controlled for transmission season, precipitation, vegetation, temperature, proportion of cases reported by CHWs, proportion of tested out of suspected cases and non-malaria outpatient visits.

RESULTS

An estimated 10 988 (95% CI 5694 to 18 188) malaria cases were averted in IRS districts the year following the 2020 IRS campaign, representing a 15.9% reduction compared with if IRS had not been deployed. Case incidence in IRS districts dropped by 27.7% (incidence rate ratio (IRR) 0.723, 95% CI 0.592 to 0.885) the month after the campaign. In the 8 months after the 2021 campaign, 14 170 (95% CI 13 133 to 15 025) estimated cases were averted, a 24.7% reduction, and incidence in IRS districts dropped by 37.9% (IRR 0.621, 95% CI 0.462 to 0.835) immediately after IRS. Case incidence in control districts did not change following IRS either year (p>0.05) and the difference in incidence level change between IRS and control districts was significant both years (p<0.05).

CONCLUSION

Deployment of clothianidin-based IRS was associated with a reduction in malaria case rates in two districts of Côte d'Ivoire following IRS deployment in 2020 and 2021.

Malaria trends in districts that were targeted and not-targeted for seasonal malaria chemoprevention in children under 5 years of age in Guinea, 2014-2021

BACKGROUND

Seasonal malaria chemoprevention (SMC) is a main intervention to prevent and reduce childhood malaria. Since 2015, Guinea has implemented SMC targeting children aged 3-59 months (CU5) in districts with high and seasonal malaria transmission.

OBJECTIVE

We assessed the programmatic impact of SMC in Guinea's context of scaled up malaria intervention programming by comparing malaria-related outcomes in 14 districts that had or had not been targeted for SMC.

METHODS

Using routine health management information system data, we compared the district-level monthly test positivity rate (TPR) and monthly uncomplicated and severe malaria incidence for the whole population and disaggregated age groups (<5 years and ≥5 years of age). Changes in malaria indicators through time were analysed by calculating the district-level compound annual growth rate (CAGR) from 2014 to 2021; we used statistical analyses to describe trends in tested clinical cases, TPR, uncomplicated malaria incidence and severe malaria incidence.

RESULTS

The CAGR of TPR of all age groups was statistically lower in SMC (median=-7.8%) compared with non-SMC (median=-3.0%) districts. Similarly, the CAGR in uncomplicated malaria incidence was significantly lower in SMC (median=1.8%) compared with non-SMC (median=11.5%) districts. For both TPR and uncomplicated malaria incidence, the observed difference was also significant when age disaggregated. The CAGR of severe malaria incidence showed that all age groups experienced a decline in severe malaria in both SMC and non-SMC districts. However, this decline was significantly higher in SMC (median=-22.3%) than in non-SMC (median=-5.1%) districts for the entire population, as well as both CU5 and people over 5 years of age.

CONCLUSION

Even in an operational programming context, adding SMC to the malaria intervention package yields a positive epidemiological impact and results in a greater reduction in TPR, as well as the incidence of uncomplicated and severe malaria in CU5.

From efficacy to effectiveness: a comprehensive framework for monitoring, evaluating and optimizing seasonal malaria chemoprevention programmes

BACKGROUND

Seasonal Malaria Chemoprevention (SMC) is a highly effective intervention for preventing malaria, particularly in areas with highly seasonal transmission. Monitoring and evaluating (M&E) SMC programmes are complex due to the scale, time-sensitive delivery of the programme, and influence of external factors. This paper describes the process followed to develop a comprehensive M&E framework tailored specifically for the SMC context.

METHODS

The Framework was developed through a literature and programme review, and stakeholder dialogues across three implementing countries-Burkina Faso, Chad, and Nigeria. Expert consultation further refined the Framework through an iterative approach drawing upon data collected through the three sources. The Framework was designed using the Logical Framework Approach incorporating external factors and intentionally aligned with global malaria M&E standards.

RESULTS

An overall aim and seven programme objectives were developed measured by 70 indicators. The indicators also capture the causal links between the implementation and results of the programme. The Framework leverages the use of current data sources and existing mechanisms, ensuring efficient data use without requiring a significant increase in resources for overall programme optimization. It also promotes the use of data triangulation, and stratification for a more nuanced understanding of factors affecting programme performance and timely data informed decision-making.

CONCLUSIONS

The SMC M&E Framework presented here provides a standardized approach for programme implementers to enhance decision-making for optimal programme performance. This is an essential tool as the scope of SMC programmes expands to new geographies and target age groups.

Effectiveness of community case management of malaria on severe malaria and inpatient malaria deaths in Zambia: a dose-response study using routine health information system data

BACKGROUND

Community case management of malaria (CCM) has been expanded in many settings, but there are limited data describing the impact of these services in routine implementation settings or at large scale. Zambia has intensively expanded CCM since 2013, whereby trained volunteer community health workers (CHW) use rapid diagnostic tests and artemether-lumefantrine to diagnose and treat uncomplicated malaria.

METHODS

This retrospective, observational study explored associations between changing malaria service point (health facility or CHW) density per 1000 people and severe malaria admissions or malaria inpatient deaths by district and month in a dose-response approach, using existing routine and programmatic data. Negative binomial generalized linear mixed-effect models were used to assess the impact of increasing one additional malaria service point per 1000 population, and of achieving Zambia's interim target of 1 service point per 750 population. Access to insecticide-treated nets, indoor-residual spraying, and rainfall anomaly were included in models to reduce potential confounding.

RESULTS

The study captured 310,855 malaria admissions and 7158 inpatient malaria deaths over 83 districts (seven provinces) from January 2015 to May 2020. Total CHWs increased from 43 to 4503 during the study period, while health facilities increased from 1263 to 1765. After accounting for covariates, an increase of one malaria service point per 1000 was associated with a 19% reduction in severe malaria admissions among children under five (incidence rate ratio [IRR] 0.81, 95% confidence interval [CI] 0.75-0.87, p < 0.001) and 23% reduction in malaria deaths among under-fives (IRR 0.77, 95% CI 0.66-0.91). After categorizing the exposure of population per malaria service point, there was evidence for an effect on malaria admissions and inpatient malaria deaths among children under five only when reaching the target of one malaria service point per 750 population.

CONCLUSIONS

CCM is an effective strategy for preventing severe malaria and deaths in areas such as Zambia where malaria diagnosis and treatment access remains challenging. These results support the continued investment in CCM scale-up in similar settings, to improve access to malaria diagnosis and treatment.

The use of routine health facility data for micro-stratification of malaria risk in mainland Tanzania

Background

Current efforts to estimate the spatially diverse malaria burden in malaria-endemic countries largely involve the use of epidemiological modelling methods for describing temporal and spatial heterogeneity using sparse interpolated prevalence data from periodic cross-sectional surveys. However, more malaria-endemic countries are beginning to consider local routine data for this purpose. Nevertheless, routine information from health facilities (HFs) remains widely under-utilized despite improved data quality, including increased access to diagnostic testing and the adoption of the electronic District Health Information System (DHIS2). This paper describes the process undertaken in mainland Tanzania using routine data to develop a high-resolution, micro-stratification risk map to guide future malaria control efforts.

Methods

Combinations of various routine malariometric indicators collected from 7098 HFs were assembled across 3065 wards of mainland Tanzania for the period 2017–2019. The reported council-level prevalence classification in school children aged 5–16 years (PfPR5–16) was used as a benchmark to define four malaria risk groups. These groups were subsequently used to derive cut-offs for the routine indicators by minimizing misclassifications and maximizing overall agreement. The derived-cutoffs were converted into numbered scores and summed across the three indicators to allocate wards into their overall risk stratum.

Results

Of 3065 wards, 353 were assigned to the very low strata (10.5% of the total ward population), 717 to the low strata (28.6% of the population), 525 to the moderate strata (16.2% of the population), and 1470 to the high strata (39.8% of the population). The resulting micro-stratification revealed malaria risk heterogeneity within 80 councils and identified wards that would benefit from community-level focal interventions, such as community-case management, indoor residual spraying and larviciding.

Conclusion

The micro-stratification approach employed is simple and pragmatic, with potential to be easily adopted by the malaria programme in Tanzania. It makes use of available routine data that are rich in spatial resolution and that can be readily accessed allowing for a stratification of malaria risk below the council level. Such a framework is optimal for supporting evidence-based, decentralized malaria control planning, thereby improving the effectiveness and allocation efficiency of malaria control interventions.

Long-lasting insecticidal nets provide protection against malaria for only a single year in Burundi, an African highland setting with marked malaria seasonality

BACKGROUND

Long-lasting insecticidal nets (LLINs) are one of the key interventions in the global fight against malaria. Since 2014, mass distribution campaigns of LLINs aim for universal access by all citizens of Burundi. In this context, we assess the impact of LLINs mass distribution campaigns on malaria incidence, focusing on the endemic highland health districts. We also explored the possible correlation between observed trends in malaria incidence with any variations in climate conditions.

METHODS

Malaria cases for 2011-2019 were obtained from the National Health Information System. We developed a generalised additive model based on a time series of routinely collected data with malaria incidence as the response variable and timing of LLIN distribution as an explanatory variable to investigate the duration and magnitude of the LLIN effect on malaria incidence. We added a seasonal and continuous-time component as further explanatory variables, and health district as a random effect to account for random natural variation in malaria cases between districts.

RESULTS

Malaria transmission in Burundian highlands was clearly seasonal and increased non-linearly over the study period. Further, a fast and steep decline of malaria incidence was noted during the first year after mass LLIN distribution (p<0.0001). In years 2 and 3 after distribution, malaria cases started to rise again to levels higher than before the control intervention.

CONCLUSION

This study highlights that LLINs did reduce the incidence in the first year after a mass distribution campaign, but in the context of Burundi, LLINs lost their impact after only 1 year.

Impact of seasonal malaria chemoprevention on prevalence of malaria infection in malaria indicator surveys in Burkina Faso and Nigeria

BACKGROUND

In 2012, the WHO issued a policy recommendation for the use of seasonal malaria chemoprevention (SMC) to children 3-59 months in areas of highly seasonal malaria transmission. Clinical trials have found SMC to prevent around 75% of clinical malaria. Impact under routine programmatic conditions has been assessed during research studies but there is a need to identify sustainable methods to monitor impact using routinely collected data.

METHODS

Data from Demographic Health Surveys were merged with rainfall, geographical and programme data in Burkina Faso (2010, 2014, 2017) and Nigeria (2010, 2015, 2018) to assess impact of SMC. We conducted mixed-effects logistic regression to predict presence of malaria infection in children aged 6-59 months (rapid diagnostic test (RDT) and microscopy, separately).

RESULTS

We found strong evidence that SMC administration decreases odds of malaria measured by RDT during SMC programmes, after controlling for seasonal factors, age, sex, net use and other variables (Burkina Faso OR 0.28, 95% CI 0.21 to 0.37, p<0.001; Nigeria OR 0.40, 95% CI 0.30 to 0.55, p<0.001). The odds of malaria were lower up to 2 months post-SMC in Burkina Faso (1-month post-SMC: OR 0.29, 95% CI 0.12 to 0.72, p=0.01; 2 months post-SMC: OR: 0.33, 95% CI 0.17 to 0.64, p<0.001). The odds of malaria were lower up to 1 month post-SMC in Nigeria but was not statistically significant (1-month post-SMC 0.49, 95% CI 0.23 to 1.05, p=0.07). A similar but weaker effect was seen for microscopy (Burkina Faso OR 0.38, 95% CI 0.29 to 0.52, p<0.001; Nigeria OR 0.53, 95% CI 0.38 to 0.76, p<0.001).

CONCLUSIONS

Impact of SMC can be detected in reduced prevalence of malaria from data collected through household surveys if conducted during SMC administration or within 2 months afterwards. Such evidence could contribute to broader evaluation of impact of SMC programmes.

A quasi-experimental study estimating the impact of long-lasting insecticidal nets with and without piperonyl butoxide on pregnancy outcomes

Background

Long-lasting insecticidal nets (LLINs) are the main vector control tool for pregnant women, but their efficacy may be compromised, in part, due to pyrethroid resistance. In 2017, the Ugandan Ministry of Health embedded a cluster randomized controlled trial into the national LLIN campaign, where a random subset of health subdistricts (HSDs) received LLINs treated with piperonyl butoxide (PBO), a chemical synergist known to partially restore pyrethroid sensitivity. Using data from a small, non-randomly selected subset of HSDs, this secondary analysis used quasi-experimental methods to quantify the overall impact of the LLIN campaign on pregnancy outcomes. In an exploratory analysis, differences between PBO and conventional (non-PBO) LLINs on pregnancy outcomes were assessed.

Methods

Birth registry data (n = 39,085) were retrospectively collected from 21 health facilities across 12 HSDs, 29 months before and 9 months after the LLIN campaign (from 2015 to 2018). Of the 12 HSDs, six received conventional LLINs, five received PBO LLINs, and one received a mix of conventional and PBO LLINs. Interrupted time-series analyses (ITSAs) were used to estimate changes in monthly incidence of stillbirth and low birthweight (LBW; <2500 g) before-and-after the campaign. Poisson regression with robust standard errors modeled campaign effects, adjusting for health facility-level differences, seasonal variation, and time-varying maternal characteristics. Comparisons between PBO and conventional LLINs were estimated using difference-in-differences estimators.

Results

ITSAs estimated the campaign was associated with a 26% [95% CI: 7–41] reduction in stillbirth incidence (incidence rate ratio (IRR) = 0.74 [0.59–0.93]) and a 15% [-7, 33] reduction in LBW incidence (IRR=0.85 [0.67–1.07]) over a 9-month period. The effect on stillbirth incidence was greatest for women delivering 7–9 months after the campaign (IRR=0.60 [0.41–0.87]) for whom the LLINs would have covered most of their pregnancy. The IRRs estimated from difference-in-differences analyses comparing PBO to conventional LLINs was 0.78 [95% CI: 0.52, 1.16] for stillbirth incidence and 1.15 [95% CI: 0.87, 1.52] for LBW incidence.

Conclusions

In this region of Uganda, where pyrethroid resistance is high, this study found that a mass LLIN campaign was associated with reduced stillbirth incidence. Effects of the campaign were greatest for women who would have received LLINs early in pregnancy, suggesting malaria protection early in pregnancy can have important benefits that are not necessarily realized through antenatal malaria services. Results from the exploratory analyses comparing PBO and conventional LLINs on pregnancy outcomes were inconclusive, largely due to the wide confidence intervals that crossed the null. Thus, future studies with larger sample sizes are needed.

Resurgence of malaria in Uganda despite sustained indoor residual spraying and repeated long lasting insecticidal net distributions

Five years of sustained indoor residual spraying (IRS) of insecticide from 2014 to 2019, first using a carbamate followed by an organophosphate, was associated with a marked reduction in the incidence of malaria in five districts of Uganda. We assessed changes in malaria incidence over an additional 21 months, corresponding to a change in IRS formulations using clothianidin with and without deltamethrin. Using enhanced health facility surveillance data, our objectives were to 1) estimate the impact of IRS on monthly malaria case counts at five surveillance sites over a 6.75 year period, and 2) compare monthly case counts at five facilities receiving IRS to ten facilities in neighboring districts not receiving IRS. For both objectives, we specified mixed effects negative binomial regression models with random intercepts for surveillance site adjusting for rainfall, season, care-seeking, and malaria diagnostic. Following the implementation of IRS, cases were 84% lower in years 4-5 (adjusted incidence rate ratio [aIRR] = 0.16, 95% CI 0.12-0.22), 43% lower in year 6 (aIRR = 0.57, 95% CI 0.44-0.74), and 39% higher in the first 9 months of year 7 (aIRR = 1.39, 95% CI 0.97-1.97) compared to pre-IRS levels. Cases were 67% lower in IRS sites than non-IRS sites in year 6 (aIRR = 0.33, 95% CI 0.17-0.63) but 38% higher in the first 9 months of year 7 (aIRR = 1.38, 95% CI 0.90-2.11). We observed a resurgence in malaria to pre-IRS levels despite sustained IRS. The timing of this resurgence corresponded to a change of active ingredient. Further research is needed to determine causality.

Seasonality of confirmed malaria cases from 2008 to 2017 in Togo: a time series analysis by health district and target group

Background

This study aimed to assess the seasonality of confirmed malaria cases in Togo and to provide new indicators of malaria seasonality to the National Malaria Control Programme (NMCP).

Methods

Aggregated data of confirmed malaria cases were collected monthly from 2008 to 2017 by the Togo’s NMCP and stratified by health district and according to three target groups: children < 5 years old, children ≥ 5 years old and adults, and pregnant women. Time series analysis was carried out for each target group and health district. Seasonal decomposition was used to assess the seasonality of confirmed malaria cases. Maximum and minimum seasonal indices, their corresponding months, and the ratio of maximum/minimum seasonal indices reflecting the importance of malaria transmission, were provided by health district and target group.

Results

From 2008 to 2017, 7,951,757 malaria cases were reported in Togo. Children < 5 years old, children ≥ 5 years old and adults, and pregnant women represented 37.1%, 57.7% and 5.2% of the confirmed malaria cases, respectively. The maximum seasonal indices were observed during or shortly after a rainy season and the minimum seasonal indices during the dry season between January and April in particular. In children < 5 years old, the ratio of maximum/minimum seasonal indices was higher in the north, suggesting a higher seasonal malaria transmission, than in the south of Togo. This is also observed in the other two groups but to a lesser extent.

Conclusions

This study contributes to a better understanding of malaria seasonality in Togo. The indicators of malaria seasonality could allow for more accurate forecasting in malaria interventions and supply planning throughout the year.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12879-021-06893-z.

Health management information system (HMIS) data quality and associated factors in Massaguet district, Chad

Background

Quality data from Health Management Information Systems (HMIS) are important for tracking the effectiveness of malaria control interventions. However, HMIS data in many resource-limited settings do not currently meet standards set by the World Health Organization (WHO). We aimed to assess HMIS data quality and associated factors in Chad.

Methods

A cross-sectional study was conducted in 14 health facilities in Massaguet district. Data on children under 15 years were obtained from the HMIS and from the external patient register covering the period January–December 2018. An additional questionnaire was administered to 16 health centre managers to collect data on contextual variables. Patient registry data were aggregated and compared with the HMIS database at district and health centre level. Completeness and accuracy indicators were calculated as per WHO guidelines. Multivariate logistic regressions were performed on the Verification Factor for attendance, suspected and confirmed malaria cases for three age groups (1 to < 12 months, 1 to < 5 years and 5 to < 15 years) to identify associations between health centre characteristics and data accuracy.

Results

Health centres achieved a high level of data completeness in HMIS. Malaria data were over-reported in HMIS for children aged under 15 years. There was an association between workload and higher odds of inaccuracy in reporting of attendance among children aged 1 to < 5 years (Odds ratio [OR]: 10.57, 95% CI 2.32–48.19) and 5– < 15 years (OR: 6.64, 95% CI 1.38–32.04). Similar association was found between workload and stock-outs in register books, and inaccuracy in reporting of malaria confirmed cases. Meanwhile, we found that presence of a health technician, and of dedicated staff for data management, were associated with lower inaccuracy in reporting of clinic attendance in children aged under five years.

Conclusion

Data completeness was high while the accuracy was low. Factors associated with data inaccuracy included high workload and the unavailability of required data collection tools. The results suggest that improvement in working conditions for clinic personnel may improve HMIS data quality. Upgrading from paper-based forms to a web-based HMIS may provide a solution for improving data accuracy and its utility for future evaluations of health interventions. Results from this study can inform the Ministry of Health and it partners on the precautions to be taken in the use of HMIS data and inform initiatives for improving its quality.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01684-7.

The Potential of Digital Data Collection Tools for Long-lasting Insecticide-Treated Net Mass Campaigns in Nigeria: Formative Study

Background

Nigeria has the world’s largest malaria burden, accounting for 27% of the world’s malaria cases and 23% of malaria mortality globally. This formative study describes the operational process of the mass distribution of long-lasting insecticide-treated nets (LLINs) during a campaign program in Nigeria.

Objective

This study aims to assess whether and how digital data collection and management tools can change current practices and help resolve major implementation issues.

Methods

Qualitative data on the technical features and operational processes of paper-based and information and communication technology (ICT)–based systems in the Edo and Kwara states from June 2 to 30, 2017, were collected on the basis of documented operation manuals, field observations, and informant interviews. During the LLIN campaign in Edo State, we recruited 6 local government area focal persons and monitors and documented daily review meetings during household mobilization (9 days) and net distribution (5 days) to understand the major program implementation issues associated with the following three aspects: logistic issues, technical issues, and demand creation. Each issue was categorized according to the expected degree (low, mid, and high) of change by the ICT system.

Results

The net campaign started with microplanning and training, followed by a month-long implementation process, which included household mobilization, net movement, net distribution, and end process monitoring. The ICT system can improve management and oversight issues related to data reporting and processes through user-centered interface design, built-in data quality control logic flow or algorithms, and workflow automation. These often require more than 50% of staff time and effort in the current paper-based practice. Compared with the current paper-based system, the real-time system is expected to reduce the time to payment compensation for health workers by about 20 days and produce summary campaign statistics for at least 20 to 30 days.

Conclusions

The ICT system can facilitate the measurement of population coverage beyond program coverage during an LLIN campaign with greater data reliability and timeliness, which are often compromised due to the limited workforce capacity in a paper-based system.

Monthly biological larviciding associated with a tenfold decrease in larval density in fish farming ponds and reduced community-wide malaria incidence in northwestern Brazil

Background

Larvicides are typically applied to fixed and findable mosquito breeding sites, such as fish farming ponds used in commercial aquaculture, to kill immature forms and thereby reduce the size of adult malaria vector populations. However, there is little evidence suggesting that larviciding may suppress community-wide malaria transmission outside Africa. Here, we tested whether the biological larvicide VectoMax FG applied at monthly intervals to fish farming ponds can reduce malaria incidence in Amazonian Brazil.

Methods

This study was carried out in Vila Assis Brasil (VAB; population 1700), a peri-urban malaria hotspot in northwestern Brazil with a baseline annual parasite incidence of 553 malaria cases per 1000 inhabitants. The intervention consisted of monthly treatments with 20 kg/ha of VectoMax FG of all water-filled fish ponds in VAB (n ranging between 167 and 170) with a surface area between 20 and 8000 m², using knapsack power mistblowers. We used single-group interrupted time-series analysis to compare monthly larval density measurements in fish ponds during a 14-month pre-intervention period (September 2017–October 2018), with measurements made during November 2018–October 2019 and shortly after the 12-month intervention (November 2019). We used interrupted time-series analysis with a comparison group to contrast the malaria incidence trends in VAB and nearby nonintervention localities before and during the intervention.

Results

Average larval densities decreased tenfold in treated fish farming ponds, from 0.467 (95% confidence interval [CI], 0.444–0.490) anopheline larvae per dip pre-intervention (September 2017–October 2018) to 0.046 (95% CI, 0.041–0.051) larvae per dip during (November 2018–October 2019) and shortly after the intervention (November 2019). Average malaria incidence rates decreased by 0.08 (95% CI, 0.04–0.11) cases per 100 person-months (P < 0.0001) during the intervention in VAB and remained nearly unchanged in comparison localities. We estimate that the intervention averted 24.5 (95% CI, 6.2–42.8) malaria cases in VAB between January and December 2019.

Conclusions

Regular larviciding is associated with a dramatic decrease in larval density and a modest but significant decrease in community-wide malaria incidence. Larviciding may provide a valuable complementary vector control strategy in commercial aquaculture settings across the Amazon.

Graphical abstract

Malaria micro-stratification using routine surveillance data in Western Kenya

BACKGROUND

There is an increasing need for finer spatial resolution data on malaria risk to provide micro-stratification to guide sub-national strategic plans. Here, spatial-statistical techniques are used to exploit routine data to depict sub-national heterogeneities in test positivity rate (TPR) for malaria among patients attending health facilities in Kenya.

METHODS

Routine data from health facilities (n = 1804) representing all ages over 24 months (2018-2019) were assembled across 8 counties (62 sub-counties) in Western Kenya. Statistical model-based approaches were used to quantify heterogeneities in TPR and uncertainty at fine spatial resolution adjusting for missingness, population distribution, spatial data structure, month, and type of health facility.

RESULTS

The overall monthly reporting rate was 78.7% (IQR 75.0-100.0) and public-based health facilities were more likely than private facilities to report ≥ 12 months (OR 5.7, 95% CI 4.3-7.5). There was marked heterogeneity in population-weighted TPR with sub-counties in the north of the lake-endemic region exhibiting the highest rates (exceedance probability > 70% with 90% certainty) where approximately 2.7 million (28.5%) people reside. At micro-level the lowest rates were in 14 sub-counties (exceedance probability < 30% with 90% certainty) where approximately 2.2 million (23.1%) people lived and indoor residual spraying had been conducted since 2017.

CONCLUSION

The value of routine health data on TPR can be enhanced when adjusting for underlying population and spatial structures of the data, highlighting small-scale heterogeneities in malaria risk often masked in broad national stratifications. Future research should aim at relating these heterogeneities in TPR with traditional community-level prevalence to improve tailoring malaria control activities at sub-national levels.

Estimating malaria incidence from routine health facility-based surveillance data in Uganda

Background

Accurate measures of malaria incidence are essential to track progress and target high-risk populations. While health management information system (HMIS) data provide counts of malaria cases, quantifying the denominator for incidence using these data is challenging because catchment areas and care-seeking behaviours are not well defined. This study’s aim was to estimate malaria incidence using HMIS data by adjusting the population denominator accounting for travel time to the health facility.

Methods

Outpatient data from two public health facilities in Uganda (Kihihi and Nagongera) over a 3-year period (2011–2014) were used to model the relationship between travel time from patient village of residence (available for each individual) to the facility and the relative probability of attendance using Poisson generalized additive models. Outputs from the model were used to generate a weighted population denominator for each health facility and estimate malaria incidence. Among children aged 6 months to 11 years, monthly HMIS-derived incidence estimates, with and without population denominators weighted by probability of attendance, were compared with gold standard measures of malaria incidence measured in prospective cohorts.

Results

A total of 48,898 outpatient visits were recorded across the two sites over the study period. HMIS incidence correlated with cohort incidence over time at both study sites (correlation in Kihihi = 0.64, p < 0.001; correlation in Nagongera = 0.34, p = 0.045). HMIS incidence measures with denominators unweighted by probability of attendance underestimated cohort incidence aggregated over the 3 years in Kihihi (0.5 cases per person-year (PPY) vs 1.7 cases PPY) and Nagongera (0.3 cases PPY vs 3.0 cases PPY). HMIS incidence measures with denominators weighted by probability of attendance were closer to cohort incidence, but remained underestimates (1.1 cases PPY in Kihihi and 1.4 cases PPY in Nagongera).

Conclusions

Although malaria incidence measured using HMIS underestimated incidence measured in cohorts, even when adjusting for probability of attendance, HMIS surveillance data are a promising and scalable source for tracking relative changes in malaria incidence over time, particularly when the population denominator can be estimated by incorporating information on village of residence.

Addressing challenges in routine health data reporting in Burkina Faso through Bayesian spatiotemporal prediction of weekly clinical malaria incidence

Sub-Saharan African (SSA) countries’ health systems are often vulnerable to unplanned situations that can hinder their effectiveness in terms of data completeness and disease control. For instance, in Burkina Faso following a workers' strike, comprehensive data on several diseases were unavailable for a long period in 2019. Weather, seasonal-malaria-chemoprevention (SMC), free healthcare, and other contextual data, which are purported to influence malarial disease, provide opportunities to fit models to describe the clinical malaria data and predict the disease spread. Bayesian spatiotemporal modeling was applied to weekly malaria surveillance data from Burkina Faso (2011–2018) while considering the effects of weather, health programs and contextual factors. Then, a prediction was used to deal with weekly missing data for the entire year of 2019, and SMC and free healthcare effects were quantified. Our proposed model accurately predicted weekly clinical malaria incidence (correlation coefficient, r = 0.90). The distribution of clinical malaria incidence was heterogeneous across the country. Overall, national predicted clinical malaria incidence in 2019 (605 per 1000 [95% CrI: 360–990]) increased by 24.7% compared with the year 2015. SMC and the interaction between free healthcare and health facility attendance were associated with a reduction in clinical malaria incidence. Our modeling approach could be a useful tool for strengthening health systems’ resilience by addressing data completeness and could support SSA countries in developing appropriate targets and indicators to facilitate the subnational control effort.

Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review

BACKGROUND

When randomisation is not possible, interrupted time series (ITS) design has increasingly been advocated as a more robust design to evaluating health system quality improvement (QI) interventions given its ability to control for common biases in healthcare QI. However, there is a potential risk of producing misleading results when this rather robust design is not used appropriately. We performed a methodological systematic review of the literature to investigate the extent to which the use of ITS has followed best practice standards and recommendations in the evaluation of QI interventions.

METHODS

We searched multiple databases from inception to June 2018 to identify QI intervention studies that were evaluated using ITS. There was no restriction on date, language and participants. Data were synthesised narratively using appropriate descriptive statistics. The risk of bias for ITS studies was assessed using the Cochrane Effective Practice and Organisation of Care standard criteria. The systematic review protocol was registered in PROSPERO (registration number: CRD42018094427).

RESULTS

Of 4061 potential studies and 2028 unique records screened for inclusion, 120 eligible studies assessed eight QI strategies and were from 25 countries. Most studies were published since 2010 (86.7%), reported data using monthly interval (71.4%), used ITS without a control (81%) and modelled data using segmented regression (62.5%). Autocorrelation was considered in 55% of studies, seasonality in 20.8% and non-stationarity in 8.3%. Only 49.2% of studies specified the ITS impact model. The risk of bias was high or very high in 72.5% of included studies and did not change significantly over time.

CONCLUSIONS

The use of ITS in the evaluation of health system QI interventions has increased considerably over the past decade. However, variations in methodological considerations and reporting of ITS in QI remain a concern, warranting a need to develop and reinforce formal reporting guidelines to improve its application in the evaluation of health system QI interventions.

Using routine health information data for research in low- and middle-income countries: a systematic review

BACKGROUND

Routine health information systems (RHISs) support resource allocation and management decisions at all levels of the health system, as well as strategy development and policy-making in many low- and middle-income countries (LMICs). Although RHIS data represent a rich source of information, such data are currently underused for research purposes, largely due to concerns over data quality. Given that substantial investments have been made in strengthening RHISs in LMICs in recent years, and that there is a growing demand for more real-time data from researchers, this systematic review builds upon the existing literature to summarize the extent to which RHIS data have been used in peer-reviewed research publications.

METHODS

Using terms 'routine health information system', 'health information system', or 'health management information system' and a list of LMICs, four electronic peer-review literature databases were searched from inception to February 202,019: PubMed, Scopus, EMBASE, and EconLit. Articles were assessed for inclusion based on pre-determined eligibility criteria and study characteristics were extracted from included articles using a piloted data extraction form.

RESULTS

We identified 132 studies that met our inclusion criteria, originating in 37 different countries. Overall, the majority of the studies identified were from Sub-Saharan Africa and were published within the last 5 years. Malaria and maternal health were the most commonly studied health conditions, although a number of other health conditions and health services were also explored.

CONCLUSIONS

Our study identified an increasing use of RHIS data for research purposes, with many studies applying rigorous study designs and analytic methods to advance program evaluation, monitoring and assessing services, and epidemiological studies in LMICs. RHIS data represent an underused source of data and should be made more available and further embraced by the research community in LMIC health systems.

Evaluating the Impact of Programmatic Mass Drug Administration for Malaria in Zambia Using Routine Incidence Data

Background

In 2016, the Zambian National Malaria Elimination Centre started programmatic mass drug administration (pMDA) campaigns with dihydroartemisinin-piperaquine as a malaria elimination tool in Southern Province. Two rounds were administered, 2 months apart (coverage 70% and 57%, respectively). We evaluated the impact of 1 year of pMDA on malaria incidence using routine data.

Methods

We conducted an interrupted time series with comparison group analysis on monthly incidence data collected at the health facility catchment area (HFCA) level, with a negative binomial model using generalized estimating equations. Programmatic mass drug administration was conducted in HFCAs with greater than 50 cases/1000 people per year. Ten HFCAs with incidence rates marginally above this threshold (pMDA group) were compared with 20 HFCAs marginally below (comparison group).

Results

The pMDA HFCAs saw a 46% greater decrease in incidence at the time of intervention than the comparison areas (incidence rate ratio = 0.536; confidence interval = 0.337–0.852); however, incidence increased toward the end of the season. No HFCAs saw a transmission interruption.

Conclusions

Programmatic mass drug administration, implemented during 1 year with imperfect coverage in low transmission areas with suboptimal vector control coverage, significantly reduced incidence. However, elimination will require additional tools. Routine data are important resources for programmatic impact evaluations and should be considered for future analyses.

Understanding the challenges associated with the use of data from routine health information systems in low- and middle-income countries: A systematic review

BACKGROUND

Routine health information systems (RHISs) are crucial to informing decision-making at all levels of the health system. However, the use of RHIS data in low- and middle-income countries (LMICs) is limited due to concerns regarding quality, accuracy, timeliness, completeness and representativeness.

OBJECTIVE

This study systematically reviewed technical, behavioural and organisational/environmental challenges that hinder the use of RHIS data in LMICs and strategies implemented to overcome these challenges.

METHOD

Four electronic databases were searched for studies describing challenges associated with the use of RHIS data and/or strategies implemented to circumvent these challenges in LMICs. Identified articles were screened against inclusion and exclusion criteria by two independent reviewers.

RESULTS

Sixty studies met the inclusion criteria and were included in this review, 55 of which described challenges in using RHIS data and 20 of which focused on strategies to address these challenges. Identified challenges and strategies were organised by their technical, behavioural and organisational/environmental determinants and by the core steps of the data process. Organisational/environmental challenges were the most commonly reported barriers to data use, while technical challenges were the most commonly addressed with strategies.

CONCLUSION

Despite the known benefits of RHIS data for health system strengthening, numerous challenges continue to impede their use in practice.

IMPLICATIONS

Additional research is needed to identify effective strategies for addressing the determinants of RHIS use, particularly given the disconnect identified between the type of challenge most commonly described in the literature and the type of challenge most commonly targeted for interventions.

Sub-national stratification of malaria risk in mainland Tanzania: a simplified assembly of survey and routine data

BACKGROUND

Recent malaria control efforts in mainland Tanzania have led to progressive changes in the prevalence of malaria infection in children, from 18.1% (2008) to 7.3% (2017). As the landscape of malaria transmission changes, a sub-national stratification becomes crucial for optimized cost-effective implementation of interventions. This paper describes the processes, data and outputs of the approach used to produce a simplified, pragmatic malaria risk stratification of 184 councils in mainland Tanzania.

METHODS

Assemblies of annual parasite incidence and fever test positivity rate for the period 2016-2017 as well as confirmed malaria incidence and malaria positivity in pregnant women for the period 2015-2017 were obtained from routine district health information software. In addition, parasite prevalence in school children (PfPR5to16) were obtained from the two latest biennial council representative school malaria parasitaemia surveys, 2014-2015 and 2017. The PfPR5to16 served as a guide to set appropriate cut-offs for the other indicators. For each indicator, the maximum value from the past 3 years was used to allocate councils to one of four risk groups: very low (< 1%PfPR5to16), low (1- < 5%PfPR5to16), moderate (5- < 30%PfPR5to16) and high (≥ 30%PfPR5to16). Scores were assigned to each risk group per indicator per council and the total score was used to determine the overall risk strata of all councils.

RESULTS

Out of 184 councils, 28 were in the very low stratum (12% of the population), 34 in the low stratum (28% of population), 49 in the moderate stratum (23% of population) and 73 in the high stratum (37% of population). Geographically, most of the councils in the low and very low strata were situated in the central corridor running from the north-east to south-west parts of the country, whilst the areas in the moderate to high strata were situated in the north-west and south-east regions.

CONCLUSION

A stratification approach based on multiple routine and survey malaria information was developed. This pragmatic approach can be rapidly reproduced without the use of sophisticated statistical methods, hence, lies within the scope of national malaria programmes across Africa.

Evaluating malaria programmes in moderate- and low-transmission settings: practical ways to generate robust evidence

BACKGROUND

Many countries have made substantial progress in scaling-up and sustaining malaria intervention coverage, leading to more focalized and heterogeneous transmission in many settings. Evaluation provides valuable information for programmes to understand if interventions have been implemented as planned and with quality, if the programme had the intended impact on malaria burden, and to guide programmatic decision-making. Low-, moderate-, and heterogeneous-transmission settings present unique evaluation challenges because of dynamic and targeted intervention strategies. This paper provides illustration of evaluation approaches and methodologies for these transmission settings, and suggests how to answer evaluation questions specific to the local context.

METHODS

The Roll Back Malaria Monitoring and Evaluation Reference Group formed a task force in October 2017 to lead development of this framework. The task force includes representatives from National Malaria Programmes, funding agencies, and malaria research and implementing partners. The framework builds on existing guidance for process and outcome evaluations and impact evaluations specifically in high transmission settings.

RESULTS

The theory of change describes how evaluation questions asked by national malaria programmes in different contexts influence evaluation design. The transmission setting, existing stratification, and data quality and availability are also key considerations. The framework is intended for adaption by countries to their local context, and use for evaluation at sub-national level. Confirmed malaria incidence is recommended as the primary impact indicator due to its sensitivity to detect changes in low-transmission settings. It is expected that process evaluations provide sufficient evidence for programme monitoring and improvement, while impact evaluations are needed following adoption of new mixes of interventions, operational strategies, tools or policies, particularly in contexts of changing malaria epidemiology. Impact evaluations in low-, moderate-, or heterogeneous-transmission settings will likely use plausibility designs, and methods highlighted by the framework include interrupted time series, district-level dose-response analyses, and constructed control methods. Triangulating multiple data sources and analyses is important to strengthen the plausibility argument.

CONCLUSIONS

This framework provides a structure to assist national malaria programmes and partners to design evaluations in low-, moderate- or heterogeneous-transmission settings. Emphasizing a continuous cycle along the causal pathway linking process evaluation to impact evaluation and then programmatic decision-making, the framework provides practical guidance in evaluation design, analysis, and interpretation to ensure that the evaluation meets national malaria programme priority questions and guides decision-making at national and sub-national levels.

Introduction and Evaluation of an Electronic Tool for Improved Data Quality and Data Use during Malaria Case Management Supportive Supervision

Although on-site supervision programs are implemented in many countries to assess and improve the quality of care, few publications have described the use of electronic tools during health facility supervision. The President's Malaria Initiative-funded MalariaCare project developed the MalariaCare Electronic Data System (EDS), a custom-built, open-source, Java-based, Android application that links to District Health Information Software 2, for data storage and visualization. The EDS was used during supervision visits at 4,951 health facilities across seven countries in Africa. The introduction of the EDS led to dramatic improvements in both completeness and timeliness of data on the quality of care provided for febrile patients. The EDS improved data completeness by 47 percentage points (42-89%) on average when compared with paper-based data collection. The average time from data submission to a final data analysis product dropped from over 5 months to 1 month. With more complete and timely data available, the Ministry of Health and the National Malaria Control Program (NMCP) staff could more effectively plan corrective actions and promptly allocate resources, ultimately leading to several improvements in the quality of malaria case management. Although government staff used supervision data during MalariaCare-supported lessons learned workshops to develop plans that led to improvements in quality of care, data use outside of these workshops has been limited. Additional efforts are required to institutionalize the use of supervision data within ministries of health and NMCPs.

Trends in reported malaria cases and the effects of malaria control in the Democratic Republic of the Congo

Background

Considerable upscaling of malaria control efforts have taken place over the last 15 years in the Democratic Republic of Congo, the country with the second highest malaria case load after Nigeria. Malaria control interventions have been strengthened in line with the Millenium Development Goals. We analysed the effects of these interventions on malaria cases at health facility level, using a retrospective trend analysis of malaria cases between 2005 and 2014. Data were collected from outpatient and laboratory registers based on a sample of 175 health facilities that represents all eco-epidemiological malaria settings across the country.

Methods

We applied a time series analysis to assess trends of suspected and confirmed malaria cases, by health province and for different age groups. A linear panel regression model controlled for non-malaria outpatient cases, rain fall, nightlight intensity, health province and time fixed effects, was used to examine the relationship between the interventions and malaria case occurrences, as well as test positivity rates.

Results

Overall, recorded suspected and confirmed malaria cases in the DRC have increased. The sharp increase in confirmed cases from 2010 coincides with the introduction of the new treatment policy and the resulting scale-up of diagnostic testing. Controlling for confounding factors, the introduction of rapid diagnostic tests (RDTs) was significantly associated with the number of tested and confirmed cases. The test positivity rate fluctuated around 40% without showing any trend.

Conclusion

The sharp increase in confirmed malaria cases from 2010 is unlikely to be due to a resurgence of malaria, but is clearly associated with improved diagnostic availability, mainly the introduction of RDTs. Before that, a great part of malaria cases were treated based on clinical suspicion. This finding points to a better detection of cases that potentially contributed to improved case management. Furthermore, the expansion of diagnostic testing along with the increase in confirmed cases implies that before 2010, cases were underreported, and that the accuracy of routine data to describe malaria incidence has improved.

Use of Routine Health Information System Data to Evaluate Impact of Malaria Control Interventions in Zanzibar, Tanzania from 2000 to 2015

BACKGROUND

Impact evaluations allow countries to assess public health gains achieved through malaria investments. This study uses routine health management information system (HMIS) data from Zanzibar to describe changes in confirmed malaria incidence and impact of case management and vector control interventions during 2000-2015.

METHODS

HMIS data from 129 (82%) public outpatient facilities were analyzed using interrupted time series models to estimate the impact of artemisinin-based combination therapy (ACT), indoor residual spray, and long-lasting insecticidal nets. Evaluation periods were defined as pre-intervention (January 2000 to August 2003), ACT-only (September 2003 to December 2005) and ACT plus vector control (2006-2015).

FINDINGS

After accounting for climate, seasonality, diagnostic testing rates, and outpatient attendance, average monthly incidence of confirmed malaria showed no trend over the pre-intervention period 2000-2003 (incidence rate ratio (IRR) 0.998, 95% CI 0.995-1.000). During the ACT-only period (2003-2005), the average monthly malaria incidence rate declined compared to the pre-intervention period, showing an overall declining trend during the ACT-only period (IRR 0.984, 95% CI 0.978-0.990). There was no intercept change at the start of the ACT-only period (IRR 1.081, 95% CI 0.968-1.208), but a drop in intercept was identified at the start of the ACT plus vector control period (IRR 0.683, 95% CI 0.597-0.780). During the ACT plus vector control period (2006-2015), the rate of decline in average monthly malaria incidence slowed compared to the ACT-only period, but the incidence rate continued to show an overall slight declining trend during 2006-2015 (IRR 0.993, 95% CI 0.992-0.994).

INTERPRETATION

This study presents a rigorous approach to the use of HMIS data in evaluating the impact of malaria control interventions. Evidence is presented for a rapid decline in malaria incidence during the period of ACT roll out compared to pre-intervention, with a rapid drop in malaria incidence following introduction of vector control and a slower declining incidence trend thereafter.

Evaluation of direct and indirect effects of seasonal malaria chemoprevention in Mali

Randomized controlled trials have established that seasonal malaria chemoprevention (SMC) in children is a promising strategy to reduce malaria transmission in Sahelian West Africa. This strategy was recently introduced in a dozen countries, and about 12 million children received SMC in 2016. However, evidence on SMC effectiveness under routine programme conditions is sparse. We aim to measure the effects of the nationwide SMC programme in Mali on the prevalence of malaria and anemia in children 6-59 months. We used data from the 2015 nationally representative malaria indicator survey. A post-test only with non-randomized control group study was designed. We fitted a generalized structural equation model that controlled for potential bias on observed and non-observed variables (endogenous treatment effect model). Having received SMC reduced by 44% (95% CI [0.39-0.49]) the risk of having a positive rapid diagnostic test for malaria. In addition, the programme indirectly reduced by 18% the risk of moderate-to-severe anemia (95% CI [0.15-0.21]). SMC in Mali has substantial protective effects under routine nationwide programme conditions. Endogenous treatment effects analyses can contribute to rigorously measuring the effectiveness of health programmes and to bridging a widening gap in evaluation methods to measure progress towards achieving malaria elimination.