Using routine health information systems for well-designed health evaluations in low- and middle-income countries

Effect of Performance-Based Nonfinancial Incentives on Data Quality in Individual Medical Records of Institutional Births: Quasi-Experimental Study

BACKGROUND

Despite the potential of routine health information systems in tackling persistent maternal deaths stemming from poor service quality at health facilities during and around childbirth, research has demonstrated their suboptimal performance, evident from the incomplete and inaccurate data unfit for practical use. There is a consensus that nonfinancial incentives can enhance health care providers' commitment toward achieving the desired health care quality. However, there is limited evidence regarding the effectiveness of nonfinancial incentives in improving the data quality of institutional birth services in Ethiopia.

OBJECTIVE

This study aimed to evaluate the effect of performance-based nonfinancial incentives on the completeness and consistency of data in the individual medical records of women who availed institutional birth services in northwest Ethiopia.

METHODS

We used a quasi-experimental design with a comparator group in the pre-post period, using a sample of 1969 women's medical records. The study was conducted in the "Wegera" and "Tach-armacheho" districts, which served as the intervention and comparator districts, respectively. The intervention comprised a multicomponent nonfinancial incentive, including smartphones, flash disks, power banks, certificates, and scholarships. Personal records of women who gave birth within 6 months before (April to September 2020) and after (February to July 2021) the intervention were included. Three distinct women's birth records were examined: the integrated card, integrated individual folder, and delivery register. The completeness of the data was determined by examining the presence of data elements, whereas the consistency check involved evaluating the agreement of data elements among women's birth records. The average treatment effect on the treated (ATET), with 95% CIs, was computed using a difference-in-differences model.

RESULTS

In the intervention district, data completeness in women's personal records was nearly 4 times higher (ATET 3.8, 95% CI 2.2-5.5; P=.02), and consistency was approximately 12 times more likely (ATET 11.6, 95% CI 4.18-19; P=.03) than in the comparator district.

CONCLUSIONS

This study indicates that performance-based nonfinancial incentives enhance data quality in the personal records of institutional births. Health care planners can adapt these incentives to improve the data quality of comparable medical records, particularly pregnancy-related data within health care facilities. Future research is needed to assess the effectiveness of nonfinancial incentives across diverse contexts to support successful scale-up.

Reduction of malaria case incidence following the introduction of clothianidin-based indoor residual spraying in previously unsprayed districts: an observational analysis using health facility register data from Côte d'Ivoire, 2018-2022

BACKGROUND

Indoor residual spraying (IRS) using neonicotinoid-based insecticides (clothianidin and combined clothianidin with deltamethrin) was deployed in two previously unsprayed districts of Côte d'Ivoire in 2020 and 2021 to complement standard pyrethroid insecticide-treated nets. This retrospective observational study uses health facility register data to assess the impact of IRS on clinically reported malaria case incidence.

METHODS

Health facility data were abstracted from consultation registers for the period September 2018 to April 2022 in two IRS districts and two control districts that did not receive IRS. Malaria cases reported by community health workers (CHWs) were obtained from district reports and District Health Information Systems 2. Facilities missing complete data were excluded. Controlled interrupted time series models were used to estimate the effect of IRS on monthly all-ages population-adjusted confirmed malaria cases and cases averted by IRS. Models controlled for transmission season, precipitation, vegetation, temperature, proportion of cases reported by CHWs, proportion of tested out of suspected cases and non-malaria outpatient visits.

RESULTS

An estimated 10 988 (95% CI 5694 to 18 188) malaria cases were averted in IRS districts the year following the 2020 IRS campaign, representing a 15.9% reduction compared with if IRS had not been deployed. Case incidence in IRS districts dropped by 27.7% (incidence rate ratio (IRR) 0.723, 95% CI 0.592 to 0.885) the month after the campaign. In the 8 months after the 2021 campaign, 14 170 (95% CI 13 133 to 15 025) estimated cases were averted, a 24.7% reduction, and incidence in IRS districts dropped by 37.9% (IRR 0.621, 95% CI 0.462 to 0.835) immediately after IRS. Case incidence in control districts did not change following IRS either year (p>0.05) and the difference in incidence level change between IRS and control districts was significant both years (p<0.05).

CONCLUSION

Deployment of clothianidin-based IRS was associated with a reduction in malaria case rates in two districts of Côte d'Ivoire following IRS deployment in 2020 and 2021.

Malaria trends in districts that were targeted and not-targeted for seasonal malaria chemoprevention in children under 5 years of age in Guinea, 2014-2021

BACKGROUND

Seasonal malaria chemoprevention (SMC) is a main intervention to prevent and reduce childhood malaria. Since 2015, Guinea has implemented SMC targeting children aged 3-59 months (CU5) in districts with high and seasonal malaria transmission.

OBJECTIVE

We assessed the programmatic impact of SMC in Guinea's context of scaled up malaria intervention programming by comparing malaria-related outcomes in 14 districts that had or had not been targeted for SMC.

METHODS

Using routine health management information system data, we compared the district-level monthly test positivity rate (TPR) and monthly uncomplicated and severe malaria incidence for the whole population and disaggregated age groups (<5 years and ≥5 years of age). Changes in malaria indicators through time were analysed by calculating the district-level compound annual growth rate (CAGR) from 2014 to 2021; we used statistical analyses to describe trends in tested clinical cases, TPR, uncomplicated malaria incidence and severe malaria incidence.

RESULTS

The CAGR of TPR of all age groups was statistically lower in SMC (median=-7.8%) compared with non-SMC (median=-3.0%) districts. Similarly, the CAGR in uncomplicated malaria incidence was significantly lower in SMC (median=1.8%) compared with non-SMC (median=11.5%) districts. For both TPR and uncomplicated malaria incidence, the observed difference was also significant when age disaggregated. The CAGR of severe malaria incidence showed that all age groups experienced a decline in severe malaria in both SMC and non-SMC districts. However, this decline was significantly higher in SMC (median=-22.3%) than in non-SMC (median=-5.1%) districts for the entire population, as well as both CU5 and people over 5 years of age.

CONCLUSION

Even in an operational programming context, adding SMC to the malaria intervention package yields a positive epidemiological impact and results in a greater reduction in TPR, as well as the incidence of uncomplicated and severe malaria in CU5.

Vision impairment and access to eye care in an integrated network of eye care system in Southern and Eastern India

PURPOSE

The World Report on Vision highlights that health information systems (HIS) should collect information on the determinants of eye conditions and vision impairment (VI) as a step toward integrated people-centered eye care (IPEC). Thus, this retrospective study aims to elucidate VI trends across age and gender of the patients who visited our centres in southern and eastern India.

METHODS

Electronic medical records of all new patients who visited the network were included. VI was defined as visual acuity less than 6/12; unilateral VI was based on the worse presenting eye (the other eye being normal), and bilateral VI was based on the better eye. "Total VI" includes both unilateral and bilateral VI.

RESULTS

The records of 7,31,307 patients from January to December 2019 were extracted. Males were 54%. The mean age was 40 years (SD: 19.16 years). The majority of patients (46.54%) visited a primary care centre. Centres in Andhra Pradesh saw the largest number of patients (46%). Approximately 65% of all patients presented had no VI in either eye. Total VI was 23.5% at the primary-centre level and approximately 45% at other levels, the highest being in Odisha. More females accessed care at primary and secondary centres (46.6%) than at higher levels of care (44%). Odisha had the lowest number of females accessing care (43.01%).

CONCLUSION

Non-visually impairing conditions form an important reason for patients visiting our eye care facilities. Strategies focusing on improving access to eye care for females should be planned at all levels, especially in Odisha.

Electronic aggregated data collection on cervical cancer screening in Bangladesh since 2014: what the data tells us?

INTRODUCTION

To reduce the high prevalence of cervical cancers among the Bangladeshi women, the Government of Bangladesh established a national cervical cancer screening programme in 2005 for women aged 30 to 60 years. The District Health Information System Version 2 (DHIS2) based electronic aggregated data collection system is used since the year 2013. This study summarises data from the year 2014 to 2022 to assess the effectiveness of the electronic data collection system in understanding the outcome of the screening programme.

METHODS

This is a descriptive study based on secondary data extracted in MS Excel from the DHIS2-based electronic repository of the national cervical cancer screening programme of Bangladesh. The respondents were women aged 30-60 years, screened for cervical cancer using VIA (Visual Inspection of cervix with Acetic acid) method in 465 government health facilities. The data were collected on the participants' residential location, month and year of screening, name and type of health facilities performing VIA, and VIA screening results.

RESULTS

The national screening programme reported a total 3.36 million VIA tests from 465 government hospitals in 8 years (2014 to 2022). The national average VIA-positivity rate was 3.6%, which varied from 1.4 to 9.5% among the districts. This national screening programme witnessed an exponential growth, year after year, with 83.3% increase in VIA test from 2014 to 2022. The primary and the secondary care hospitals were the highest collective contributors of VIA tests (86.2%) and positive cases (77.8%). The VIA-positivity rates in different hospital types varied widely, 7.0% in the medical university hospital, 5.7% in the medical college hospitals, 3.9% in the district/general hospitals, and 3.0% in the upazila health complexes.

CONCLUSIONS

A national cervical cancer screening programme using VIA method and a DHIS2-based electronic data collection backbone, is effective, sustainable, and useful to understand the screening coverage, VIA positivity rate and geographic distribution of the participants and case load to initiate policy recommendations and actions. Decentralization of the screening programme and more efforts at the primary and secondary care level is required to increase screening performances.

Application of the Expert Recommendations for Implementing Change (ERIC) compilation of strategies to health intervention implementation in low- and middle-income countries: a systematic review

BACKGROUND

The Expert Recommendations for Implementing Change (ERIC) project developed a compilation of implementation strategies that are intended to standardize reporting and evaluation. Little is known about the application of ERIC in low- and middle-income countries (LMICs). We systematically reviewed the literature on the use and specification of ERIC strategies for health intervention implementation in LMICs to identify gaps and inform future research.

METHODS

We searched peer-reviewed articles published through March 2023 in any language that (1) were conducted in an LMIC and (2) cited seminal ERIC articles or (3) mentioned ERIC in the title or abstract. Two co-authors independently screened all titles, abstracts, and full-text articles, then abstracted study, intervention, and implementation strategy characteristics of included studies.

RESULTS

The final sample included 60 studies describing research from all world regions, with over 30% published in the final year of our review period. Most studies took place in healthcare settings (n = 52, 86.7%), while 11 (18.2%) took place in community settings and four (6.7%) at the policy level. Across studies, 548 distinct implementation strategies were identified with a median of six strategies (range 1-46 strategies) included in each study. Most studies (n = 32, 53.3%) explicitly matched implementation strategies used for the ERIC compilation. Among those that did, 64 (87.3%) of the 73 ERIC strategies were represented. Many of the strategies not cited included those that target systems- or policy-level barriers. Nearly 85% of strategies included some component of strategy specification, though most only included specification of their action (75.2%), actor (57.3%), and action target (60.8%). A minority of studies employed randomized trials or high-quality quasi-experimental designs; only one study evaluated implementation strategy effectiveness.

CONCLUSIONS

While ERIC use in LMICs is rapidly growing, its application has not been consistent nor commonly used to test strategy effectiveness. Research in LMICs must better specify strategies and evaluate their impact on outcomes. Moreover, strategies that are tested need to be better specified, so they may be compared across contexts. Finally, strategies targeting policy-, systems-, and community-level determinants should be further explored.

TRIAL REGISTRATION

PROSPERO, CRD42021268374.

Development of novel composite data quality scores to evaluate facility-level data quality in electronic data in Kenya: a nationwide retrospective cohort study

BACKGROUND

In this evaluation, we aim to strengthen Routine Health Information Systems (RHIS) through the digitization of data quality assessment (DQA) processes. We leverage electronic data from the Kenya Health Information System (KHIS) which is based on the District Health Information System version 2 (DHIS2) to perform DQAs at scale. We provide a systematic guide to developing composite data quality scores and use these scores to assess data quality in Kenya.

METHODS

We evaluated 187 HIV care facilities with electronic medical records across Kenya. Using quarterly, longitudinal KHIS data from January 2011 to June 2018 (total N = 30 quarters), we extracted indicators encompassing general HIV services including services to prevent mother-to-child transmission (PMTCT). We assessed the accuracy (the extent to which data were correct and free of error) of these data using three data-driven composite scores: 1) completeness score; 2) consistency score; and 3) discrepancy score. Completeness refers to the presence of the appropriate amount of data. Consistency refers to uniformity of data across multiple indicators. Discrepancy (measured on a Z-scale) refers to the degree of alignment (or lack thereof) of data with rules that defined the possible valid values for the data.

RESULTS

A total of 5,610 unique facility-quarters were extracted from KHIS. The mean completeness score was 61.1% [standard deviation (SD) = 27%]. The mean consistency score was 80% (SD = 16.4%). The mean discrepancy score was 0.07 (SD = 0.22). A strong and positive correlation was identified between the consistency score and discrepancy score (correlation coefficient = 0.77), whereas the correlation of either score with the completeness score was low with a correlation coefficient of -0.12 (with consistency score) and -0.36 (with discrepancy score). General HIV indicators were more complete, but less consistent, and less plausible than PMTCT indicators.

CONCLUSION

We observed a lack of correlation between the completeness score and the other two scores. As such, for a holistic DQA, completeness assessment should be paired with the measurement of either consistency or discrepancy to reflect distinct dimensions of data quality. Given the complexity of the discrepancy score, we recommend the simpler consistency score, since they were highly correlated. Routine use of composite scores on KHIS data could enhance efficiencies in DQA at scale as digitization of health information expands and could be applied to other health sectors beyondHIV clinics.

Identifying barriers to the production and use of routine health information in Western Province, Zambia

Recent decades of improvements to routine health information systems in low- and middle-income countries (LMICs) have increased the volume of health data collected. However, countries continue to face several challenges with quality production and use of information for decision-making at sub-national levels, limiting the value of health information for policy, planning and research. Improving the quality of data production and information use is thus a priority in many LMICs to improve decision-making and health outcomes. This qualitative study identified the challenges of producing and using routine health information in Western Province, Zambia. We analysed the interview responses from 37 health and social sector professionals at the national, provincial, district and facility levels to understand the barriers to using data from the Zambian health management information system (HMIS). Respondents raised several challenges that we categorized into four themes: governance and health system organization, geographic barriers, technical and procedural barriers, and challenges with human resource capacity and staff training. Staff at the facility and district levels were arguably the most impacted by these barriers as they are responsible for much of the labour to collect and report routine data. However, facility and district staff had the least authority and ability to mitigate the barriers to data production and information use. Expectations for information use should therefore be clearly outlined for each level of the health system. Further research is needed to understand to what extent the available HMIS data address the needs and purposes of the staff at facilities and districts.

Use of routine health information systems data in developing and monitoring district and facility health plans: a scoping review

BACKGROUND

Routine Health Information Systems data should be used in a systematic and institutionalised manner to support the making of plans, the monitoring of plans and in supportive supervision. To explore to what extent there is discussion about the linkage between planning, monitoring and supervision of sub-national programs using routine data we conducted a scoping review. The review question was: How are routine health information systems used in developing and monitoring health plans at district and facility level?

METHODS

From a search of Ovid Medline (all), EMBASE and Web of Science along with a review of grey literature and involving a number of key stakeholders in identifying any missing resources a total of over 2200 documents were reviewed and data from 13 documents were extracted.

RESULTS

Overall, there are many descriptions of how to implement and strengthen systems, ways to assess and improve data availability and quality, tools to improve the data use context, training in data use and mechanisms to involve stakeholders and strengthen infrastructure. However, there are gaps in examples of routine health data being used in the development, monitoring and supervision of plans at district and facility level.

CONCLUSIONS

There appears to be no institutionalised obligation of planners to monitor plans, very little guidance on how to practically monitor programs and minimal discussion about how to use the routinely available data to supportively supervise the implementation of the plans. To overcome these shortcomings, we recommend that practical procedures to ensure linkage of existing district plans to regular monitoring of priority programs are institutionalised, that mechanisms for making managers institutionally accountable for monitoring and supervising these plans are put in place, and that practical guidelines for linking plans with routine health information system data and regular monitoring and supportive supervision are developed.

A roadmap for using DHIS2 data to track progress in key health indicators in the Global South: experience from sub-saharan Africa

High quality health data as collected by health management information systems (HMIS) is an important building block of national health systems. District Health Information System 2 (DHIS2) software is an innovation in data management and monitoring for strengthening HMIS that has been widely implemented in low and middle-income countries in the last decade. However, analysts and decision-makers still face significant challenges in fully utilizing the capabilities of DHIS2 data to pursue national and international health agendas. We aimed to (i) identify the most relevant health indicators captured by DHIS2 for tracking progress towards the Sustainable Development goals in sub-Saharan African countries and (ii) present a clear roadmap for improving DHIS2 data quality and consistency, with a special focus on immediately actionable solutions. We identified that key indicators in child and maternal health (e.g. vaccine coverage, maternal deaths) are currently being tracked in the DHIS2 of most countries, while other indicators (e.g. HIV/AIDS) would benefit from streamlining the number of indicators collected and standardizing case definitions. Common data issues included unreliable denominators for calculation of incidence, differences in reporting among health facilities, and programmatic differences in data quality. We proposed solutions for many common data pitfalls at the analysis level, including standardized data cleaning pipelines, k-means clustering to identify high performing health facilities in terms of data quality, and imputation methods. While we focus on immediately actionable solutions for DHIS2 analysts, improvements at the point of data collection are the most rigorous. By investing in improving data quality and monitoring, countries can leverage the current global attention on health data to strengthen HMIS and progress towards national and international health priorities.

Effects of primary healthcare reform on routine health information systems (RHISs): a mixed-methods study in Lesotho

BACKGROUND AND OBJECTIVE

The Ministry of Health of Lesotho and Partners In Health piloted the Lesotho National Primary Health Care Reform (LPHCR) from July 2014 to June 2017 to improve quality and quantity of service delivery and enhance health system management. This initiative included improvement of routine health information systems (RHISs) to map disease burden and reinforce data utilisation for clinical quality improvement.

METHODS AND ANALYSIS

The WHO Data Quality Assurance framework's core indicators were used to compare the completeness of health data before versus after the LPHCR in 60 health centres and 6 hospitals across four districts. To examine change in data completeness, we conducted an interrupted time series analysis using multivariable logistic mixed-effects regression. Additionally, we conducted 25 key informant interviews with healthcare workers (HCWs) at the different levels of Lesotho's health system, following a purposive sampling approach. Interviews were analysed using deductive coding based on the Performance of Routine Information System Management framework, which inspected organisational, technical and behavioural factors influencing RHIS processes and outputs associated with the LPHCR.

RESULTS

In multivariable analyses, trends in monthly data completion rate were higher after versus before the LPHCR for: documenting first antenatal care visit (adjusted OR (AOR): 1.24, 95% CI: 1.14 to 1.36) and institutional delivery (AOR: 1.19, 95% CI: 1.07 to 1.32). When discussing processes, HCWs highlighted the value of establishing clear roles and responsibilities in reporting under a new organisational structure, improved community programmes among district health management teams, and enhanced data sharing and monitoring by districts.

CONCLUSION

The Ministry of Health had a strong data completion rate pre-LPHCR, which was sustained throughout the LPHCR despite increased service utilisation. The data completion rate was optimised through improved behavioural, technical and organisational factors introduced as part of the LPHCR.

Scrutinizing human resources for health availability and distribution in Mozambique between 2016 and 2020: a subnational descriptive longitudinal study

INTRODUCTION

Overall, resilient health systems build upon sufficient, qualified, well-distributed, and motivated health workers; however, this precious resource is limited in numbers to meet people's demands, particularly in LMICs. Understanding the subnational distribution of health workers from different lens is critical to ensure quality healthcare and improving health outcomes.

METHODS

Using data from Health Personnel Information System, facility-level Service Availability and Readiness Assessment, and other sources, we performed a district-level longitudinal analysis to assess health workforce density and the ratio of male to female health workers between January 2016 and June 2020 across all districts in Mozambique.

RESULTS

22 011 health workers were sampled, of whom 10 405 (47.3%) were male. The average age was 35 years (SD: 9.4). Physicians (1025, 4.7%), maternal and child health nurses (4808, 21.8%), and nurses (6402, 29.1%) represented about 55% of the sample. In January 2016, the average district-level workforce density was 75.8 per 100 000 population (95% CI 65.9, 87.1), and was increasing at an annual rate of 8.0% (95% CI 6.00, 9.00) through January 2018. The annual growth rate declined to 3.0% (95% CI 2.00, 4.00) after January 2018. Two provinces, Maputo City and Maputo Province, with 268.3 (95% CI 186.10, 387.00) and 104.6 (95% CI 84.20, 130.00) health workers per 100 000 population, respectively, had the highest workforce density at baseline (2016). There were 3122 community health workers (CHW), of whom 72.8% were male, in January 2016. The average number of CHWs per 10 000 population was 1.33 (95% CI 1.11, 1.59) in 2016 and increased by 18% annually between January 2016 and January 2018. This trend reduced to 11% (95% CI 0.00, 13.00) after January 2018. The sex ratio was twice as high for all provinces in the central and northern regions relative to Maputo Province. Maputo City (OR: 0.34; 95% CI 0.32, 0.34) and Maputo Province (OR: 0.56; 95% CI 0.49, 0.65) reported the lowest sex ratio at the baseline. Encouragingly, important sex ratio improvements were observed after January 2018, particularly in the northern and central regions.

CONCLUSION

Mozambique made substantial progress in health workers' availability during the study period; however, with a critical slowdown after 2018. Despite the progress, meaningful shortages and distribution disparities persist.

Monitoring and evaluating the implementation of essential packages of health services

Essential packages of health services (EPHS) are a critical tool for achieving universal health coverage, especially in low-income and lower middle-income countries. However, there is a lack of guidance and standards for monitoring and evaluation (M&E) of EPHS implementation. This paper is the final in a series of papers reviewing experiences using evidence from the Disease Control Priorities, third edition publications in EPHS reforms in seven countries. We assess current approaches to EPHS M&E, including case studies of M&E approaches in Ethiopia and Pakistan. We propose a step-by-step process for developing a national EPHS M&E framework. Such a framework would start with a theory of change that links to the specific health system reforms the EPHS is trying to accomplish, including explicit statements about the ‘what’ and ‘for whom’ of M&E efforts. Monitoring frameworks need to consider the additional demands that could be placed on weak and already overstretched data systems, and they must ensure that processes are put in place to act quickly on emergent implementation challenges. Evaluation frameworks could learn from the field of implementation science; for example, by adapting the Reach, Effectiveness, Adoption, Implementation and Maintenance framework to policy implementation. While each country will need to develop its own locally relevant M&E indicators, we encourage all countries to include a set of core indicators that are aligned with the Sustainable Development Goal 3 targets and indicators. Our paper concludes with a call to reprioritise M&E more generally and to use the EPHS process as an opportunity for strengthening national health information systems. We call for an international learning network on EPHS M&E to generate new evidence and exchange best practices.

Effect of cost-reduction interventions on facility-based deliveries in Burkina Faso: a controlled interrupted time-series study with multiple non-equivalent dependent variables

BACKGROUND

Evaluating health intervention effectiveness in low-income countries involves many methodological challenges to be addressed. The objective of this study was to estimate the sustained effects of two interventions to improve financial access to facility-based deliveries.

METHODS

In an innovative controlled interrupted time-series study with primary data, we used four non-equivalent dependent variables (antenatal care) as control outcomes to estimate the effects of a national subsidy for deliveries (January 2007-December 2013) and a local 'free delivery' intervention (June 2007-December 2010) on facility-based deliveries. The statistical analysis used spline linear regressions with random intercepts and slopes.

RESULTS

The analysis involved 20 877 observations for the national subsidy and 8842 for the 'free delivery' intervention. The two interventions did not have immediate effects. However, both were associated with positive trend changes varying from 0.21 to 0.52 deliveries per month during the first 12 months and from 0.78 to 2.39 deliveries per month during the first 6 months. The absolute effects, evaluated 84 and 42 months after introduction, ranged from 2.64 (95% CI 0.51 to 4.77) to 10.78 (95% CI 8.52 to 13.03) and from 9.57 (95% CI 5.97 to 13.18) to 14.47 (95% CI 10.47 to 18.47) deliveries per month for the national subsidy and the 'free delivery' intervention, respectively, depending on the type of antenatal care used as a control outcome.

CONCLUSION

The results suggest that both interventions were associated with sustained non-linear increases in facility-based deliveries. The use of multiple control groups strengthens the credibility of the results, making them useful for policy makers seeking solutions for universal health coverage.

Availability of priority maternal and newborn health indicators: Cross-sectional analysis of pregnancy, childbirth and postnatal care registers from 21 countries

Data from national health information systems are essential for routinely tracking progress, programmatic decision-making and to improve quality of services. Understanding the data elements captured in patient registers which are building blocks of national HMIS indicators, enables us to standardize data collection and measurement of key indicators for tracking progress towards achieving maternal and newborn health goals. This analysis was done through a review of antenatal care (ANC), childbirth and postnatal care (PNC) registers from 21 countries across five geographic regions. Between July and October 2020, country-based maternal and newborn experts, implementing agencies, program managers, and ministry of health personnel were asked to share the registers in use. Both paper-based and electronic registers were obtained. Twenty ANC registers, eighteen childbirth and thirteen PNC were available and analyzed. Both longitudinal and cross-sectional ANC and PNC registers were obtained, while the childbirth registers included in the analysis were all cross-sectional. Fifty-five percent (11/20) ANC registers and 54% (7/13) PNC registers were longitudinal. In four countries, the registers were electronic, while the rest were paper-based (17 countries). Sub-analysis of registers from four countries (Ghana, Kenya, Nigeria, and Zambia) where the 2017/2018 and 2019/2020 registers were available showed that the latest versions included 21/27 (78%) of data elements that are critical in the computation of key maternal and newborn care indicators. This analysis highlights some areas in where there are data gaps in data on pregnancy and childbirth. Program managers and health workers should use data gathered routinely to monitor the performance of their national health system and to guide the continuous improvement of health care services for women and newborns. The findings can help to inform the standardization of pregnancy and childbirth registers, and provide information for other countries seeking to introduce indicators in their health systems.

Early effects of COVID-19 on maternal and child health service disruption in Mozambique

This article is part of the Research Topic 'Health Systems Recovery in the Context of COVID-19 and Protracted Conflict'.

Introduction

After the World Health Organization declared COVID-19 a pandemic, more than 184 million cases and 4 million deaths had been recorded worldwide by July 2021. These are likely to be underestimates and do not distinguish between direct and indirect deaths resulting from disruptions in health care services. The purpose of our research was to assess the early impact of COVID-19 in 2020 and early 2021 on maternal and child healthcare service delivery at the district level in Mozambique using routine health information system data, and estimate associated excess maternal and child deaths.

Methods

Using data from Mozambique's routine health information system (SISMA, Sistema de Informação em Saúde para Monitoria e Avaliação), we conducted a time-series analysis to assess changes in nine selected indicators representing the continuum of maternal and child health care service provision in 159 districts in Mozambique. The dataset was extracted as counts of services provided from January 2017 to March 2021. Descriptive statistics were used for district comparisons, and district-specific time-series plots were produced. We used absolute differences or ratios for comparisons between observed data and modeled predictions as a measure of the magnitude of loss in service provision. Mortality estimates were performed using the Lives Saved Tool (LiST).

Results

All maternal and child health care service indicators that we assessed demonstrated service delivery disruptions (below 10% of the expected counts), with the number of new users of family planing and malaria treatment with Coartem (number of children under five treated) experiencing the largest disruptions. Immediate losses were observed in April 2020 for all indicators, with the exception of treatment of malaria with Coartem. The number of excess deaths estimated in 2020 due to loss of health service delivery were 11,337 (12.8%) children under five, 5,705 (11.3%) neonates, and 387 (7.6%) mothers.

Conclusion

Findings from our study support existing research showing the negative impact of COVID-19 on maternal and child health services utilization in sub-Saharan Africa. This study offers subnational and granular estimates of service loss that can be useful for health system recovery planning. To our knowledge, it is the first study on the early impacts of COVID-19 on maternal and child health care service utilization conducted in an African Portuguese-speaking country.

Healthcare utilization and maternal and child mortality during the COVID-19 pandemic in 18 low- and middle-income countries: An interrupted time-series analysis with mathematical modeling of administrative data

BACKGROUND

The Coronavirus Disease 2019 (COVID-19) pandemic has had wide-reaching direct and indirect impacts on population health. In low- and middle-income countries, these impacts can halt progress toward reducing maternal and child mortality. This study estimates changes in health services utilization during the pandemic and the associated consequences for maternal, neonatal, and child mortality.

METHODS AND FINDINGS

Data on service utilization from January 2018 to June 2021 were extracted from health management information systems of 18 low- and lower-middle-income countries (Afghanistan, Bangladesh, Cameroon, Democratic Republic of the Congo (DRC), Ethiopia, Ghana, Guinea, Haiti, Kenya, Liberia, Madagascar, Malawi, Mali, Nigeria, Senegal, Sierra Leone, Somalia, and Uganda). An interrupted time-series design was used to estimate the percent change in the volumes of outpatient consultations and maternal and child health services delivered during the pandemic compared to projected volumes based on prepandemic trends. The Lives Saved Tool mathematical model was used to project the impact of the service utilization disruptions on child and maternal mortality. In addition, the estimated monthly disruptions were also correlated to the monthly number of COVID-19 deaths officially reported, time since the start of the pandemic, and relative severity of mobility restrictions. Across the 18 countries, we estimate an average decline in OPD volume of 13.1% and average declines of 2.6% to 4.6% for maternal and child services. We projected that decreases in essential health service utilization between March 2020 and June 2021 were associated with 113,962 excess deaths (110,686 children under 5, and 3,276 mothers), representing 3.6% and 1.5% increases in child and maternal mortality, respectively. This excess mortality is associated with the decline in utilization of the essential health services included in the analysis, but the utilization shortfalls vary substantially between countries, health services, and over time. The largest disruptions, associated with 27.5% of the excess deaths, occurred during the second quarter of 2020, regardless of whether countries reported the highest rate of COVID-19-related mortality during the same months. There is a significant relationship between the magnitude of service disruptions and the stringency of mobility restrictions. The study is limited by the extent to which administrative data, which varies in quality across countries, can accurately capture the changes in service coverage in the population.

CONCLUSIONS

Declines in healthcare utilization during the COVID-19 pandemic amplified the pandemic's harmful impacts on health outcomes and threaten to reverse gains in reducing maternal and child mortality. As efforts and resource allocation toward prevention and treatment of COVID-19 continue, essential health services must be maintained, particularly in low- and middle-income countries.

Assessing the Use of Hospital Information Systems (HIS) to Support Decision-Making: A Cross-Sectional Study in Public Hospitals in the Huíla Health Region of Southern Angola

Hospital information systems could be relevant tools to inform hospital managers, support better management decisions in healthcare, and increase efficiency. Nonetheless, hospital managers’ effective use of these systems to support decision-making in Angola is unknown. Our study aimed to analyse the use of hospital information systems as a tool to support decision-making by hospital managers in Huíla, Angola. It was a descriptive, cross-sectional study inducted between July and September 2017 in seven hospitals in Huíla Province, Angola, specifically in the cities of Lubango and Matala. Thirty-six members of the hospital boards filled out a self-questionnaire that consisted of twenty questions based on the following issues: Characterisation of the interviewee’s profile; availability of information in the institution; and quality and usefulness of the available operational information. At least two thirds of the participants reported being unsatisfied or relatively satisfied with each assessed hospital information systems-specific feature. More than 50% have rarely or never used the health information system to support decision-making. Most managers do not use hospital information systems to support management-related decision-making in Angola. Improving the ability of hospital information systems to compute adequate indicators and training for hospital managers could be targets for future interventions to support better management-related decision-making in Angolan healthcare.

Type of occupation and early antenatal care visit among women in sub-Saharan Africa

Background

Type of occupation has been linked to early antenatal care visits whereby women in different occupation categories tend to have different timing for antenatal care visits. Different occupations require varying levels of commitment, remuneration and energy requirements. This study, therefore, sought to investigate the association between the type of occupation and early antenatal care visits in sub-Saharan Africa.

Methods

This is a secondary analysis of Demographic and Health Survey data from 29 countries in sub-Saharan Africa conducted between 2010 and 2018. The study included 131,912 working women. We employed binary logistic regression models to assess the association between type of occupation and timely initiation of antenatal care visits.

Results

The overall prevalence of early initiation of antenatal care visits was 39.9%. Early antenatal care visit was high in Liberia (70.1%) but low in DR Congo (18.6%). We noted that compared to managerial workers, women in all other work categories had lower odds of early antenatal care visit and this was prominent among agricultural workers [aOR = 0.74, CI = 0.69, 0.79]. Women from Liberia [aOR = 3.14, CI = 2.84, 3.48] and Senegal [aOR = 2.55, CI = 2.31, 2.81] had higher tendency of early antenatal care visits compared with those from Angola.

Conclusion

The findings bring to bear some essential elements worth considering to enhance early antenatal care visits within sub-Saharan Africa irrespective of the type of occupation. Women in the agricultural industry need much attention in order to bridge the early antenatal care visit gap between them and workers of other sectors. A critical review of the maternal health service delivery in DR Congo is needed considering the low rate of early antenatal care visits.

Impact of seasonal malaria chemoprevention on prevalence of malaria infection in malaria indicator surveys in Burkina Faso and Nigeria

BACKGROUND

In 2012, the WHO issued a policy recommendation for the use of seasonal malaria chemoprevention (SMC) to children 3-59 months in areas of highly seasonal malaria transmission. Clinical trials have found SMC to prevent around 75% of clinical malaria. Impact under routine programmatic conditions has been assessed during research studies but there is a need to identify sustainable methods to monitor impact using routinely collected data.

METHODS

Data from Demographic Health Surveys were merged with rainfall, geographical and programme data in Burkina Faso (2010, 2014, 2017) and Nigeria (2010, 2015, 2018) to assess impact of SMC. We conducted mixed-effects logistic regression to predict presence of malaria infection in children aged 6-59 months (rapid diagnostic test (RDT) and microscopy, separately).

RESULTS

We found strong evidence that SMC administration decreases odds of malaria measured by RDT during SMC programmes, after controlling for seasonal factors, age, sex, net use and other variables (Burkina Faso OR 0.28, 95% CI 0.21 to 0.37, p<0.001; Nigeria OR 0.40, 95% CI 0.30 to 0.55, p<0.001). The odds of malaria were lower up to 2 months post-SMC in Burkina Faso (1-month post-SMC: OR 0.29, 95% CI 0.12 to 0.72, p=0.01; 2 months post-SMC: OR: 0.33, 95% CI 0.17 to 0.64, p<0.001). The odds of malaria were lower up to 1 month post-SMC in Nigeria but was not statistically significant (1-month post-SMC 0.49, 95% CI 0.23 to 1.05, p=0.07). A similar but weaker effect was seen for microscopy (Burkina Faso OR 0.38, 95% CI 0.29 to 0.52, p<0.001; Nigeria OR 0.53, 95% CI 0.38 to 0.76, p<0.001).

CONCLUSIONS

Impact of SMC can be detected in reduced prevalence of malaria from data collected through household surveys if conducted during SMC administration or within 2 months afterwards. Such evidence could contribute to broader evaluation of impact of SMC programmes.

A type III effectiveness-implementation hybrid evaluation of a multicomponent patient navigation strategy for advanced-stage Kaposi's sarcoma: protocol

Background

For people with advanced-stage Kaposi’s sarcoma (KS), a common HIV-associated malignancy in sub-Saharan Africa, mortality is estimated to be 45% within 2 years after KS diagnosis, despite increasingly wide-spread availability of antiretroviral therapy and chemotherapy. For advanced-stage KS, chemotherapy in addition to antiretroviral therapy improves outcomes and saves lives, but currently, only ~50% of people with KS in western Kenya who have an indication for chemotherapy actually receive it. This protocol describes the evaluation of a multicomponent patient navigation strategy that addresses common barriers to service penetration of and fidelity to evidence-based chemotherapy among people with advanced-stage KS in Kenya.

Methods

This is a hybrid type III effectiveness-implementation study using a non-randomized, pre- post-design nested within a longitudinal cohort. We will compare the delivery of evidence-based chemotherapy for advanced-stage KS during the period before (2016–2020) to the period after (2021–2024), the rollout of a multicomponent patient navigation strategy. The multicomponent patient navigation strategy was developed in a systematic process to address key determinants of service penetration of and fidelity to chemotherapy in western Kenya and includes (1) physical navigation and care coordination, (2) video-based education, (3) travel stipend, (4) health insurance enrollment assistance, (5) health insurance stipend, and (6) peer mentorship. We will compare the pre-navigation period to the post-navigation period to assess the impact of this multicomponent patient navigation strategy on (1) implementation outcomes: service penetration (chemotherapy initiation) and fidelity (chemotherapy completion) and (2) service and client outcomes: timeliness of cancer care, mortality, quality of life, stigma, and social support. We will also describe the implementation process and the determinants of implementation success for the multicomponent patient navigation strategy.

Discussion

This study addresses an urgent need for effective implementation strategies to improve the initiation and completion of evidence-based chemotherapy in advanced-stage KS. By using a clearly specified, theory-based implementation strategy and validated frameworks, this study will contribute to a more comprehensive understanding of how to improve cancer treatment in advanced-stage KS.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00281-7.

Continuity of community-based healthcare provision during COVID-19: a multicountry interrupted time series analysis

BACKGROUND

Pandemics often precipitate declines in essential health service utilisation, which can ultimately kill more people than the disease outbreak itself. There is some evidence, however, that the presence of adequately supported community health workers (CHWs), that is, financially remunerated, trained, supplied and supervised in line with WHO guidelines, may blunt the impact of health system shocks. Yet, adequate support for CHWs is often missing or uneven across countries. This study assesses whether adequately supported CHWs can maintain the continuity of essential community-based health service provision during the COVID-19 pandemic.

METHODS

Interrupted time series analysis. Monthly routine data from 27 districts across four countries in sub-Saharan Africa were extracted from CHW and facility reports for the period January 2018-June 2021. Descriptive analysis, null hypothesis testing, and segmented regression analysis were used to assess the presence and magnitude of a possible disruption in care utilisation after the earliest reported cases of COVID-19.

RESULTS

CHWs across all sites were supported in line with the WHO Guideline and received COVID-19 adapted protocols, training and personal protective equipment within 45 days after the first case in each country. We found no disruptions to the coverage of proactive household visits or integrated community case management (iCCM) assessments provided by these prepared and protected CHWs, as well as no disruptions to the speed with which iCCM was received, pregnancies were registered or postnatal care received.

CONCLUSION

CHWs who were equipped and prepared for the pandemic were able to maintain speed and coverage of community-delivered care during the pandemic period. Given that the majority of CHWs globally remain unpaid and largely unsupported, this paper suggests that the opportunity cost of not professionalising CHWs may be larger than previously estimated, particularly in light of the inevitability of future pandemics.

COVID-19-related healthcare impacts: an uncontrolled, segmented time-series analysis of tuberculosis diagnosis services in Mozambique, 2017-2020

Introduction

Currently, COVID-19 dominates the public health agenda and poses a permanent threat, leading to health systems’ exhaustion and unprecedented service disruption. Primary healthcare services, including tuberculosis services, are at increased risk of facing severe disruptions, particularly in low-income and middle-income countries. Indeed, corroborating model-based forecasts, there is increasing evidence of the COVID-19 pandemic’s negative impact on tuberculosis case detection.

Methods

Applying a segmented time-series analysis, we assessed the effects of COVID-19-related measures on tuberculosis diagnosis service across districts in Mozambique. Ministry health information system data were used from the first quarter of 2017 to the end of 2020. The model, performed under the Bayesian premises, was estimated as a negative binomial with random effects for districts and provinces.

Results

A total of 154 districts were followed for 16 consecutive quarters. Together, these districts reported 96 182 cases of all forms of tuberculosis in 2020. At baseline (first quarter of 2017), Mozambique had an estimated incidence rate of 283 (95% CI 200 to 406) tuberculosis cases per 100 000 people and this increased at a 5% annual rate through the end of 2019. We estimated that 17 147 new tuberculosis cases were potentially missed 9 months after COVID-19 onset, resulting in a 15.1% (95% CI 5.9 to 24.0) relative loss in 2020. The greatest impact was observed in the southern region at 40.0% (95% CI 30.1 to 49.0) and among men at 15% (95% CI 4.0 to 25.0). The incidence of pulmonary tuberculosis increased at an average rate of 6.6% annually; however, an abrupt drop (15%) was also observed immediately after COVID-19 onset in March 2020.

Conclusion

The most significant impact of the state of emergency was observed between April and June 2020, the quarter after COVID-19 onset. Encouragingly, by the end of 2020, clear signs of health system recovery were visible despite the initial shock.

The impact of introducing multidisciplinary care assessments on access to rheumatology care in British Columbia: an interrupted time series analysis

Background

In 2011 the British Columbia (BC) Ministry of Health introduced a new fee-for-service billing code that allowed “Multidisciplinary Care Assessment” (MCA). This change has the potential to change access to and quality of care for patients. This study aimed to explore the impact on access to rheumatology services in the province.

Methods

Fee-for-service rheumatology billings were evaluated for each rheumatologist 2 years before and after use of the MCA code. Numbers of 1) unique patients and 2) services provided per month were used as proxy measures of access to care. A multiple-baseline interrupted time series model assessed the impact of the MCA on levels and trends of the access outcomes.

Results

Our analysis consisted of 82,360 patients cared for by 26 rheumatologists who billed for an MCA. In our primary analysis we observed a sustained increase in the mean number of unique patients of 4.9% (95% CI: 0.0% to 9.9%, p = 0.049) and the mean number of services of 7.1% (95% CI: 1.0% to 13.6%, (p = 0.021), per month provided by a rheumatologist, corresponding to the initial use of MCA.

Conclusion

The introduction of the MCA code was associated with an initial increase in the measures of access, which was maintained but did not increase over time. Our study suggests that the use of Multidisciplinary Care Assessment can contribute to expanding and/or sustaining access to care for people with complex chronic conditions, like rheumatic diseases.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-07715-x.

Direct financial assistance for improved maternal and child health data: a pilot study supporting the health management information system in Malawi

Background

The health management information system (HMIS) is an integral component of a strong health care system. Despite its importance for decision-making, the quality of HMIS data remains of concern in low- and middle-income countries. To address challenges with the quality of maternal and child health (MCH) data gathered within Malawi’s HMIS, we conducted a pilot study evaluating different support modalities to district-level HMIS offices. We hypothesized that providing regular, direct financial assistance to HMIS offices would enable staff to establish strategies and priorities based on local context, resulting in more accurate, timely, and complete MCH data.

Methods

The pilot intervention was implemented in Mwanza district, while Chikwawa, Neno, and Ntchisi districts served as control sites given support received from other institutions. The intervention consisted of providing direct financial assistance to Mwanza’s HMIS office following the submission of detailed budgets and lists of planned activities. In the control districts, we performed interviews with the HMIS officers to track the HMIS-related activities. We evaluated the intervention by comparing data quality between the post- and pre-intervention periods in the intervention and control districts. Additionally, we conducted interviews with Mwanza’s HMIS office staff to determine the acceptability and appropriateness of the intervention.

Results

Following the 10-month intervention period, we observed improvements in MCH data quality in Mwanza. The availability and completeness of MCH data collected in the registers increased by 22 and 18 percentage points, respectively. The consistency of MCH data between summary reports and electronic HMIS also improved. In contrast, 2/3 control districts noted minimal changes or reductions in data quality after 10 months. The qualitative interviews confirmed that, despite some challenges, the intervention was well received by the participating HMIS office. HMIS staff preferred our strategy to other conventional strategies that fail to give them the independence to make decisions.

Conclusions

This pilot intervention demonstrated an alternative approach to support HMIS offices in their daily efforts to improve data quality. Given the Ministry of Health’s (MoH) interest in strengthening its HMIS, our intervention provides a strategy that the MoH and local and international partners could consider to rapidly improve HMIS data with minimal oversight.

Routine Health Information Utilization and Associated Factors among Health Professionals Working in Public Health Facilities of the South Region, Ethiopia

BACKGROUND

Routine health information is the pillar for planning and management of health services and plays a vital role in effective and efficient health service delivery, decision making, and the improvement of programs. Therefore, this study aimed to assess routine health information utilization and associated factors among health professionals working in public health facilities of the south region.

METHODS

Institution based cross-sectional study design was employed. Data was collected from randomly selected 719 participants using a pre-tested, interviewer administered structured questionnaire. Bivariate and multivariate logistic regression analyses were carried out.

RESULT

The overall utilization of routine health information was 63.1. Place of residence, HMIS personnel, HMIS code, overwhelming data source, population based data, data quality control, feedback, monitoring chart, 8.467) and data transfer policy were factors significantly associated with utilization of routine health information.

CONCLUSION

Six out of ten health professionals had utilized routine health information. Place of residence, HMIS personnel, HMIS code, overwhelming data source, population based data, data quality control, feedback, monitoring chart and data transfer policy had significant associations with routine health information utilization. Therefore, concerned health authorities need to work on these factors to improve the utilization.

Kenyan HIV Clinics With Youth-Friendly Services and Trained Providers Have a Higher Prevalence of Viral Suppression Among Adolescents and Young Adults: Results From an Observational Study

ABSTRACT

Sustained viral suppression in adolescents and young adults living with HIV (AYALWH) is necessary for epidemic control. We evaluated facility and individual correlates of viral suppression using programmatic data from AYALWH between ages 10 and 24 years at 24 HIV clinics in Kenya. Binomial regression was used to evaluate correlates of viral load (VL) suppression (<1,000 copies/ml). Of 5,316 AYALWH on antiretroviral therapy ≥6 months, 2,081 (39%) had VLs available in the medical record, of which 76% were virally suppressed. In multivariable analyses, antiretroviral therapy initiation among AYALWH older than 10 years was associated with higher viral suppression than initiation younger than 10 years (adjusted risk ratio [aRR] 10-14 = 1.03, 95% confidence interval [CI] 0.97-1.10; aRR 15-19 = 1.30, 95% CI 1.19-1.41; aRR 20-24 = 1.43, 95% CI 1.24-1.63). Facilities with both youth-friendly services (YFS) and trained providers had significantly higher VL suppression compared with facilities without YFS or trained providers (adjusted odds ratio: 2.07, 95% CI: 1.71-2.52). Viral suppression remains suboptimal among AYALWH. YFS and trained providers plus greater use of VL data may help increase viral suppression among AYALWH.

Can We Use Routine Data for Strategic Decision Making? A Time Trend Comparison Between Survey and Routine Data in Mali

Routine data, which is available more regularly than the "gold standard" survey data, can be used to inform programmatic decisions in Mali at the national level. However, caution must be used if using data at a subnational level.

Background:

Countries with scarce resources need timely and high-quality data on coverage of health interventions to make strategic decisions about where to allocate investments in health. Household survey data are generally regarded as “gold standard,” high-quality data. This study assessed the comparability of intervention coverage time trends from routine and survey data at national and subnational levels in Mali.

Methods:

We compared 3 coverage indicators: contraceptive prevalence rate, institutional delivery, and 3 doses of diphtheria, pertussis, and tetanus (DPT3) vaccine, using 3 Mali Demographic and Health Surveys (DHS 2001, 2006, and 2012–2013) and routine health system data covering 2001–2012. For routine data, we used local health information system (HIS) annual reports and an HIS database. To compare time trends between the data sources, we calculated the percentage point change and 95% confidence interval from 2001–2006 and 2006–2012. We then computed the absolute and relative differences between the 2 data sources for each indicator over time at national and regional levels and assessed their level of significance.

Results:

The direction and magnitude of the time trends of contraceptive prevalence rate, institutional delivery, and DPT3 vaccine from 2001 to 2012 were similar at the national level between data sources. At the regional level, there were significant differences in the magnitude and direction of time trends for institutional delivery and the DPT3 vaccine; contraceptive prevalence trends were more consistent. Routine data tended to overestimate DPT3 coverage, and underestimate institutional delivery and contraceptive prevalence relative to survey data.

Conclusion:

Routine data in Mali—particularly at the national level—appear to be appropriate for use to inform program planning and prioritization, but routine time trends should be interpreted with caution at the subnational level. For program evaluations, routine data may not be appropriate to draw accurate inferences about program impact.

Exploring data quality and use of the routine health information system in Ethiopia: a mixed-methods study

OBJECTIVE

A routine health information system (RHIS) enables decision making in the healthcare system. We aimed to analyse data quality at the district and regional level and explore factors and perceptions affecting the quality and use of routine data.

DESIGN

This was a mixed-methods study. We used the WHO toolkit for analysing data quality and interviewed staff at the point of data generation and along with the flow of data. Data were analysed using the Performance of Routine Information System Management framework.

SETTING

This study was performed in eight districts in four regions of Ethiopia. The study was nested within a 2-year programme of the Operational Research and Coaching for government Analysts.

PARTICIPANTS

We visited 45 health posts, 1 district hospital, 16 health centres and 8 district offices for analysis of routine RHIS data and interviewed 117 staff members for the qualitative assessment.

OUTCOME MEASURES

We assessed availability of source documents, completeness, timeliness and accuracy of reporting of routine data, and explored data quality and use perceptions.

RESULTS

There was variable quality of both indicator and data element. Data on maternal health and immunisation were of higher quality than data on child nutrition. Issues ranged from simple organisational factors, such as availability of register books, to intricate technical issues, like complexity of indicators and choice of denominators based on population estimates. Respondents showed knowledge of the reporting procedures, but also demonstrated limited skills, lack of supportive supervision and reporting to please the next level. We saw limited examples of the use of data by the staff who were responsible for data reporting.

CONCLUSION

We identified important organisational, technical, behavioural and process factors that need further attention to improve the quality and use of RHIS data in Ethiopia.

Integration of pre-exposure prophylaxis services into public HIV care clinics in Kenya: a pragmatic stepped-wedge randomised trial

BACKGROUND

Successful and sustainable models for HIV pre-exposure prophylaxis (PrEP) delivery in public health systems in Africa are needed. We aimed to evaluate the implementation of PrEP delivery integrated in public HIV care clinics in Kenya.

METHODS

As part of Kenya's national PrEP roll-out, we conducted a stepped-wedge cluster-randomised pragmatic trial to catalyse scale-up of PrEP delivery integrated in 25 public HIV care clinics. We selected high-volume clinics in these regions (ie, those with a high number of people living with HIV enrolled in HIV care and treatment). Clinics (each representing a cluster) were stratified by region and randomly assigned to the order in which clinic staff would receive PrEP training and ongoing technical support using numbered opaque balls picked from a bag. There was no masking. PrEP provision was done by clinic staff without additional financial support. Data were abstracted from records of individuals initiating PrEP. The primary outcome was the number of people initiating PrEP per clinic per month comparing intervention to control periods. Other outcomes included PrEP continuation, adherence, and incident HIV infections. This trial is registered with ClinicalTrials.gov, NCT03052010.

FINDINGS

After the baseline period, which started in January, 2017, every month two to six HIV care clinics crossed over from control to intervention, until August, 2017, when all clinics were implementing the intervention. Of 4898 individuals initiating PrEP (27 during the control period and 4871 during the intervention period), 2640 (54%) were women, the median age was 31 years (IQR 25-39), and 4092 (84%) reported having a partner living with HIV. The mean monthly number of PrEP initiations per clinic was 0·1 (SD 0·5) before the intervention and 7·5 (2·7) after intervention introduction (rate ratio 23·7, 95% CI 14·2-39·5, p<0·0001). PrEP continuation was 57% at 1 month, 44% at 3 months, and 34% at 6 months, and 12% of those who missed a refill returned later for PrEP re-initiation. Tenofovir diphosphate was detected in 68 (96%) of 71 blood samples collected from a randomly selected subset of participants. Six HIV infections were observed over 2531 person-years of observation (incidence 0·24 cases per 100 person-years), three of which occurred at the first visit after PrEP initiation.

INTERPRETATION

We observed high uptake, reasonable continuation with high adherence, frequent PrEP restarts, and low HIV incidence. Integration of PrEP services within public HIV care clinics in Africa is feasible.

FUNDING

National Institute of Mental Health and Bill & Melinda Gates Foundation.

Assessing the quality of care for children attending health facilities: a systematic review of assessment tools

INTRODUCTION

Assessing quality of healthcare is integral in determining progress towards equitable health outcomes worldwide. Using the WHO 'Standards for improving quality of care for children and young adolescents in health facilities' as a reference standard, we aimed to evaluate existing tools that assess quality of care for children.

METHODS

We undertook a systematic literature review of publications/reports between 2008 and 2020 that reported use of quality of care assessment tools for children (<15 years) in health facilities. Identified tools were reviewed against the 40 quality statements and 510 quality measures from the WHO Standards to determine the extent each tool was consistent with the WHO Standards. The protocol was registered in PROSPERO ID: CRD42020175652.

RESULTS

Nine assessment tools met inclusion criteria. Two hospital care tools developed by WHO-Europe and WHO-South-East Asia Offices had the most consistency with the WHO Standards, assessing 291 (57·1%) and 208 (40·8%) of the 510 quality measures, respectively. Remaining tools included between 33 (6·5%) and 206 (40·4%) of the 510 quality measures. The WHO-Europe tool was the only tool to assess all 40 quality statements. The most common quality measures absent were related to experience of care, particularly provision of educational, emotional and psychosocial support to children and families, and fulfilment of children's rights during care.

CONCLUSION

Quality of care assessment tools for children in health facilities are missing some key elements highlighted by the WHO Standards. The WHO Standards are, however, extensive and applying all the quality measures in every setting may not be feasible. A consensus of key indicators to monitor the WHO Standards is required. Existing tools could be modified to include priority indicators to strengthen progress reporting towards delivering quality health services for children. In doing so, a balance between comprehensiveness and practical utility is needed.

PROSPERO REGISTRATION NUMBER

CRD42020175652.

Improving data quality and information utilization at Metema Primary Hospital, Amhara national regional state, Northwest Ethiopia 2018: Capstone project

Quality data for evidence-based decision making become a growing concern globally. Available information needs to be disseminated on time and used for decision making. Therefore, an effective Health Management Information System is essential to make evidence-based decision. This study aimed to measure the change in data quality and information utilization before and after intervention. Facility-based pre-post interventional study design was conducted at Metema hospital from September/2016 to December30/2018. A total of 384 individual medical-records, HMIS registration-books and reports were reviewed. Training, supportive supervision and feedback were intervention packages. About 309 (80.5%) of charts were from outpatient department. Data recording completeness increased from 69.0% to 96.0%, data consistency increased from 84.0% to 99.5% and report timeliness enhanced from 66.0% to 100%. There was a statistically significant difference for data recording completeness between pre and post-intervention results with mean difference of -0.246 (-0.412, -0.081). Also, after the intervention, gap-filling feedback and supportive supervision were given to all departments. In addition, four quality improvement projects were developed at post-intervention phase. The level of data quality and use was improved after the intervention. So, designing and implementing intervention strategies based on the root causes will help to improve data quality and use.

Can routine health facility data be used to monitor subnational coverage of maternal, newborn and child health services in Uganda?

Background

Routine health facility data are a critical source of local monitoring of progress and performance at the subnational level. Uganda has been using district health statistics from facility data for many years. We aimed to systematically assess data quality and examine different methods to obtain plausible subnational estimates of coverage for maternal, newborn and child health interventions.

Methods

Annual data from the Uganda routine health facility information system 2015–2019 for all 135 districts were used, as well as national surveys for external comparison and the identification of near-universal coverage interventions. The quality of reported data on antenatal and delivery care and child immunization was assessed through completeness of facility reporting, presence of extreme outliers and internal data consistencies. Adjustments were made when necessary. The denominators for the coverage indicators were derived from population projections and health facility data on near-universal coverage interventions. The coverage results with different denominators were compared with the results from household surveys.

Results

Uganda’s completeness of reporting by facilities was near 100% and extreme outliers were rare. Inconsistencies in reported events, measured by annual fluctuations and between intervention consistency, were common and more among the 135 districts than the 15 subregions. The reported numbers of vaccinations were improbably high compared to the projected population of births or first antenatal visits – and especially so in 2015–2016. There were also inconsistencies between the population projections and the expected target population based on reported numbers of antenatal visits or immunizations. An alternative approach with denominators derived from facility data gave results that were more plausible and more consistent with survey results than based on population projections, although inconsistent results remained for substantive number of subregions and districts.

Conclusion

Our systematic assessment of the quality of routine reports of key events and denominators shows that computation of district health statistics is possible with transparent adjustments and methods, providing a general idea of levels and trends for most districts and subregions, but that improvements in data quality are essential to obtain more accurate monitoring.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06554-6.

Impact of a Multifaceted Intervention to Improve Emergency Care on Newborn and Child Health Outcomes in Rwanda

Implementing context-appropriate neonatal and pediatric advanced life support management interventions has increasingly been recommended as one of the approaches to reduce under-five mortality in resource-constrained settings like Rwanda. One such intervention is ETAT+, which stands for Emergency Triage, Assessment and Treatment plus Admission care for severely ill newborns and children. In 2013, ETAT+ was implemented in Rwandan district hospitals. We evaluated the impact of the ETAT+ intervention on newborn and child health outcomes. We used monthly time series data from the DHIS2-enabled Rwanda Health Management Information System from 2012 to 2016 to examine neonatal and pediatric hospital mortality rate. Each hospital contributed data for 12 and 36 months before and after ETAT+ implementation, respectively. Using controlled interrupted time series analysis and segmented regression model, we estimated longitudinal changes in neonatal and pediatric hospital mortality rate in intervention hospitals relative to matched concurrent control hospitals. We also studied changes in case fatality rate specifically for ETAT+ targeted conditions. Our study cohort consisted of seven intervention hospitals and fourteen matched control hospitals contributing 142,424 neonatal and pediatric hospital admissions. After controlling for secular trends and autocorrelation, we found that the ETAT+ implementation had no statistically significant impact on the rate of all-cause neonatal and pediatric hospital mortality in intervention hospitals relative to control hospitals. However, the case fatality rate for ETAT+ targeted neonatal conditions decreased immediately following implementation by 5% (95% CI: -9.25, -0.77) and over time by 0.8% monthly (95% CI: -1.36, -0.25), in intervention hospitals compared with control hospitals. Case fatality rate for ETAT+ targeted pediatric conditions did not decrease following the ETAT+ implementation. While ETAT+ focuses on improving quality of hospital care for both newborns and children, we only found an impact on neonatal hospital mortality for ETAT+ targeted conditions that should be interpreted with caution given the relatively short pre-intervention period and potential regression to the mean.

Optimizing treatment cascades for mental healthcare in Mozambique: preliminary effectiveness of the Systems Analysis and Improvement Approach for Mental Health (SAIA-MH)

Substantial investments are being made to scale-up access to mental healthcare in low- and middle-income countries, but less attention has been paid to quality and performance of nascent public-sector mental healthcare systems. This study tested the initial effectiveness of an implementation strategy to optimize routine outpatient mental healthcare cascade performance in Mozambique [the Systems Analysis and Improvement Approach for Mental Health (SAIA-MH)]. This study employed a pre-post design from September 2018 to August 2019 across four Ministry of Health clinics among 810 patients and 3234 outpatient mental health visits. Effectiveness outcomes evaluated progression through the care cascade, including: (1) initial diagnosis and medication selection; (2) enrolling in follow-up care; (3) returning after initial consultation within 60 days; (4) returning for follow-up visits on time; (5) returning for follow-up visits adherent to medication and (6) achieving function improvement. Clustered generalized linear models evaluated odds of completing cascade steps pre- vs post-intervention. Facilities prioritized improvements focused on the follow-up cascade, with 62.5% (10 of 16) monthly system modifications targeting medication adherence. At baseline, only 4.2% of patient visits achieved function improvement; during the 6 months of SAIA-MH implementation, this improved to 13.1% of patient visits. Multilevel logistic regression found increased odds of returning on time and adherent [aOR = 1.53, 95% CI (1.21, 1.94), P = 0.0004] and returning on time, adherent and with function improvement [aOR = 3.68, 95% CI (2.57, 5.44), P < 0.0001] after SAIA-MH implementation. No significant differences were observed regarding other cascade steps. The SAIA-MH implementation strategy shows promise for rapidly and significantly improving mental healthcare cascade outcomes, including the ultimate goal of patient function improvement. Given poor baseline mental healthcare cascade performance, there is an urgent need for evidence-based implementation strategies to optimize the performance of mental healthcare cascades in low- and middle-income countries.

A toolkit for costing environmental health services in healthcare facilities

Environmental health services (EHS) are critical for safe and functional healthcare facilities (HCFs). Understanding costs is important for improving and sustaining access to EHS in HCFs, yet the understanding of costs is poor and no tools exist to specifically support costing EHS in HCFs in low- and middle-income countries. We developed a toolkit to guide the following steps of costing EHS in HCFs: defining costing goals, developing and executing a data collection plan, calculating costs, and disseminating findings. The costing toolkit is divided into eight step-by-step modules with instructions, fillable worksheets, and guidance for effective data collection. It is designed for use by diverse stakeholders involved in funding, implementation, and management of EHS in HCFs and can be used by stakeholders with no prior costing experience. This paper describes the development, structure, and functionality of the toolkit; provides guidance for its application; and identifies good practices for costing, including pilot testing data collection tools and iterating the data collection process, involving diverse stakeholders, considering long-term costs, and disaggregating environmental costs in records to facilitate future costing. The toolkit itself is provided in the Supplementary Material.

Mixed-method estimation of population-level HIV viral suppression rate in the Western Cape, South Africa

INTRODUCTION

There are few population-wide data on viral suppression (VS) that can be used to monitor programmatic targets in sub-Saharan Africa. We describe how routinely collected viral load (VL) data from antiretroviral therapy (ART) programmes can be extrapolated to estimate population VS and validate this using a combination of empiric and model-based estimates.

METHODS

VL test results from were matched using a record linkage algorithm to obtain linked results for individuals. Test-level and individual-level VS rates were based on test VL values <1000 cps/mL, and individual VL <1000 cps/mL in a calendar year, respectively. We calculated population VS among people living with HIV (PLWH) in the province by combining census-derived midyear population estimates, HIV prevalence estimates and individual level VS estimates from routine VL data.

RESULTS

Approximately 1.9 million VL test results between 2008 and 2018 were analysed. Among individuals in care, VS increased from 85.5% in 2008 to 90% in 2018. Population VS among all PLWH in the province increased from 12.2% in 2008 to 51.0% in 2017. The estimates derived from this method are comparable to those from other published studies. Sensitivity analyses showed that the results are robust to variations in linkage method, but sensitive to the extreme combinations of assumed VL testing coverage and population HIV prevalence.

CONCLUSION

While validation of this method in other settings is required, this approach provides a simple, robust method for estimating population VS using routine data from ART services that can be employed by national programmes in high-burden settings.

Key informant perspectives on the challenges and opportunities for using routine health data for decision-making in Senegal

Background

Increasing the performance of routine health information systems (RHIS) is an important policy priority both globally and in Senegal. As RHIS data become increasingly important in driving decision-making in Senegal, it is imperative to understand the factors that determine their use.

Methods

Semi-structured interviews were conducted with 18 high- and mid-level key informants active in the malaria, tuberculosis and HIV programmatic areas in Senegal. Key informants were employed in the relevant divisions of the Senegal Ministry of Health or nongovernmental / civil society organizations. We asked respondents questions related to the flow, quality and use of RHIS data in their organizations. A framework approach was used to analyze the qualitative data.

Results

Although the respondents worked at the strategic levels of their respective organizations, they consistently indicated that data quality and data use issues began at the operational level of the health system before the data made its way to the central level. We classify the main identified barriers and facilitators to the use of routine data into six categories and attempt to describe their interrelated nature. We find that data quality is a central and direct determinant of RHIS data use. We report that a number of upstream factors in the Senegal context interact to influence the quality of routine data produced. We identify the sociopolitical, financial and system design determinants of RHIS data collection, dissemination and use. We also discuss the organizational and infrastructural factors that influence the use of RHIS data.

Conclusions

We recommend specific prescriptive actions with potential to improve RHIS performance in Senegal, the quality of the data produced and their use. These actions include addressing sociopolitical factors that often interrupt RHIS functioning in Senegal, supporting and motivating staff that maintain RHIS data systems as well as ensuring RHIS data completeness and representativeness. We argue for improved coordination between the various stakeholders in order to streamline RHIS data processes and improve transparency. Finally, we recommend the promotion of a sustained culture of data quality assessment and use.

Syndromic surveillance using monthly aggregate health systems information data: methods with application to COVID-19 in Liberia

BACKGROUND

Early detection of SARS-CoV-2 circulation is imperative to inform local public health response. However, it has been hindered by limited access to SARS-CoV-2 diagnostic tests and testing infrastructure. In regions with limited testing capacity, routinely collected health data might be leveraged to identify geographical locales experiencing higher than expected rates of COVID-19-associated symptoms for more specific testing activities.

METHODS

We developed syndromic surveillance tools to analyse aggregated health facility data on COVID-19-related indicators in seven low- and middle-income countries (LMICs), including Liberia. We used time series models to estimate the expected monthly counts and 95% prediction intervals based on 4 years of previous data. Here, we detail and provide resources for our data preparation procedures, modelling approach and data visualisation tools with application to Liberia.

RESULTS

To demonstrate the utility of these methods, we present syndromic surveillance results for acute respiratory infections (ARI) at health facilities in Liberia during the initial months of the COVID-19 pandemic (January through August 2020). For each month, we estimated the deviation between the expected and observed number of ARI cases for 325 health facilities and 15 counties to identify potential areas of SARS-CoV-2 circulation.

CONCLUSIONS

Syndromic surveillance can be used to monitor health facility catchment areas for spikes in specific symptoms which may indicate SARS-CoV-2 circulation. The developed methods coupled with the existing infrastructure for routine health data systems can be leveraged to monitor a variety of indicators and other infectious diseases with epidemic potential.

Routine Health Information Systems in the European Context: A Systematic Review of Systematic Reviews

(1) Background: The aim of this study is to provide a better understanding of the requirements to improve routine health information systems (RHISs) for the management of health systems, including the identification of best practices, opportunities, and challenges in the 53 countries and territories of the WHO European region. (2) Methods: We conducted an overview of systematics reviews and searched the literature in the databases MEDLINE/PubMed, Cochrane, EMBASE, and Web of Science electronic databases. After a meticulous screening, we identified 20 that met the inclusion criteria, and RHIS evaluation results were presented according to the Performance of Routine Information System Management (PRISM) framework. (3) Results: The reviews were published between 2007 and 2020, focusing on the use of different systems or technologies and aimed to analyze interventions on professionals, centers, or patients' outcomes. All reviews examined showed variability in results in accordance with the variability of interventions and target populations. We have found different areas for improvement for RHISs according to the three determinants of the PRISM framework that influence the configuration of RHISs: technical, organizational, or behavioral elements. (4) Conclusions: RHIS interventions in the European region are promising. However, new global and international strategies and the development of tools and mechanisms should be promoted to highly integrate platforms among European countries.

The impact of the COVID-19 pandemic on health services utilization in China: Time-series analyses for 2016-2020

BACKGROUND

The aim of this study is to quantify the effects of the SARS-CoV-2 pandemic on health services utilization in China using over four years of routine health information system data.

METHODS

We conducted a retrospective observational cohort study of health services utilization from health facilities at all levels in all provinces of mainland China. We analyzed monthly all-cause health facility visits and inpatient volume in health facilities before and during the SARS-CoV-2 outbreak using nationwide routine health information system data from January 2016 to June 2020. We used interrupted time series analyses and segmented negative binomial regression to examine changes in healthcare utilization attributable to the pandemic. Stratified analyses by facility type and by provincial Human Development Index (HDI) - an area-level measure of socioeconomic status - were conducted to assess potential heterogeneity in effects.

FINDINGS

In the months before the SARS-CoV-2 outbreak, a positive secular trend in patterns of healthcare utilization was observed. After the SARS-CoV-2 outbreak, we noted statistically significant decreases in all indicators, with all indicators achieving their nadir in February 2020. The magnitude of decline in February ranged from 63% (95% CI 61-65%; p<0•0001) in all-cause visits at hospitals in regions with high HDI and 71% (95% CI 70-72%; p<0•0001) in all-cause visits at primary care clinics to 33% (95% CI 24-42%; p<0•0001) in inpatient volume and 10% (95% CI 3-17%; p = 0•0076) in all-cause visits at township health centers (THC) in regions with low HDI. The reduction in health facility visits was greater than that in the number of outpatients discharged (51% versus 48%; p<0•0079). The reductions in both health facility visits and inpatient volume were greater in hospitals than in primary health care facilities (p<0•0001) and greater in developed regions than in underdeveloped regions (p<0•0001). Following the nadir of health services utilization in February 2020, all indicators showed statistically significant increases. However, even by June 2020, nearly all indicators except outpatient and inpatient volume in regions with low HDI and inpatient volume in private hospitals had not achieved their pre-SARS-COV-2 forecasted levels. In total, we estimated cumulative losses of 1020.5 (95% CI 951.2- 1089.4; P<0.0001) million or 23.9% (95% CI 22.5-25.2%; P<0.0001) health facility visits, and 28.9 (95% CI 26.1-31.6; P<0.0001) million or 21.6% (95% CI 19.7-23.4%; P<0.0001) inpatients as of June 2020.

INTERPRETATION

Inpatient and outpatient health services utilization in China declined significantly after the SARS-CoV-2 outbreak, likely due to changes in patient and provider behaviors, suspension of health facilities or their non-emergency services, massive mobility restrictions, and the potential reduction in the risk of non-SARS-COV-2 diseases. All indicators rebounded beginning in March but most had not recovered to their pre-SARS-COV-2 levels as of June 2020.

FUNDING

The National Natural Science Foundation of China (No. 72042014).

A network analysis of patient referrals in two district health systems in Tanzania

Patient referral systems are fragile and overlooked components of the health system in Tanzania. Our study aims at exploring patient referral networks in two rural districts in Tanzania, Kilolo and Msalala. Firstly, we ask whether secondary-level facilities act as gatekeepers, mediating referrals from primary- to tertiary-level facilities. Secondly, we explore the facility and network-level determinants of patient referrals focusing on treatment of childhood illnesses and non-communicable diseases. We use data collected across all public health facilities in the districts in 2018. To study gatekeeping, we employ descriptive network analysis tools. To explore the determinants of referrals, we use exponential random graph models. In Kilolo, we find a disproportionate share of patients referred directly to the largest hospital due to geographical proximity. In Msalala, small and specialized secondary-level facilities seem to attract more patients. Overall, the results call for policies to increase referrals to secondary facilities avoiding expensive referrals to hospitals, improving timeliness of care and reducing travel-related financial burden for households.

Toward Universal HIV Treatment in Haiti: Time Trends in ART Retention After Expanded ART Eligibility in a National Cohort From 2011 to 2017

BACKGROUND

The World Health Organization (WHO) recommends universal antiretroviral therapy (ART) for persons living with HIV (PLWH), but evidence about effects of expanded ART access on ART retention in low-resource settings is limited.

SETTING

Haiti's Ministry of Health endorsed universal ART for pregnant women in March 2013 (Option B+) and for all PLWH in July 2016. This study included 51,579 ART patients from 2011 to 2017 at 94 hospitals and clinics in Haiti.

METHODS

This observational, retrospective cohort study described time trends in 6-month ART retention using secondary data, and compared results during 3 periods using an interrupted time series model: pre-Option B+ (period 1: 1/11-2/13), Option B+ (period 2: 3/13-6/16), and Test and Start (T&S, period 3: 7/16-9/17).

RESULTS

From the pre-Option B+ to the T&S period, the monthly count of new ART patients increased from 366/month to 877/month, and the proportion with same-day ART increased from 6.3% to 42.1% (P < 0.001). The proportion retained on ART after 6 months declined from 78.4% to 75.0% (P < 0.001). In the interrupted time series model, ART retention improved by a rate of 1.4% per quarter during the T&S period after adjusting for patient characteristics (adjusted incidence rate ratio = 1.014; 95% confidence interval: 1.002 to 1.026, P < 0.001). However, patients with same-day ART were 14% less likely to be retained compared to those starting ART >30 days after HIV diagnosis (adjusted incidence rate ratio = 0.86; 95% confidence interval: 0.84-0.89, P < 0.001).

CONCLUSIONS

Achieving targets for HIV epidemic control will require increasing ART retention and reducing the disparity in retention for those with same-day ART.

Routine health management information system data in Ethiopia: consistency, trends, and challenges

Background: Ethiopia is investing in the routine Health Management Information System. Improved routine data are needed for decision-making in the health sector. Objective: To analyse the quality of the routine Health Management Information System data and triangulate with other sources, such as the Demographic and Health Surveys. Methods: We analysed national Health Management Information System data on 19 indicators of maternal health, neonatal survival, immunization, child nutrition, malaria, and tuberculosis over the 2012-2018 time period. The analyses were conducted by 38 analysts from the Ministry of Health, Ethiopia, and two government agencies who participated in the Operational Research and Coaching for Analysts (ORCA) project between June 2018 and June 2020. Using a World Health Organization Data Quality Review toolkit, we assessed indicator definitions, completeness, internal consistency over time and between related indicators, and external consistency compared with other data sources. Results: Several services reported coverage of above 100%. For many indicators, denominators were based on poor-quality population data estimates. Data on individual vaccinations had relatively good internal consistency. In contrast, there was low external consistency for data on fully vaccinated children, with the routine Health Management Information System showing 89% coverage but the Demographic and Health Survey estimate at 39%. Maternal health indicators displayed increasing coverage over time. Indicators on child nutrition, malaria, and tuberculosis were less consistent. Data on neonatal mortality were incomplete and operationalised as mortality on day 0-6. Our comparisons with survey and population projections indicated that one in eight early neonatal deaths were reported in the routine Health Management Information System. Data quality varied between regions. Conclusions: The quality of routine data gathered in the health system needs further attention. We suggest regular triangulation with data from other sources. We recommend addressing the denominator issues, reducing the complexity of indicators, and aligning indicators to international definitions.

'Do you trust those data?'-a mixed-methods study assessing the quality of data reported by community health workers in Kenya and Malawi

High-quality data are essential to monitor and evaluate community health worker (CHW) programmes in low- and middle-income countries striving towards universal health coverage. This mixed-methods study was conducted in two purposively selected districts in Kenya (where volunteers collect data) and two in Malawi (where health surveillance assistants are a paid cadre). We calculated data verification ratios to quantify reporting consistency for selected health indicators over 3 months across 339 registers and 72 summary reports. These indicators are related to antenatal care, skilled delivery, immunization, growth monitoring and nutrition in Kenya; new cases, danger signs, drug stock-outs and under-five mortality in Malawi. We used qualitative methods to explore perceptions of data quality with 52 CHWs in Kenya, 83 CHWs in Malawi and 36 key informants. We analysed these data using a framework approach assisted by NVivo11. We found that only 15% of data were reported consistently between CHWs and their supervisors in both contexts. We found remarkable similarities in our qualitative data in Kenya and Malawi. Barriers to data quality mirrored those previously reported elsewhere including unavailability of data collection and reporting tools; inadequate training and supervision; lack of quality control mechanisms; and inadequate register completion. In addition, we found that CHWs experienced tensions at the interface between the formal health system and the communities they served, mediated by the social and cultural expectations of their role. These issues affected data quality in both contexts with reports of difficulties in negotiating gender norms leading to skipping sensitive questions when completing registers; fabrication of data; lack of trust in the data; and limited use of data for decision-making. While routine systems need strengthening, these more nuanced issues also need addressing. This is backed up by our finding of the high value placed on supportive supervision as an enabler of data quality.

When it rains, it pours: detecting seasonal patterns in utilization of maternal healthcare in Mozambique using routine data

BACKGROUND

Climatic conditions and seasonal trends can affect population health, but typically, we consider the effect of climate on the epidemiology of communicable diseases. However, climate can also have an effect on access to care, particularly in remote rural areas of low- and middle-income countries. In this study, we investigate associations between the rainy season and the utilization of maternal health services in Mozambique.

METHODS

We examined patterns in the number of women receiving antenatal care (ANC) and delivering at a health facility for 2012-2019, using data from Mozambique's Health Management Information Systems. We investigated the association between seasonality (rainfall) and maternal health service utilization (ANC and institutional delivery) at national and provincial level. We fit a negative binomial regression model for institutional delivery and used it to estimate the yearly reduction in institutional deliveries due to the rainy season, with other factors held constant. We used the Lives Saved Tool (LiST) to model increases in mortality due to this estimated decrease in institutional delivery associated with the rainy season.

RESULTS

In our national analysis, the rate of ANC visits was 1% lower during the rainy season, adjusting for year and province (IRR = 0.99, 95% CI: 0.96-1.03). The rate of institutional deliveries was 6% lower during the rainy season than the dry season, after adjusting for time and province (IRR = 0.94, 95% CI: 0.92-0.96). In provincial analyses, all provinces except for Maputo-Cidade, Maputo-Province, Nampula, and Niassa showed a statistically significantly lower rate of institutional deliveries in the rainy season. None were statistically significantly lower for ANC. We estimate that, due to reductions in institutional delivery attributable only to the rainy season, there were 74 additional maternal deaths and 726 additional deaths of children under the age of 1 month in 2021, that would not have died if the mothers had instead delivered at a facility.

CONCLUSION

Fewer women deliver at a health facility during the rainy season in Mozambique than during the dry season. Barriers to receiving care during pregnancy and childbirth must be addressed using a multisectoral approach, considering the impact of geographical inequities.

Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review

BACKGROUND

When randomisation is not possible, interrupted time series (ITS) design has increasingly been advocated as a more robust design to evaluating health system quality improvement (QI) interventions given its ability to control for common biases in healthcare QI. However, there is a potential risk of producing misleading results when this rather robust design is not used appropriately. We performed a methodological systematic review of the literature to investigate the extent to which the use of ITS has followed best practice standards and recommendations in the evaluation of QI interventions.

METHODS

We searched multiple databases from inception to June 2018 to identify QI intervention studies that were evaluated using ITS. There was no restriction on date, language and participants. Data were synthesised narratively using appropriate descriptive statistics. The risk of bias for ITS studies was assessed using the Cochrane Effective Practice and Organisation of Care standard criteria. The systematic review protocol was registered in PROSPERO (registration number: CRD42018094427).

RESULTS

Of 4061 potential studies and 2028 unique records screened for inclusion, 120 eligible studies assessed eight QI strategies and were from 25 countries. Most studies were published since 2010 (86.7%), reported data using monthly interval (71.4%), used ITS without a control (81%) and modelled data using segmented regression (62.5%). Autocorrelation was considered in 55% of studies, seasonality in 20.8% and non-stationarity in 8.3%. Only 49.2% of studies specified the ITS impact model. The risk of bias was high or very high in 72.5% of included studies and did not change significantly over time.

CONCLUSIONS

The use of ITS in the evaluation of health system QI interventions has increased considerably over the past decade. However, variations in methodological considerations and reporting of ITS in QI remain a concern, warranting a need to develop and reinforce formal reporting guidelines to improve its application in the evaluation of health system QI interventions.

Bridging research integrity and global health epidemiology (BRIDGE) statement: guidelines for good epidemiological practice

BACKGROUND

Research integrity and research fairness have gained considerable momentum in the past decade and have direct implications for global health epidemiology. Research integrity and research fairness principles should be equally nurtured to produce high-quality impactful research-but bridging the two can lead to practical and ethical dilemmas. In order to provide practical guidance to researchers and epidemiologist, we set out to develop good epidemiological practice guidelines specifically for global health epidemiology, targeted at stakeholders involved in the commissioning, conduct, appraisal and publication of global health research.

METHODS

We developed preliminary guidelines based on targeted online searches on existing best practices for epidemiological studies and sought to align these with key elements of global health research and research fairness. We validated these guidelines through a Delphi consultation study, to reach a consensus among a wide representation of stakeholders.

RESULTS

A total of 45 experts provided input on the first round of e-Delphi consultation and 40 in the second. Respondents covered a range of organisations (including for example academia, ministries, NGOs, research funders, technical agencies) involved in epidemiological studies from countries around the world (Europe: 19; Africa: 10; North America: 7; Asia: 5; South-America: 3 Australia: 1). A selection of eight experts were invited for a face-to-face meeting. The final guidelines consist of a set of 6 standards and 42 accompanying criteria including study preparation, protocol development, data collection, data management, data analysis, dissemination and communication.

CONCLUSION

While guidelines will not by themselves guard global health from questionable and unfair research practices, they are certainly part of a concerted effort to ensure not only mutual accountability between individual researchers, their institutions and their funders but most importantly their joint accountability towards the communities they study and society at large.