Data-driven quality improvement in low-and middle-income country health systems: lessons from seven years of implementation experience across Mozambique, Rwanda, and Zambia

Travelling numbers and broken loops: A qualitative systematic review on collecting and reporting maternal and neonatal health data in low-and lower-middle income countries

Data and indicator estimates are considered vital to document persisting challenges in maternal and newborn health and track progress towards global goals. However, prioritization of standardised, comparable quantitative data can preclude the collection of locally relevant information and pose overwhelming burdens in low-resource settings, with negative effects on the provision of quality of care. A growing body of qualitative studies aims to provide a place-based understanding of the complex processes and human experiences behind the generation and use of maternal and neonatal health data. We conducted a qualitative systematic review exploring how national or international requirements to collect and report data on maternal and neonatal health indicators are perceived and experienced at the sub-national and country level in low-income and lower-middle income countries. We systematically searched six electronic databases for qualitative and mixed-methods studies published between January 2000 and March 2023. Following screening of 4084 records by four reviewers, 47 publications were included in the review. Data were analysed thematically and synthesised from a Complex Adaptive Systems (CAS) theoretical perspective. Our findings show maternal and neonatal health data and indicators are not fixed, neutral entities, but rather outcomes of complex processes. Their collection and uptake is influenced by a multitude of system hardware elements (human resources, relevancy and adequacy of tools, infrastructure, and interoperability) and software elements (incentive systems, supervision and feedback, power and social relations, and accountability). When these components are aligned and sufficiently supportive, data and indicators can be used for positive system adaptivity through performance evaluation, prioritization, learning, and advocacy. Yet shortcomings and broken loops between system components can lead to unforeseen emergent behaviors such as blame, fear, and data manipulation. This review highlights the importance of measurement approaches that prioritize local relevance and feasibility, necessitating participatory approaches to define context-specific measurement objectives and strategies.

Perceived progress toward scale of 14 maternal, newborn, child health, and nutrition (MNCHN) assets at subnational level in Burkina Faso, Ethiopia, India, Kenya, and Nigeria

Inequitable coverage of evidence-based MNCHN interventions is particularly pronounced in low and middle income countries where access and delivery of these interventions can vary dramatically at the subnational level. We conducted health system assessments in nine subnational geographies in five countries (Burkina Faso, Ethiopia, India, Kenya and Nigeria) to explore progress toward scale of 14 evidence-based MNCHN interventions (iron-folic acid, oxytocin, magnesium sulfate, misoprostol; 7.1% chlorhexidine for umbilical cord care, neonatal resuscitation, kangaroo mother care, community regimen for the treatment of possible severe bacterial infection; amoxicillin dispersible tablets, multiple micronutrient supplements, balanced energy protein supplementation, early and exclusive breastfeeding, feeding of small and sick newborns, and management of severe and moderate acute malnutrition in children less than five years old). Between March and October 2021, we conducted key informant interviews with a purposive sample of 275 healthcare providers and 94 district health management (DHMT) staff to better understand bottlenecks, facilitators and uptake of the interventions across varied subnational settings. Across all interventions and geographies, providers and DHMT staff perceived lack of robust HMIS data as the most significant barrier to scale followed by weak facility infrastructure. DHMT staff viewed limited budget allocation and training as a much larger barrier than healthcare providers, most likely given their purview as subnational managers. Healthcare providers were focused on supply chain and staffing, which affect workflows and service provision. Understanding provider and health facility management views of why interventions do or do not advance towards effective coverage can assist in creating enabling environments for the scale of best practices. These types of data are most helpful when collected at the subnational level, which allows for comparisons both within and between countries to show health disparities. Importantly, this strategic data collection can provide a starting point for improvement efforts to address existing health system gaps.

Quality improvement in public-private partnerships in low- and middle-income countries: a systematic review

BACKGROUND

Public-private partnerships (PPP) are often how health improvement programs are implemented in low-and-middle-income countries (LMICs). We therefore aimed to systematically review the literature about the aim and impacts of quality improvement (QI) approaches in PPP in LMICs.

METHODS

We searched SCOPUS and grey literature for studies published before March 2022. One reviewer screened abstracts and full-text studies for inclusion. The study characteristics, setting, design, outcomes, and lessons learned were abstracted using a standard tool and reviewed in detail by a second author.

RESULTS

We identified 9,457 citations, of which 144 met the inclusion criteria and underwent full-text abstraction. We identified five key themes for successful QI projects in LMICs: 1) leadership support and alignment with overarching priorities, 2) local ownership and engagement of frontline teams, 3) shared authentic learning across teams, 4) resilience in managing external challenges, and 5) robust data and data visualization to track progress. We found great heterogeneity in QI tools, study designs, participants, and outcome measures. Most studies had diffuse aims and poor descriptions of the intervention components and their follow-up. Few papers formally reported on actual deployment of private-sector capital, and either provided insufficient information or did not follow the formal PPP model, which involves capital investment for a explicit return on investment. Few studies discussed the response to their findings and the organizational willingness to change.

CONCLUSIONS

Many of the same factors that impact the success of QI in healthcare in high-income countries are relevant for PPP in LMICs. Vague descriptions of the structure and financial arrangements of the PPPs, and the roles of public and private entities made it difficult to draw meaningful conclusions about the impacts of the organizational governance on the outcomes of QI programs in LMICs. While we found many articles in the published literature on PPP-funded QI partnerships in LMICs, there is a dire need for research that more clearly describes the intervention details, implementation challenges, contextual factors, leadership and organizational structures. These details are needed to better align incentives to support the kinds of collaboration needed for guiding accountability in advancing global health. More ownership and power needs to be shifted to local leaders and researchers to improve research equity and sustainability.

Quality indicators for evaluating cancer care in low-income and middle-income country settings: a multinational modified Delphi study

This Policy Review sourced opinions from experts in cancer care across low-income and middle-income countries (LMICs) to build consensus around high-priority measures of care quality. A comprehensive list of quality indicators in medical, radiation, and surgical oncology was identified from systematic literature reviews. A modified Delphi study consisting of three 90-min workshops and two international electronic surveys integrating a global range of key clinical, policy, and research leaders was used to derive consensus on cancer quality indicators that would be both feasible to collect and were high priority for cancer care systems in LMICs. Workshop participants narrowed the list of 216 quality indicators from the literature review to 34 for inclusion in the subsequent surveys. Experts' responses to the surveys showed consensus around nine high-priority quality indicators for measuring the quality of hospital-based cancer care in LMICs. These quality indicators focus on important processes of care delivery from accurate diagnosis (eg, histologic diagnosis via biopsy and TNM staging) to adequate, timely, and appropriate treatment (eg, completion of radiotherapy and appropriate surgical intervention). The core indicators selected could be used to implement systems of feedback and quality improvement.

Service delivery interventions to improve maternal and newborn health in low- and middle-income countries: scoping review of quality improvement, implementation research and health system strengthening approaches

INTRODUCTION

This review explores the characteristics of service delivery-related interventions to improve maternal and newborn health (MNH) in low-and middle-income countries (LMICs) over the last two decades, comparing three common framings of these interventions, namely, quality improvement (QI), implementation science/research (IS/IR), and health system strengthening (HSS).

METHODS

The review followed the staged scoping review methodology proposed by Levac et al. (2010). We developed and piloted a systematic search strategy, limited to English language peer-reviewed articles published on LMICs between 2000 and March 2022. Analysis was conducted in two-quantitative and qualitative-phases. In the quantitative phase, we counted the year of publication, country(-ies) of origin, and the presence of the terms 'quality improvement', 'health system strengthening' or 'implementation science'/ 'implementation research' in titles, abstracts and key words. From this analysis, a subset of papers referred to as 'archetypes' (terms appearing in two or more of titles, abstract and key words) was analysed qualitatively, to draw out key concepts/theories and underlying mechanisms of change associated with each approach.

RESULTS

The searches from different databases resulted in a total of 3,323 hits. After removal of duplicates and screening, a total of 231 relevant articles remained for data extraction. These were distributed across the globe; more than half (n = 134) were published since 2017. Fifty-five (55) articles representing archetypes of the approach (30 QI, 16 IS/IR, 9 HSS) were analysed qualitatively. As anticipated, we identified distinct patterns in each approach. QI archetypes tended towards defined process interventions (most typically, plan-do-study-act cycles); IS/IR archetypes reported a wide variety of interventions, but had in common evaluation methodologies and explanatory theories; and HSS archetypes adopted systemic perspectives. Despite their distinctiveness, there was also overlap and fluidity between approaches, with papers often referencing more than one approach. Recognising the complexity of improving MNH services, there was an increased orientation towards participatory, context-specific designs in all three approaches.

CONCLUSIONS

Programmes to improve MNH outcomes will benefit from a better appreciation of the distinctiveness and relatedness of different approaches to service delivery strengthening, how these have evolved and how they can be combined.

Cost-effectiveness analysis of Smart Triage, a data-driven pediatric sepsis triage platform in Eastern Uganda

BACKGROUND

Sepsis, characterized by organ dysfunction due to presumed or proven infection, has a case-fatality over 20% in severe cases in low-and-middle income countries. Early diagnosis and treatment have proven benefits, prompting our implementation of Smart Triage at Jinja Regional Referral Hospital in Uganda, a program that expedites treatment through a data-driven triage platform. We conducted a cost-effectiveness analysis of Smart Triage to explore its impact on patients and inform multicenter scale up.

METHODS

The parent clinical trial for Smart Triage was pre-post in design, using the proportion of children receiving sepsis treatment within one hour as the primary outcome, a measure linked to mortality benefit in existing literature. We used a decision-analytic model with Monte Carlo simulation to calculate the cost per year-of-life-lost (YLL) averted of Smart Triage from societal, government, and patient perspectives. Healthcare utilization and lost work for seven days post-discharge were translated into costs and productivity losses via secondary linkage data.

RESULTS

In 2021 United States dollars, Smart Triage requires an annuitized program cost of only $0.05 per child, but results in $15.32 saved per YLL averted. At a willingness-to-pay threshold of only $3 per YLL averted, well below published cost-effectiveness threshold estimates for Uganda, Smart Triage approaches 100% probability of cost-effectiveness over the baseline manual triage system. This cost-effectiveness was observed from societal, government, and patient perspectives. The cost-effectiveness observed was driven by a reduction in admission that, while explainable by an improved triage mechanism, may also be partially attributable to changes in healthcare utilization influenced by the coronavirus pandemic. However, Smart Triage remains cost-effective in sensitivity analyses introducing a penalty factor of up to 50% in the reduction in admission.

CONCLUSION

Smart Triage's ability to both save costs and avert YLLs indicates that patients benefit both economically and clinically, while its high probability of cost-effectiveness strongly supports multicenter scale up. Areas for further research include the incorporation of years lived with disability when sepsis disability weights in low-resource settings become available and analyzing budget impact during multicenter scale up.

TRIAL REGISTRATION

NCT04304235 (registered on 11/03/2020, clinicaltrials.gov).

Effects of primary healthcare reform on routine health information systems (RHISs): a mixed-methods study in Lesotho

BACKGROUND AND OBJECTIVE

The Ministry of Health of Lesotho and Partners In Health piloted the Lesotho National Primary Health Care Reform (LPHCR) from July 2014 to June 2017 to improve quality and quantity of service delivery and enhance health system management. This initiative included improvement of routine health information systems (RHISs) to map disease burden and reinforce data utilisation for clinical quality improvement.

METHODS AND ANALYSIS

The WHO Data Quality Assurance framework's core indicators were used to compare the completeness of health data before versus after the LPHCR in 60 health centres and 6 hospitals across four districts. To examine change in data completeness, we conducted an interrupted time series analysis using multivariable logistic mixed-effects regression. Additionally, we conducted 25 key informant interviews with healthcare workers (HCWs) at the different levels of Lesotho's health system, following a purposive sampling approach. Interviews were analysed using deductive coding based on the Performance of Routine Information System Management framework, which inspected organisational, technical and behavioural factors influencing RHIS processes and outputs associated with the LPHCR.

RESULTS

In multivariable analyses, trends in monthly data completion rate were higher after versus before the LPHCR for: documenting first antenatal care visit (adjusted OR (AOR): 1.24, 95% CI: 1.14 to 1.36) and institutional delivery (AOR: 1.19, 95% CI: 1.07 to 1.32). When discussing processes, HCWs highlighted the value of establishing clear roles and responsibilities in reporting under a new organisational structure, improved community programmes among district health management teams, and enhanced data sharing and monitoring by districts.

CONCLUSION

The Ministry of Health had a strong data completion rate pre-LPHCR, which was sustained throughout the LPHCR despite increased service utilisation. The data completion rate was optimised through improved behavioural, technical and organisational factors introduced as part of the LPHCR.

Decolonizing Epidemiological Research: A Critical Perspective

Decolonizing epidemiological research is a crucial endeavor. Historically, colonial and imperialistic ideologies have pervaded epidemiology, leading to an emphasis on Western perspectives and the neglect of indigenous and other marginalized communities' needs and experiences. To effectively address health disparities and promote justice and equality, acknowledging and addressing these power imbalances are imperative. In this article, I highlight the need of decolonizing epidemiological research and make recommendations. These include increasing the representation of researchers from underrepresented communities, ensuring that epidemiological research is contextually relevant and responsive to the experiences of these communities, and collaborating with policymakers and advocacy groups to inform policies and practices that benefit all populations. Moreover, I underscore the importance of recognizing and valuing the knowledge and skills of marginalized populations, and integrating traditional knowledge-the distinct, culturally specific understanding unique to a particular group-into research efforts. I also emphasize the need of capacity building and equitable research collaborations and authorship as well as epidemiological journal editorship. Decolonizing epidemiology research is a continual process that requires continuing discourse, collaboration, and education.

How digital transformation can accelerate data use in health systems

Introduction

We are in an era of rapid technological advance and digitalization. Countries around the world want to leverage technology to improve health outcomes by accelerating data use and increasing evidence-based decision-making to inform action in the health sector. Yet, there is no “one size fits all” approach to achieving this. To understand more, PATH and Cooper/Smith conducted a study documenting and analyzing the experiences of five African countries (Burkina Faso, Ethiopia, Malawi, South Africa, and Tanzania) that are on this digitalization journey. The goal was to examine their different approaches and develop a holistic model of digital transformation for data use that identifies what the essential components for digitalization success are and how they interact with each other.

Methods

Our research had two phases: first, we analyzed documentation from the five countries to identify core components and enabling factors for successful digital transformation, as well as barriers encountered; and second, we held interviews with key informants and focus groups within the countries to fill gaps and validate findings.

Findings

Our findings show that the core components of digital transformation success are highly interrelated. We found that the more successful digitalization efforts address issues that cut across components—such as stakeholder engagement, health workforce capacity, and governance structures—and consider more than just systems and tools. Specifically, we found two critical components of digital transformation that have not been addressed in previous models like the eHealth strategy building blocks developed by the World Health Organization and the International Telecommunication Union: (a) cultivating a culture of data use throughout the health sector and (b) managing the process of system-wide behavior change required to move from manual or paper-based to digital systems.

Conclusion

The resulting model is based on the study's findings and is intended to inform low- and middle-income (LMIC) country governments, global policymakers (such as WHO), implementers, and funders. It provides specific, concrete, evidence-based strategies these key stakeholders can implement to improve digital transformation for data use in health systems, planning, and service delivery.

Spatio-Temporal Bayesian Models for Malaria Risk Using Survey and Health Facility Routine Data in Rwanda

INTRODUCTION

Malaria is a life-threatening disease ocuring mainly in developing countries. Almost half of the world's population was at risk of malaria in 2020. Children under five years age are among the population groups at considerably higher risk of contracting malaria and developing severe disease. Most countries use Demographic and Health Survey (DHS) data for health programs and evaluation. However, malaria elimination strategies require a real-time, locally-tailored response based on malaria risk estimates at the lowest administrative levels. In this paper, we propose a two-step modeling framework using survey and routine data to improve estimates of malaria risk incidence in small areas and enable quantifying malaria trends.

METHODS

To improve estimates, we suggest an alternative approach to modeling malaria relative risk by combining information from survey and routine data through Bayesian spatio-temporal models. We model malaria risk using two steps: (1) fitting a binomial model to the survey data, and (2) extracting fitted values and using them in the Poison model as nonlinear effects in the routine data. We modeled malaria relative risk among under-five-year old children in Rwanda.

RESULTS

The estimation of malaria prevalence among children who are under five years old using Rwanda demographic and health survey data for the years 2019-2020 alone showed a higher prevalence in the southwest, central, and northeast of Rwanda than the rest of the country. Combining with routine health facility data, we detected clusters that were undetected based on the survey data alone. The proposed approach enabled spatial and temporal trend effect estimation of relative risk in local/small areas in Rwanda.

CONCLUSIONS

The findings of this analysis suggest that using DHS combined with routine health services data for active malaria surveillance may provide provide more precise estimates of the malaria burden, which can be used toward malaria elimination targets. We compared findings from geostatistical modeling of malaria prevalence among under-five-year old children using DHS 2019-2020 and findings from malaria relative risk spatio-temporal modeling using both DHS survey 2019-2020 and health facility routine data. The strength of routinely collected data at small scales and high-quality data from the survey contributed to a better understanding of the malaria relative risk at the subnational level in Rwanda.

Understanding maternity care providers' use of data in Southern Tanzania

INTRODUCTION

Health information management system data is collected for national planning and evaluation but is rarely used for healthcare improvements at subnational or facility-level in low-and-middle-income countries. Research suggests that perceived data quality and lack of feedback are contributing factors. We aimed to understand maternity care providers' perceptions of data and how they use it, with a view to co-design interventions to improve data quality and use.

METHODS

We based our research on constructivist grounded theory. We conducted 14 in-depth interviews, two focus group discussions with maternity care providers and 48 hours of observations in maternity wards to understand maternity providers' interaction with data in two rural hospitals in Southern Tanzania. Constant comparative data analysis was applied to develop initial and focused codes, subcategories and categories were continuously validated through peer and member checks.

RESULTS

Maternity care providers found routine health information data of little use to reconcile demands from managers, the community and their challenging working environment within their daily work. They thus added informal narrative documentation sources. They created alternative narratives through data of a maternity care where mothers and babies were safeguarded. The resulting documentation system, however, led to duplication and increased systemic complexity.

CONCLUSIONS

Current health information systems may not meet all data demands of maternity care providers, or other healthcare workers. Policy makers and health information system specialists need to acknowledge different ways of data use beyond health service planning, with an emphasis on healthcare providers' data needs for clinical documentation.

Characterising performance information use in the primary healthcare systems of El Salvador, Lebanon and Malawi: multiple qualitative case study protocol

INTRODUCTION

Governments in low-income and middle-income countries (LMICs) and official development assistance agencies use a variety of performance measurement and management approaches to improve the performance of healthcare systems. The effectiveness of such approaches is contingent on the extent to which managers and care providers use performance information. To date, major knowledge gaps exist about the contextual factors that contribute, or not, to performance information use by primary healthcare (PHC) decision-makers in LMICs. This study will address three research questions: (1) How do decision-makers use performance information, and for what purposes? (2) What are the contextual factors that influence the use or non-use of performance information? and (3) What are the proximal outcomes reported by PHC decision-makers from performance information use?

METHODS AND ANALYSIS

We present the protocol of a theory-driven, qualitative study with a multiple case study design to be conducted in El Salvador, Lebanon and Malawi.Data sources include semi structured in-depth interviews and document review. Interviews will be conducted with approximately 60 respondents including PHC system decision-makers and providers. We follow an interdisciplinary theoretical framework that draws on health policy and systems research, public administration, organisational science and health service research. Data will be analysed using thematic analysis to explore how respondents use performance information or not, and for what purposes as well as barriers and facilitators of use.

ETHICS AND DISSEMINATION

The ethical boards of the participating universities approved the protocol presented here. Study results will be disseminated through peer-reviewed journals and global health conferences.

Implementing value-based healthcare using a digital health exchange platform to improve pregnancy and childbirth outcomes in urban and rural Kenya

Maternal and neonatal mortality rates in many low- and middle-income countries (LMICs) are still far above the targets of the United Nations Sustainable Development Goal 3. Value-based healthcare (VBHC) has the potential to outperform traditional supply-driven approaches in changing this dismal situation, and significantly improve maternal, neonatal and child health (MNCH) outcomes. We developed a theory of change and used a cohort-based implementation approach to create short and long learning cycles along which different components of the VBHC framework were introduced and evaluated in Kenya. At the core of the approach was a value-based care bundle for maternity care, with predefined cost and quality of care using WHO guidelines and adjusted to the risk profile of the pregnancy. The care bundle was implemented using a digital exchange platform that connects pregnant women, clinics and payers. The platform manages financial transactions, enables bi-directional communication with pregnant women via SMS, collects data from clinics and shares enriched information via dashboards with payers and clinics. While the evaluation of health outcomes is ongoing, first results show improved adherence to evidence-based care pathways at a predictable cost per enrolled person. This community case study shows that implementation of the VBHC framework in an LMIC setting is possible for MNCH. The incremental, cohort-based approach enabled iterative learning processes. This can support the restructuring of health systems in low resource settings from an output-driven model to a value based financing-driven model.

Progress towards achieving child survival goals in Kenya after devolution: Geospatial analysis with scenario-based projections, 2015-2025

Subnational projections of under-5 mortality (U5M) have increasingly become an essential planning tool to support Sustainable Development Goals (SDGs) agenda and strategies for improving child survival. To support child health policy, planning, and tracking child development goals in Kenya, we projected U5M at units of health decision making. County-specific annual U5M were estimated using a multivariable Bayesian space-time hierarchical model based on intervention coverage from four alternate intervention scale-up scenarios assuming 1) the highest subnational intervention coverage in 2014, 2) projected coverage based on the fastest county-specific rate of change observed in the period between 2003-2014 for each intervention, 3) the projected national coverage based on 2003-2014 trends and 4) the country-specific targets of intervention coverage relative to business as usual (BAU) scenario. We compared the percentage change in U5M based on the four scale-up scenarios relative to BAU and examined the likelihood of reaching SDG 3.2 target of at least 25 deaths/1,000 livebirths by 2022 and 2025. Projections based on 10 factors assuming BAU, showed marginal reductions in U5M across counties with all the counties except Mandera county not achieving the SDG 3.2 target by 2025. Further, substantial reductions in U5M would be achieved based on the various intervention scale-up scenarios, with 63.8% (30), 74.5% (35), 46.8% (22) and 61.7% (29) counties achieving SDG target for scenarios 1,2,3 and 4 respectively by 2025. Scenario 2 yielded the highest reductions of U5M with individual scale-up of access to improved water, recommended treatment of fever and accelerated HIV prevalence reduction showing considerable impact on U5M reduction (≥ 20%) relative to BAU. Our results indicate that sustaining an ambitious intervention scale-up strategy matching the fastest rate observed between 2003-2014 would substantially reduce U5M in Kenya. However, despite this ambitious scale-up scenario, 25% (12 of 47) of the Kenya's counties would still not achieve SDG 3.2 target by 2025.

The use of and need for data and information by health professionals supporting the delivery of palliative cancer care services in sub-Saharan Africa: A qualitative study

PURPOSE

The ability to develop and evaluate approaches to the management of advanced cancer in sub-Saharan Africa is limited by the lack of local, reliable and valid data to ensure that practice is evidence-based, replicable and reflects the needs of the population served.

METHODS

A secondary qualitative analysis of in-depth interviews with 59 health professionals delivering palliative cancer care in Nigeria, Uganda and Zimbabwe were conducted to determine the use and needs for data and information for patient care and service delivery. Framework analysis was used, informed by a conceptual model for data use in low and middle-income countries.

RESULTS

Three meta-themes include: (1) Current practice in data gathering and use; (2) Gaps for capturing, storing information and supporting communication, and; (3) Needs and opportunities for data use. Deficits in current data access and use were identified, alongside targets for improving the quality, accessibility and utility of data to inform the development of palliative cancer care.

CONCLUSIONS

The availability and use of relevant and reliable data relating to the current provision of palliative cancer care are requisite for the contextually appropriate and effective development of health services. The requirements and constraints articulated by participants can guide future development and optimisation of digital health approaches for palliative cancer care in the participating countries, with relevance to the wider sub-Saharan Africa region.

Barriers and Facilitators to Data Use for Decision Making: The Experience of the African Health Initiative Partnerships in Ethiopia, Ghana, and Mozambique

BACKGROUND

Three African Health Initiative (AHI) partnership projects in Ethiopia, Ghana, and Mozambique implemented strategies to improve the quality and evaluation of routinely collected data at the primary health care level and stimulate its use in evidence-based decision making. We compare how these programs designed and carried out data for decision-making (DDM) strategies, elaborate on barriers and facilitators to implementation success, and offer recommendations for future DDM programming.

METHODS

Researchers from each project collaboratively wrote a cross-country protocol based on these objectives. By adapting the Consolidated Framework for Implementation Research (CFIR) through a qualitative theme reduction process, they harmonized lines of inquiry on the design of the respective DDM strategies and the barriers and facilitators of effective implementation. We conducted in-depth interviews and focus group discussions with stakeholders from the primary health care systems in each country, and we carried out multistage, thematic analyses using a deductive lens.

RESULTS

Effective implementation of DDM depended on whether implementers felt that DDM was adaptable to context, feasible to trial, and easy to introduce and maintain. The prevailing policy and political environment in the wider health system, learning climate and absorptive capacity for evidence-based change in DDM settings, engagement of external change agents and internal change leaders, and promotion of opportunities and means for team-based reflection and evaluations of what works influenced the success or failure of DDM strategies.

CONCLUSION

Opportunities for team-based capacity building and individual mentorship led to effective DDM programming. External policies and associated incentives bolstered this but occasionally led to unintended consequences. Leadership engagement and availability of resources to act on recommendations; respond to capacity-building needs; and facilitate collaborations between peers, within hierarchies, and across the local health system proved crucial to DDM, as was encouraging adaptation and opportunities for iterative on-the-job learning.

Learning Health Systems to Bridge the Evidence-Policy-Practice Gap in Primary Health Care: Lessons From the African Health Initiative

The compilation of lessons in this supplement on the Doris Duke Charitable Foundation’s African Health Initiative’s work in the application of implementation research in primary health care in sub-Saharan Africa reflects the evolution of the discipline that is now increasingly recognized as integral to health systems strengthening.

Systems analysis and improvement approach to optimize outpatient mental health treatment cascades in Mozambique (SAIA-MH): study protocol for a cluster randomized trial

BACKGROUND

Significant investments are being made to close the mental health (MH) treatment gap, which often exceeds 90% in many low- and middle-income countries (LMICs). However, limited attention has been paid to patient quality of care in nascent and evolving LMIC MH systems. In system assessments across sub-Saharan Africa, MH loss-to-follow-up often exceeds 50% and sub-optimal medication adherence often exceeds 60%. This study aims to fill a gap of evidence-based implementation strategies targeting the optimization of MH treatment cascades in LMICs by testing a low-cost multicomponent implementation strategy integrated into routine government MH care in Mozambique.

METHODS

Using a cluster-randomized trial design, 16 clinics (8 intervention and 8 control) providing primary MH care will be randomized to the Systems Analysis and Improvement Approach for Mental Health (SAIA-MH) or an attentional placebo control. SAIA-MH is a multicomponent implementation strategy blending external facilitation, clinical consultation, and provider team meetings with system-engineering tools in an overall continuous quality improvement framework. Following a 6-month baseline period, intervention facilities will implement the SAIA-MH strategy for a 2-year intensive implementation period, followed by a 1-year sustainment phase. Primary outcomes will be the proportion of all patients diagnosed with a MH condition and receiving pharmaceutical-based treatment who achieve functional improvement, adherence to medication, and retention in MH care. The Consolidated Framework for Implementation Research (CFIR) will be used to assess determinants of implementation success. Specific Aim 1b will include the evaluation of mechanisms of the SAIA-MH strategy using longitudinal structural equation modeling as well as specific aim 2 estimating cost and cost-effectiveness of scaling-up SAIA-MH in Mozambique to provincial and national levels.

DISCUSSION

This study is innovative in being the first, to our knowledge, to test a multicomponent implementation strategy for MH care cascade optimization in LMICs. By design, SAIA-MH is a low-cost strategy to generate contextually relevant solutions to barriers to effective primary MH care, and thus focuses on system improvements that can be sustained over the long term. Since SAIA-MH is integrated into routine government MH service delivery, this pragmatic trial has the potential to inform potential SAIA-MH scale-up in Mozambique and other similar LMICs.

TRIAL REGISTRATION

ClinicalTrials.gov; NCT05103033 ; 11/2/2021.

The Effectiveness of the Capacity Building and Mentorship Program in Improving Evidence-Based Decision-making in the Amhara Region, Northwest Ethiopia: Difference-in-Differences Study

Background

Weak health information systems (HISs) hobble countries’ abilities to effectively manage and distribute their resources to match the burden of disease. The Capacity Building and Mentorship Program (CBMP) was implemented in select districts of the Amhara region of Ethiopia to improve HIS performance; however, evidence about the effectiveness of the intervention was meager.

Objective

This study aimed to determine the effectiveness of routine health information use for evidence-based decision-making among health facility and department heads in the Amhara region, Northwest Ethiopia.

Methods

The study was conducted in 10 districts of the Amhara region: five were in the intervention group and five were in the comparison group. We employed a quasi-experimental study design in the form of a pretest-posttest comparison group. Data were collected from June to July 2020 from the heads of departments and facilities in 36 intervention and 43 comparison facilities. The sample size was calculated using the double population formula, and we recruited 172 participants from each group. We applied a difference-in-differences analysis approach to determine the effectiveness of the intervention. Heterogeneity of program effect among subgroups was assessed using a triple differences method (ie, difference-in-difference-in-differences [DIDID] method). Thus, the β coefficients, 95% CIs, and P values were calculated for each parameter, and we determined that the program was effective if the interaction term was significant at P<.05.

Results

Data were collected using the endpoint survey from 155 out of 172 (90.1%) participants in the intervention group and 166 out of 172 (96.5%) participants in the comparison group. The average level of information use for the comparison group was 37.3% (95% CI 31.1%-43.6%) at baseline and 43.7% (95% CI 37.9%-49.5%) at study endpoint. The average level of information use for the intervention group was 52.2% (95% CI 46.2%-58.3%) at baseline and 75.8% (95% CI 71.6%-80.0%) at study endpoint. The study indicated that the net program change over time was 17% (95% CI 5%-28%; P=.003). The subgroup analysis also indicated that location showed significant program effect heterogeneity, with a DIDID estimate equal to 0.16 (95% CI 0.026-0.29; P=.02). However, sex, age, educational level, salary, and experience did not show significant heterogeneity in program effect, with DIDID estimates of 0.046 (95% CI –0.089 to 0.182), –0.002 (95% CI –0.015 to 0.009), –0.055 (95% CI –0.190 to 0.079), –1.63 (95% CI –5.22 to 1.95), and –0.006 (95% CI –0.017 to 0.005), respectively.

Conclusions

The CBMP was effective at enhancing the capacity of study participants in using the routine HIS for decision-making. We noted that urban facilities had benefited more than their counterparts. The intervention has been shown to produce positive outcomes and should be scaled up to be used in other districts. Moreover, the mentorship modalities for rural facilities should be redesigned to maximize the benefits.

Trial Registration

Pan African Clinical Trials Registry PACTR202001559723931; https://tinyurl.com/3j7e5ka5

Routine Health Information Utilization and Associated Factors among Health Professionals Working in Public Health Facilities of the South Region, Ethiopia

BACKGROUND

Routine health information is the pillar for planning and management of health services and plays a vital role in effective and efficient health service delivery, decision making, and the improvement of programs. Therefore, this study aimed to assess routine health information utilization and associated factors among health professionals working in public health facilities of the south region.

METHODS

Institution based cross-sectional study design was employed. Data was collected from randomly selected 719 participants using a pre-tested, interviewer administered structured questionnaire. Bivariate and multivariate logistic regression analyses were carried out.

RESULT

The overall utilization of routine health information was 63.1. Place of residence, HMIS personnel, HMIS code, overwhelming data source, population based data, data quality control, feedback, monitoring chart, 8.467) and data transfer policy were factors significantly associated with utilization of routine health information.

CONCLUSION

Six out of ten health professionals had utilized routine health information. Place of residence, HMIS personnel, HMIS code, overwhelming data source, population based data, data quality control, feedback, monitoring chart and data transfer policy had significant associations with routine health information utilization. Therefore, concerned health authorities need to work on these factors to improve the utilization.

Leveraging Digital Tools and Crowdsourcing Approaches to Generate High-Frequency Data for Diet Quality Monitoring at Population Scale in Rwanda

Diet quality is a critical determinant of human health and increasingly serves as a key indicator for food system sustainability. However, data on diets are limited, scattered, often project-dependent, and current data collection systems do not support high-frequency or consistent data flows. We piloted in Rwanda a data collection system, powered by the principles of citizen science, to acquire high frequency data on diets. The system was deployed through an unstructured supplementary service data platform, where respondents were invited to answer questions regarding their dietary intake. By combining micro-incentives with a normative nudge, 9,726 responses have been crowdsourced over 8 weeks of data collection. The cost per respondent was &lt; $1 (system set-up, maintenance, and a small payment to respondents), with interactions taking &lt;15 min. Exploratory analyses show that &gt;70% of respondents consume tubers and starchy vegetables, leafy vegetables, fruits, legumes, and wholegrains. Women consumed better quality diets than male respondents, revealing a sex-based disparity in diet quality. Similarly, younger respondents (age ≤ 24 years) consumed the lowest quality diets, which may pose significant risks to their health and mental well-being. Middle-income Rwandans were identified to have consumed the highest quality diets. Long-term tracking of diet quality metrics could help flag populations and locations with high probabilities of nutrition insecurity, in turn guiding relevant interventions to mitigate associated health and social risks.

Electronic health records, mobile health, and the challenge of improving global health

Technology continues to impact healthcare around the world. This provides great opportunities, but also risks. These risks are compounded in low-resource settings where errors in planning and implementation may be more difficult to overcome. Global Health Informatics provides lessons in both opportunities and risks by building off of general Global Health. Global Health Informatics also requires a thorough understanding of the local environment and the needs of low-resource settings. Forming effective partnerships and following the lead of local experts are necessary for sustainability; it also ensures that the priorities of the local community come first. There is an opportunity for partnerships between low-resource settings and high income areas that can provide learning opportunities to avoid the pitfalls that plague many digital health systems and learn how to properly implement technology that truly improves healthcare.

Exploring data quality and use of the routine health information system in Ethiopia: a mixed-methods study

OBJECTIVE

A routine health information system (RHIS) enables decision making in the healthcare system. We aimed to analyse data quality at the district and regional level and explore factors and perceptions affecting the quality and use of routine data.

DESIGN

This was a mixed-methods study. We used the WHO toolkit for analysing data quality and interviewed staff at the point of data generation and along with the flow of data. Data were analysed using the Performance of Routine Information System Management framework.

SETTING

This study was performed in eight districts in four regions of Ethiopia. The study was nested within a 2-year programme of the Operational Research and Coaching for government Analysts.

PARTICIPANTS

We visited 45 health posts, 1 district hospital, 16 health centres and 8 district offices for analysis of routine RHIS data and interviewed 117 staff members for the qualitative assessment.

OUTCOME MEASURES

We assessed availability of source documents, completeness, timeliness and accuracy of reporting of routine data, and explored data quality and use perceptions.

RESULTS

There was variable quality of both indicator and data element. Data on maternal health and immunisation were of higher quality than data on child nutrition. Issues ranged from simple organisational factors, such as availability of register books, to intricate technical issues, like complexity of indicators and choice of denominators based on population estimates. Respondents showed knowledge of the reporting procedures, but also demonstrated limited skills, lack of supportive supervision and reporting to please the next level. We saw limited examples of the use of data by the staff who were responsible for data reporting.

CONCLUSION

We identified important organisational, technical, behavioural and process factors that need further attention to improve the quality and use of RHIS data in Ethiopia.

The Contribution of Artificial Intelligence in Achieving the Sustainable Development Goals (SDGs): What Can Eye Health Can Learn From Commercial Industry and Early Lessons From the Application of Machine Learning in Eye Health Programmes

Achieving The United Nations sustainable developments goals by 2030 will be a challenge. Researchers around the world are working toward this aim across the breadth of healthcare. Technology, and more especially artificial intelligence, has the ability to propel us forwards and support these goals but requires careful application. Artificial intelligence shows promise within healthcare and there has been fast development in ophthalmology, cardiology, diabetes, and oncology. Healthcare is starting to learn from commercial industry leaders who utilize fast and continuous testing algorithms to gain efficiency and find the optimum solutions. This article provides examples of how commercial industry is benefitting from utilizing AI and improving service delivery. The article then provides a specific example in eye health on how machine learning algorithms can be purposed to drive service delivery in a resource-limited setting by utilizing the novel study designs in response adaptive randomization. We then aim to provide six key considerations for researchers who wish to begin working with AI technology which include collaboration, adopting a fast-fail culture and developing a capacity in ethics and data science.

Effect of birthweight measurement quality improvement on low birthweight prevalence in rural Ethiopia

BACKGROUND

Low birthweight (LBW) (< 2500 g) is a significant determinant of infant morbidity and mortality worldwide. In low-income settings, the quality of birthweight data suffers from measurement and recording errors, inconsistent data reporting systems, and missing data from non-facility births. This paper describes birthweight data quality and the prevalence of LBW before and after implementation of a birthweight quality improvement (QI) initiative in Amhara region, Ethiopia.

METHODS

A comparative pre-post study was performed in selected rural health facilities located in West Gojjam and South Gondar zones. At baseline, a retrospective review of delivery records from February to May 2018 was performed in 14 health centers to collect birthweight data. A birthweight QI initiative was introduced in August 2019, which included provision of high-quality digital infant weight scales (precision 5 g), routine calibration, training in birth weighing and data recording, and routine field supervision. After the QI implementation, birthweight data were prospectively collected from late August to early September 2019, and December 2019 to June 2020. Data quality, as measured by heaping (weights at exact multiples of 500 g) and rounding to the nearest 100 g, and the prevalence of LBW were calculated before and after QI implementation.

RESULTS

We retrospectively reviewed 1383 delivery records before the QI implementation and prospectively measured 1371 newborn weights after QI implementation. Heaping was most frequently observed at 3000 g and declined from 26% pre-initiative to 6.7% post-initiative. Heaping at 2500 g decreased from 5.4% pre-QI to 2.2% post-QI. The percentage of rounding to the nearest 100 g was reduced from 100% pre-initiative to 36.5% post-initiative. Before the QI initiative, the prevalence of recognized LBW was 2.2% (95% confidence interval [CI]: 1.5-3.1) and after the QI initiative increased to 11.7% (95% CI: 10.1-13.5).

CONCLUSIONS

A QI intervention can improve the quality of birthweight measurements, and data measurement quality may substantially affect estimates of LBW prevalence.

Changes in health worker knowledge and motivation in the context of a quality improvement programme in Ethiopia

A knowledgeable and motivated workforce is critical for health systems to provide high-quality services. Many low- and middle-income countries face shortages in human resources and low health worker motivation but are also home to a burgeoning number of quality improvement (QI) programmes. This study evaluates whether and how motivation and clinical knowledge in three cadres of health workers changed in the context of a QI programme for maternal and newborn health in Ethiopia. This mixed-methods study used a pre–post comparison group design with matched comparison areas. We interviewed 395 health workers at baseline in April 2018 and 404 at endline in June 2019 from seven districts (woredas) with QI and seven comparison woredas. Three cadres were interviewed: health extension workers, facility-based skilled midlevel maternal and newborn care providers, and non-patient-facing staff. A qualitative component sought to triangulate and further elucidate quantitative findings using in-depth interviews with 22 health workers. Motivation was assessed quantitatively, exploratory factor analysis was used to categorize motivation dimensions, and regression-based difference-in-difference analyses were conducted. Knowledge was assessed through a clinical vignette. Qualitative data were analysed in a deductive process based on a framework derived from quantitative results. Although knowledge of the QI programme was high (79%) among participants from QI woreda at endline, participation in QI teams was lower (56%). There was strong evidence that health worker knowledge increased more in areas with QI than comparison areas. Three motivation dimensions emerged from the data: (1) ‘helping others’, (2) ‘pride and satisfaction’ and (3) ‘external recognition and support’. We found strong evidence that motivation across these factors improved in both QI and comparison areas, with weak evidence of greater increases in comparison areas. Qualitative data suggested the QI programme may have improved motivation by allowing staff to provide better care. This study suggests that although QI programmes can increase health worker knowledge, there may be little effect on motivation. Programme evaluations should measure a wide range of outcomes to fully understand their impact.

Performance management in complex adaptive systems: a conceptual framework for health systems

Existing performance management approaches in health systems in low-income and middle-income countries are generally ineffective at driving organisational-level and population-level outcomes. They are largely directive: they try to control behaviour using targets, performance monitoring, incentives and answerability to hierarchies. In contrast, enabling approaches aim to leverage intrinsic motivation, foster collective responsibility, and empower teams to self-organise and use data for shared sensemaking and decision-making.The current evidence base is too limited to guide reforms to strengthen performance management in a particular context. Further, existing conceptual frameworks are undertheorised and do not consider the complexity of dynamic, multilevel health systems. As a result, they are not able to guide reforms, particularly on the contextually appropriate balance between directive and enabling approaches. This paper presents a framework that attempts to situate performance management within complex adaptive systems. Building on theoretical and empirical literature across disciplines, it identifies interdependencies between organisational performance management, organisational culture and software, system-level performance management, and the system-derived enabling environment. It uses these interdependencies to identify when more directive or enabling approaches may be more appropriate. The framework is intended to help those working to strengthen performance management to achieve greater effectiveness in organisational and system performance. The paper provides insights from the literature and examples of pitfalls and successes to aid this thinking. The complexity of the framework and the interdependencies it describes reinforce that there is no one-size-fits-all blueprint for performance management, and interventions must be carefully calibrated to the health system context.

Using data-driven approaches to improve delivery of animal health care interventions for public health

Rabies is arguably the exemplar of the One Health Agenda in which preventative health care in one species can improve health of other species. Interrogation of large epidemiology datasets offers the potential to deliver health care initiatives in a more efficient and cost-effective manner. However, real-life examples demonstrating this potential are limited. Here, we report a real-time, data-driven approach to improve cost effectiveness of dog vaccination campaigns in urban sub-Saharan African settings, which eliminates the need of expensive door-to-door vaccination by replacing them with strategically positioned fixed and roaming static points (SPs). This approach has the potential to act as a template for future successful and sustainable urban SP-only dog vaccination campaigns.

Rabies kills ∼60,000 people per year. Annual vaccination of at least 70% of dogs has been shown to eliminate rabies in both human and canine populations. However, delivery of large-scale mass dog vaccination campaigns remains a challenge in many rabies-endemic countries. In sub-Saharan Africa, where the vast majority of dogs are owned, mass vaccination campaigns have typically depended on a combination of static point (SP) and door-to-door (D2D) approaches since SP-only campaigns often fail to achieve 70% vaccination coverage. However, D2D approaches are expensive, labor-intensive, and logistically challenging, raising the need to develop approaches that increase attendance at SPs. Here, we report a real-time, data-driven approach to improve efficiency of an urban dog vaccination campaign. Historically, we vaccinated ∼35,000 dogs in Blantyre city, Malawi, every year over a 20-d period each year using combined fixed SP (FSP) and D2D approaches. To enhance cost effectiveness, we used our historical vaccination dataset to define the barriers to FSP attendance. Guided by these insights, we redesigned our vaccination campaign by increasing the number of FSPs and eliminating the expensive and labor-intensive D2D component. Combined with roaming SPs, whose locations were defined through the real-time analysis of vaccination coverage data, this approach resulted in the vaccination of near-identical numbers of dogs in only 11 d. This approach has the potential to act as a template for successful and sustainable future urban SP-only dog vaccination campaigns.

Data and information needs of policymakers for palliative cancer care: a multi-country qualitative study

Background

Despite regional efforts to address concerns regarding the burden of advanced cancer in Africa, urgent attention is still required. Widespread issues include late symptom presentation, inaccessibility of palliative care services, limited resources, poor data quality, disparity in data availability, and lack of stakeholder engagement. One way of helping to address these issues is by understanding and meeting the data and information needs of policymakers in palliative cancer care.

Aims

To explore the views of policymakers regarding data availability, data gaps and preferred data formats to support policy and decision making for palliative cancer care in Nigeria, Uganda and Zimbabwe.

Methods

A secondary analysis of interview data collected as part of a cross-sectional qualitative study that aimed to explore the data and information needs of patients, policymakers and caregivers in Nigeria, Uganda and Zimbabwe. Framework analysis, guided by the MEASURE evaluation framework, was used to qualitatively analyse the data.

Results

Twenty-six policymakers were recruited. The policymakers data and information concerns are aligned to the MEASURE evaluation framework of data and information use and include; assessing and improving data use (e.g. low prioritisation of cancer); identifying and engaging the data user (e.g. data processes); improving data quality (e.g. manual data collection processes); improving data availability (e.g. the accessibility of data); identifying information needs (e.g. what is ‘need to know’?); capacity building in core competencies (e.g. skills gaps); strengthening organisational data demand and use (e.g. policy frameworks); monitoring, evaluating and communicating of data demand and use (e.g. trustworthiness of data).

Conclusions

We present evidence of data sources, challenges to their access and use, guidance on data needs for policymakers, and opportunities for better engagement between data producers, brokers and users. This framework of evidence should inform the development of strategies to improve data access and use for policy and decision making to improve palliative cancer services in participating countries with relevance to the wider region.

Surgical data strengthening in Ethiopia: results of a Kirkpatrick framework evaluation of a data quality intervention

Background: One key challenge in improving surgical care in resource-limited settings is the lack of high-quality and informative data. In Ethiopia, the Safe Surgery 2020 (SS2020) project developed surgical key performance indicators (KPIs) to evaluate surgical care within the country. New data collection methods were developed and piloted in 10 SS2020 intervention hospitals in the Amhara and Tigray regions of Ethiopia.

Objective: To assess the feasibility of collecting and reporting new surgical indicators and measure the impact of a surgical Data Quality Intervention (DQI) in rural Ethiopian hospitals.

Methods: An 8-week DQI was implemented to roll-out new data collection tools in SS2020 hospitals. The Kirkpatrick Method, a widely used mixed-method evaluation framework for training programs, was used to assess the impact of the DQI. Feedback surveys and focus groups at various timepoints evaluated the impact of the intervention on surgical data quality, the feasibility of a new data collection system, and the potential for national scale-up.

Results: Results of the evaluation are largely positive and promising. DQI participants reported knowledge gain, behavior change, and improved surgical data quality, as well as greater teamwork, communication, leadership, and accountability among surgical staff. Barriers remained in collection of high-quality data, such as lack of adequate human resources and electronic data reporting infrastructure.

Conclusions: Study results are largely positive and make evident that surgical data capture is feasible in low-resource settings and warrants more investment in global surgery efforts. This type of training and mentorship model can be successful in changing individual behavior and institutional culture regarding surgical data collection and reporting. Use of the Kirkpatrick Framework for evaluation of a surgical DQI is an innovative contribution to literature and can be easily adapted and expanded for use within global surgery.

Interventions to improve district-level routine health data in low-income and middle-income countries: a systematic review

BACKGROUND

Routine health information system(s) (RHIS) facilitate the collection of health data at all levels of the health system allowing estimates of disease prevalence, treatment and preventive intervention coverage, and risk factors to guide disease control strategies. This core health system pillar remains underdeveloped in many low-income and middle-income countries. Efforts to improve RHIS data coverage, quality and timeliness were launched over 10 years ago.

METHODS

A systematic review was performed across 12 databases and literature search engines for both peer-reviewed articles and grey literature reports on RHIS interventions. Studies were analysed in three stages: (1) categorisation of RHIS intervention components and processes; (2) comparison of intervention component effectiveness and (3) whether the post-intervention outcome improved above the WHO integrated disease surveillance response framework data quality standard of 80% or above.

RESULTS

5294 references were screened, resulting in 56 studies. Three key performance determinants-technical, organisational and behavioural-were proposed as critical to RHIS strengthening. Seventy-seven per cent [77%] of studies identified addressed all three determinants. The most frequently implemented intervention components were 'providing training' and 'using an electronic health management information systems'. Ninety-three per cent [93%] of pre-post or controlled trial studies showed improvements in one or more data quality outputs, but after applying a standard threshold of >80% post-intervention, this number reduced to 68%. There was an observed benefit of multi-component interventions that either conducted data quality training or that addressed improvement across multiple processes and determinants of RHIS.

CONCLUSION

Holistic data quality interventions that address multiple determinants should be continuously practised for strengthening RHIS. Studies with clearly defined and pragmatic outcomes are required for future RHIS improvement interventions. These should be accompanied by qualitative studies and cost analyses to understand which investments are needed to sustain high-quality RHIS in low-income and middle-income countries.

Barriers and enablers to routine register data collection for newborns and mothers: EN-BIRTH multi-country validation study

BACKGROUND

Policymakers need regular high-quality coverage data on care around the time of birth to accelerate progress for ending preventable maternal and newborn deaths and stillbirths. With increasing facility births, routine Health Management Information System (HMIS) data have potential to track coverage. Identifying barriers and enablers faced by frontline health workers recording HMIS source data in registers is important to improve data for use.

METHODS

The EN-BIRTH study was a mixed-methods observational study in five hospitals in Bangladesh, Nepal and Tanzania to assess measurement validity for selected Every Newborn coverage indicators. We described data elements required in labour ward registers to track these indicators. To evaluate barriers and enablers for correct recording of data in registers, we designed three interview tools: a) semi-structured in-depth interview (IDI) guide b) semi-structured focus group discussion (FGD) guide, and c) checklist assessing care-to-documentation. We interviewed two groups of respondents (January 2018-March 2019): hospital nurse-midwives and doctors who fill ward registers after birth (n = 40 IDI and n = 5 FGD); and data collectors (n = 65). Qualitative data were analysed thematically by categorising pre-identified codes. Common emerging themes of barriers or enablers across all five hospitals were identified relating to three conceptual framework categories.

RESULTS

Similar themes emerged as both barriers and enablers. First, register design was recognised as crucial, yet perceived as complex, and not always standardised for necessary data elements. Second, register filling was performed by over-stretched nurse-midwives with variable training, limited supervision, and availability of logistical resources. Documentation complexity across parallel documents was time-consuming and delayed because of low staff numbers. Complete data were valued more than correct data. Third, use of register data included clinical handover and monthly reporting, but little feedback was given from data users.

CONCLUSION

Health workers invest major time recording register data for maternal and newborn core health indicators. Improving data quality requires standardised register designs streamlined to capture only necessary data elements. Consistent implementation processes are also needed. Two-way feedback between HMIS levels is critical to improve performance and accurately track progress towards agreed health goals.

Uterotonics for prevention of postpartum haemorrhage: EN-BIRTH multi-country validation study

BACKGROUND

Postpartum haemorrhage (PPH) is a leading cause of preventable maternal mortality worldwide. The World Health Organization (WHO) recommends uterotonic administration for every woman after birth to prevent PPH. There are no standardised data collected in large-scale measurement platforms. The Every Newborn Birth Indicators Research Tracking in Hospitals (EN-BIRTH) is an observational study to assess the validity of measurement of maternal and newborn indicators, and this paper reports findings regarding measurement of coverage and quality for uterotonics.

METHODS

The EN-BIRTH study took place in five hospitals in Bangladesh, Nepal and Tanzania, from July 2017 to July 2018. Clinical observers collected tablet-based, time-stamped data. We compared observation data for uterotonics to routine hospital register-records and women's report at exit-interview survey. We analysed the coverage and quality gap for timing and dose of administration. The register design was evaluated against gap analyses and qualitative interview data assessing the barriers and enablers to data recording and use.

RESULTS

Observed uterotonic coverage was high in all five hospitals (> 99%, 95% CI 98.7-99.8%). Survey-report underestimated coverage (79.5 to 91.7%). "Don't know" replies varied (2.1 to 14.4%) and were higher after caesarean (3.7 to 59.3%). Overall, there was low accuracy in survey data for details of uterotonic administration (type and timing). Register-recorded coverage varied in four hospitals capturing uterotonics in a specific column (21.6, 64.5, 97.6, 99.4%). The average coverage measurement gap was 18.1% for register-recorded and 6.0% for survey-reported coverage. Uterotonics were given to 15.9% of women within the "right time" (1 min) and 69.8% within 3 min. Women's report of knowing the purpose of uterotonics after birth ranged from 0.4 to 64.9% between hospitals. Enabling register design and adequate staffing were reported to improve routine recording.

CONCLUSIONS

Routine registers have potential to track uterotonic coverage - register data were highly accurate in two EN-BIRTH hospitals, compared to consistently underestimated coverage by survey-report. Although uterotonic coverage was high, there were gaps in observed quality for timing and dose. Standardisation of register design and implementation could improve data quality and data flow from registers into health management information reporting systems, and requires further assessment.

TB notification rates across parliamentary constituencies in India: a step towards data-driven political engagement

OBJECTIVE

National averages obscure geographic variation in program performance. We determined Parliamentary Constituency (PC)-wise estimates of TB notification to guide political engagement.

METHODS

We extracted district-level TB notification data from the 2018 annual TB report. We derived PC-level estimates by building a 'cross-walk' between districts and PCs using boundary shapefiles. We described the spatial distribution of the PC-wise estimates of Total Notification Rate and percentage of Private Sector Notification.

RESULTS

The median PC-wise Total Notification Rate was 126.24/100 000 (IQR: 94.86/100 000, 162.22/100 000). The median PC-wise Percentage Private Sector Notification was 18.03% (IQR: 9.56%, 26.84%). Only 16 (2.94%) PCs met the target of 50% private sector notification. Most of high notification rates in PCs were driven by high notification in public sector. There was geographic - both interstate and within state inter-PC - variation in the estimates of these indicators. The study identified some geographic patterns of notification - high positive outlier PCs with adjoining PCs in lower deciles of notification rates, intra-state differences in PC performance, and similarities in notification rates of adjoining PCs in different states.

CONCLUSION

In addition to regional inequality, the study identified geospatial patterns that can aid in the formulation of suitable interventions. These include decongestion of overburdened facilities by strengthening poorly performing units. The PCs with a high percentage Private Sector Notification can act as role models for neighbouring PCs to improve private sector engagement. MPs can play a crucial role in mobilising additional resources, creating awareness, and establishing inter-PC and inter-state collaboration to improve TB program performance.

Using data to improve outcomes of supplemental immunisation activities: 2017/2018 Nigeria measles vaccination campaign

BACKGROUND

Despite the enormous resources committed to the implementation of supplemental immunisation activities in Nigeria, achieving the required coverage (post-campaign survey) to halt the transmission of vaccine-preventable diseases has continued to seem like an impossibility. A vast volume of data is generated and transmitted during mass vaccination campaigns, but this administrative data does not always culminate into improved coverage. The absence of data-informed guidance from stakeholders with long years of experience in planning and implementing mass vaccination campaigns has impeded achieving 95% coverage in measles campaigns in Nigeria. This study reviews the use of data to guide the implementation of the 2017/2018 measles vaccination campaign in Nigeria.

METHODS

A central coordinating body was formed at the national level with the same replicated in every state. Tools were developed to measure the performance of the different phases and activities required for the implementation of a mass vaccination campaign as recommended in the international guidelines. Stakeholders were engaged to help ensure that feedback provided by the national measles technical coordinating committee was implemented at the lower level.

RESULTS

Monitoring and analysis of daily data submission caused a proper spread of senior supervisors, vaccination posts location during the campaign and helped identify areas targeted for mop-up. Although the verification of states' microplan increased the operational target population by 11.2%, the process aided the distribution of resources as appropriate. Maps showing the likely areas that needed additional effort to achieve required coverage with recommendation on the necessary approach to be deployed were transmitted to the states implementing the campaign.

CONCLUSION

The improvement in the use of data to guide implementation of the Nigeria 2017/2018 measles vaccination campaign caused an increase in the number of states that achieved higher coverage in the post-campaign coverage survey.

Ethiopian Pediatric Society Quality Improvement Initiative: a pragmatic approach to facility-based quality improvement in low-resource settings

Objectives

To describe critical features of the Ethiopian Pediatric Society (EPS) Quality Improvement (QI) Initiative and to present formative research on mentor models.

Setting

General and referral hospitals in the Addis Ababa area of Ethiopia.

Participants

Eighteen hospitals selected for proximity to the EPS headquarters, prior participation in a recent newborn care training cascade and minimal experience with QI.

Interventions

Education in QI in a 2-hour workshop setting followed by implementation of a facility-based QI project with the support of virtual mentorship or in-person mentorship.

Primary and secondary outcome measures

Primary outcome—QI progress, measured using an adapted Institute for Healthcare Improvement Scale; secondary outcome—contextual factors affecting QI success as measured by the Model for Understanding Success in Quality.

Results

The dose and nature of mentoring encounters differed based on a virtual versus in-person mentoring approach. All QI teams conducted at least one large-scale change. Education of staff was the most common change implemented in both groups. We did not identify contextual factors that predicted greater QI progress.

Conclusions

The EPS QI Initiative demonstrates that education in QI paired with external mentorship can support implementation of QI in low-resource settings. This pragmatic approach to facility-based QI may be a scalable strategy for improving newborn care and outcomes. Further research is needed on the most appropriate instruments for measuring contextual factors in low/middle-income country settings.

Driving the usage of tuberculosis diagnostic data through capacity building in low- and middle-income countries

BACKGROUND

Connectivity platforms collect a wealth of data from connected GeneXpert instruments, with the potential to provide valuable insights into the burden of disease and effectiveness of tuberculosis programmes. The challenge faced by many countries is a lack of training, analytical skills, and resources required to understand and translate this data into patient management and programme improvement.

OBJECTIVE

We describe a novel training programme, the tuberculosis Data Fellowship, designed to build capacity in low- and middle- income countries for tuberculosis data analytics.

METHODS

The programme consisted of classroom and remote training plus mentorship over a 12-month period. The focus was on skills development in Tableau software, followed by training in exploration, analysis, and interpretation of GeneXpert tuberculosis data across five key programme areas: patient services, programme monitoring, quality of testing, inventory management, and disease burden.

RESULTS

The programme was piloted in six countries (Bangladesh, Ethiopia, Ghana, Malawi, Mozambique) in July 2018 and Nigeria in September 2018; 20 participants completed the training. A number of key outputs have been achieved, such as improved instrument utilisation rates, decreased error rates, and improved instrument management.

CONCLUSION

The training programme empowers local tuberculosis programme staff to discover and fix critical inefficiencies, provides high-level technical and operational support to the tuberculosis programme, and provides a platform for continued sharing of insights and best practices between countries. It supports the notion that connectivity can increase efficiencies and clinical benefits with better data for decision making, if coupled with commensurate capacity building in data analysis and interpretation.

Improving the quality of routine maternal and newborn data captured in primary health facilities in Gombe State, Northeastern Nigeria: a before-and-after study

OBJECTIVES

Primary objective: to assess nine data quality metrics for 14 maternal and newborn health data elements, following implementation of an integrated, district-focused data quality intervention.

SECONDARY OBJECTIVE

to consider whether assessing the data quality metrics beyond completeness and accuracy of facility reporting offered new insight into reviewing routine data quality.

DESIGN

Before-and-after study design.

SETTING

Primary health facilities in Gombe State, Northeastern Nigeria.

PARTICIPANTS

Monitoring and evaluation officers and maternal, newborn and child health coordinators for state-level and all 11 local government areas (district-equivalent) overseeing 492 primary care facilities offering maternal and newborn care services.

INTERVENTION

Between April 2017 and December 2018, we implemented an integrated data quality intervention which included: introduction of job aids and regular self-assessment of data quality, peer-review and feedback, learning workshops, work planning for improvement, and ongoing support through social media.

OUTCOME MEASURES

9 metrics for the data quality dimensions of completeness and timeliness, internal consistency of reported data, and external consistency.

RESULTS

The data quality intervention was associated with improvements in seven of nine data quality metrics assessed including availability and timeliness of reporting, completeness of data elements, accuracy of facility reporting, consistency between related data elements, and frequency of outliers reported. Improvement differed by data element type, with content of care and commodity-related data improving more than contact-related data. Increases in the consistency between related data elements demonstrated improved internal consistency within and across facility documentation.

CONCLUSIONS

An integrated district-focused data quality intervention-including regular self-assessment of data quality, peer-review and feedback, learning workshops, work planning for improvement, and ongoing support through social media-can increase the completeness, accuracy and internal consistency of facility-based routine data.

District Governance and Improved Maternal, Neonatal and Child Health in South Africa: Pathways of Change

District-level initiatives to improve maternal, neonatal and child health (MNCH) generally do not take governance as their primary lens on health system strengthening. This paper is a case study of a district and sub-district governance mechanism, the Monitoring and Response Unit (MRU), which aimed to improve MNCH outcomes in two districts of South Africa. The MRU was introduced as a decision-making and accountability structure, and constituted of a "triangle" of managers, clinicians and information officers. An independent evaluation of the MRU initiative was conducted, three years after establishment, involving interviews with 89 district actors. Interviewees reported extensive changes in the scope, quality and organization of MNCH services, attributing these to the introduction of the MRU and enhanced support from district clinicians. We describe both the formal and informal aspects of the MRU as a governance mechanism, and then consider the pathways through which the MRU plausibly acted as a catalyst for change, using the institutional constructs of credible commitment, coordination and cooperation. In particular, the MRU promoted the formation of non-hierarchical collaborative networks; improved coordination between community, PHC and hospital services; and shaped collective sense-making in positive ways. We conclude that innovations in governance could add significant value to the district health system strengthening for improved MNCH. However, this requires a shift in focus from strengthening the front-line of service delivery, to change at the meso-level of sub-district and district decision-making; and from purely technical, data-driven to more holistic approaches that engage collective mindsets, widen participation in decision-making and nurture political leadership skills.

Harnessing the health systems strengthening potential of quality improvement using realist evaluation: an example from southern Tanzania

Quality improvement (QI) is a problem-solving approach in which stakeholders identify context-specific problems and create and implement strategies to address these. It is an approach that is increasingly used to support health system strengthening, which is widely promoted in Sub-Saharan Africa. However, few QI initiatives are sustained and implementation is poorly understood. Here, we propose realist evaluation to fill this gap, sharing an example from southern Tanzania. We use realist evaluation to generate insights around the mechanisms driving QI implementation. These insights can be harnessed to maximize capacity strengthening in QI and to support its operationalization, thus contributing to health systems strengthening. Realist evaluation begins by establishing an initial programme theory, which is presented here. We generated this through an elicitation approach, in which multiple sources (theoretical literature, a document review and previous project reports) were collated and analysed retroductively to generate hypotheses about how the QI intervention is expected to produce specific outcomes linked to implementation. These were organized by health systems building blocks to show how each block may be strengthened through QI processes. Our initial programme theory draws from empowerment theory and emphasizes the self-reinforcing nature of QI: the more it is implemented, the more improvements result, further empowering people to use it. We identified that opportunities that support skill- and confidence-strengthening are essential to optimizing QI, and thus, to maximizing health systems strengthening through QI. Realist evaluation can be used to generate rich implementation data for QI, showcasing how it can be supported in ‘real-world’ conditions for health systems strengthening.

Improving the quality and use of immunization and surveillance data: Summary report of the Working Group of the Strategic Advisory Group of Experts on Immunization

Concerns about the quality and use of immunization and vaccine-preventable disease (VPD) surveillance data have been highlighted on the global agenda for over two decades. In August 2017, the Strategic Advisory Group of Experts (SAGE) established a Working Group (WG) onthe Quality and Use of Global Immunization and Surveillance Data to review the current status and evidence to make recommendations, which were presented to SAGE in October 2019. The WG synthesized evidence from landscape analyses, literature reviews, country case-studies, a data triangulation analysis, as well as surveys of experts. Data quality (DQ) was defined as data that are accurate, precise, relevant, complete, and timely enough for the intended purpose (fit-for-purpose), and data use as the degree to which data are actually used for defined purposes, e.g., immunization programme management, performance monitoring, decision-making. The WG outlined roles and responsibilities for immunization and surveillance DQ and use by programme level. The WG found that while DQ is dependent on quality data collection at health facilities, many interventions have targeted national and subnational levels, or have focused on new technologies, rather than the people and enabling environments required for functional information systems. The WG concluded that sustainable improvements in immunization and surveillance DQ and use will require efforts across the health system - governance, people, tools, and processes, including use of data for continuous quality improvement (CQI) - and that the approaches need to be context-specific, country-owned and driven from the frontline up. At the country level, major efforts are needed to: (1) embed monitoring DQ and use alongside monitoring of immunization and surveillance performance, (2) increase workforce capacity and capability for DQ and use, starting at the facility level, (3) improve the accuracy of immunization programme targets (denominators), (4) enhance use of existing data for tailored programme action (e.g., immunization programme planning, management and policy-change), (5) adopt a data-driven CQI approach as part of health system strengthening, (6) strengthen governance around piloting and implementation of new information and communication technology tools, and (7) improve data sharing and knowledge management across areas and organizations for improved transparency and efficiency. Global and regional partners are requested to support countries in adopting relevant recommendations for their setting and to continue strengthening the reporting and monitoring of immunization and VPD surveillance data through processes periodic needs assessment and revision processes. This summary of the WG's findings and recommendations can support "data-guided" implementation of the new Immunization Agenda 2030.

Improving quality and use of routine health information system data in low- and middle-income countries: A scoping review

BACKGROUND

A routine health information system is one of the essential components of a health system. Interventions to improve routine health information system data quality and use for decision-making in low- and middle-income countries differ in design, methods, and scope. There have been limited efforts to synthesise the knowledge across the currently available intervention studies. Thus, this scoping review synthesised published results from interventions that aimed at improving data quality and use in routine health information systems in low- and middle-income countries.

METHOD

We included articles on intervention studies that aimed to improve data quality and use within routine health information systems in low- and middle-income countries, published in English from January 2008 to February 2020. We searched the literature in the databases Medline/PubMed, Web of Science, Embase, and Global Health. After a meticulous screening, we identified 20 articles on data quality and 16 on data use. We prepared and presented the results as a narrative.

RESULTS

Most of the studies were from Sub-Saharan Africa and designed as case studies. Interventions enhancing the quality of data targeted health facilities and staff within districts, and district health managers for improved data use. Combinations of technology enhancement along with capacity building activities, and data quality assessment and feedback system were found useful in improving data quality. Interventions facilitating data availability combined with technology enhancement increased the use of data for planning.

CONCLUSION

The studies in this scoping review showed that a combination of interventions, addressing both behavioural and technical factors, improved data quality and use. Interventions addressing organisational factors were non-existent, but these factors were reported to pose challenges to the implementation and performance of reported interventions.

Developing a Common Understanding of Networks of Care through a Scoping Study

The phrase "Networks of Care" seems familiar but remains poorly defined. A health system that exemplifies effective Networks of Care (NOC) purposefully and effectively interconnects service delivery touch points within a catchment area to fill critical service gaps and create continuity in patient care. To more fully elaborate the concept of Networks of Care, we conducted a multi-method scoping study that included a literature review, stakeholder interviews, and descriptive case studies from five low- and middle-income countries. Our extended definition of a Network of Care features four overlapping and interdependent domains of activity at multiple levels of health systems, characterized by: 1) Agreement and Enabling Environment, 2) Operational Standards, 3) Quality, Efficiency and Responsibility, and 4) Learning and Adaptation. There are a series of key interrelated themes within each domain. Creating a common understanding of what characterizes and fosters an effective Network of Care can drive the evolution and strengthening of national health programs, especially those incorporating universal health coverage and promoting comprehensive care and integrated services. An understanding of the Networks of Care model can help guide efforts to move health service delivery toward goals that can benefit a diversity of stakeholders, including a variety of health system actors, such as health care workers, users of health systems, and the wider community at large. It can also contribute to improving poor health outcomes and reducing waste originating from fragmented services and lack of access.

The All Babies Count Initiative: Impact of a Health System Improvement Approach on Neonatal Care and Outcomes in Rwanda

A health system improvement program combining facility readiness support, clinical training/mentoring, and improvement collaboratives increased quality improvement capacity, improved maternal and newborn quality of care, and reduced neonatal mortality. These results can be used to inform system improvement approach design to transform quality of care and outcomes for newborns.

Introduction:

Poor-quality care contributes to a significant portion of neonatal deaths globally. The All Babies Count (ABC) initiative was an 18-month district-wide approach designed to improve clinical and system performance across 2 rural Rwandan districts.

Methods:

This pre-post intervention study measured change in maternal and newborn health (MNH) quality of care and neonatal mortality. Data from the facility and community health management information system and newly introduced indicators were extracted from facility registers. Medians and interquartile ranges were calculated for the health facility to assess changes over time, and a mixed-effects logistic regression model was created for neonatal mortality. A difference-in-differences analysis was conducted to compare the change in district neonatal mortality with the rest of rural Rwanda.

Results:

Improvements were seen in multiple measures of facility readiness and MNH quality of care, including antenatal care coverage, preterm labor management, and postnatal care quality. District hospital case fatality decreased, with a statistically significant reduction in district neonatal mortality (odds ratio [OR]=0.54; 95% confidence interval [CI]=0.36, 0.83) and among preterm/low birth weight neonates (OR=0.47; 95% CI=0.25, 0.90). Neonatal mortality was reduced from 30.1 to 19.6 deaths/1,000 live births in the intervention districts and remained relatively stable in the rest of rural Rwanda (difference in differences −12.9).

Conclusion:

The ABC initiative contributed to improved MNH quality of care and outcomes in rural Rwanda. A combined clinical and health system improvement approach could be an effective strategy to improve quality and reduce neonatal mortality.

Factors That Influence Data Use to Improve Health Service Delivery in Low- and Middle-Income Countries

BACKGROUND

Health service delivery indicators are designed to reveal how well health services meet a community's needs. Effective use of the data can enable targeted improvements in health service delivery. We conducted a systematic review to identify the factors that influence the use of health service delivery indicators to improve delivery of primary health care services in low- and middle-income settings.

METHODS

We reviewed empirical studies published in 2005 or later that provided evidence on the use of health service delivery data at the primary care level in low- and middle-income countries. We searched Scopus, Medline, the Cochrane Library, and citations of included studies. We also searched the gray literature, using a separate strategy. We extracted information on study design, setting, study population, study objective, key findings, and any identified lessons learned.

RESULTS

Twelve studies met the inclusion criteria. This small number of studies suggests there is insufficient evidence to draw reliable conclusions. However, a content analysis identified the following potentially influential factors, which we classified into 3 categories: governance (leadership, participatory monitoring, regular review of data); production of information (presentation of findings, data quality, qualitative data); and health information system resources (electronic health management information systems, organizational structure, training). Contextual factors and performance-based financing were also each found to have a role; however, discussing these as mediating factors may not be practical in terms of promoting data use.

CONCLUSION

Scant evidence exists regarding factors that influence the use of health service delivery indicators to improve delivery of primary health care services in low- and middle-income countries. However, the existing evidence highlights some factors that may have a role in improving data use. Further research may benefit from comparing data use factors across different types of program indicators or using our classification as a framework for field experiments.

Perceptions of key informants on the provision of cervical cancer prevention and control programme in Uganda: implication for cervical cancer policy

Background

Uganda has one of the highest burdens of cervical cancer globally. In 2010 the Ugandan Ministry of Health launched the Strategic Plan for Cervical Cancer Prevention and Control with the hope of developing cervical cancer policy in Uganda. This study explored the beliefs of senior key informants in Uganda about cervical cancer prevention, the control programme, and the relevance of cervical cancer policy.

Methods

We conducted 15 key informant interviews with participants from six organisations across Northern and Central Uganda. Participants were drawn from district local government health departments, St. Mary’s Hospital Lacor, Uganda Nurses and Midwifery Council, non-governmental organisations (NGOs) and Ministry of Health in Kampala, Uganda. The interview recordings were transcribed and analysed using thematic analysis.

Results

Seven themes emerged relating to the cervical cancer prevention and control programmes in Uganda: (1) policy frameworks for cervical cancer, (2) operationalising cervical cancer prevention and control, (3) financial allocation and alignment, (4) human resources and capability, (5) essential supplies and vaccines, (6) administrative data and resource distribution, and (7) cervical cancer services.

Conclusions

The key informants perceive that the lack of a cervical cancer policy in Uganda is hindering cervical cancer prevention and control programmes. Therefore, the Ministry of Health and stakeholders need to work together in coming up with an effective policy framework that will accelerate efforts towards cervical cancer prevention and control in Uganda.

Routine Health Information System (RHIS) improvements for strengthened health system management

BACKGROUND

A well-functioning routine health information system (RHIS) can provide the information needed for health system management, for governance, accountability, planning, policy making, surveillance and quality improvement, but poor information support has been identified as a major obstacle for improving health system management.

OBJECTIVES

To assess the effects of interventions to improve routine health information systems in terms of RHIS performance, and also, in terms of improved health system management performance, and improved patient and population health outcomes.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library, MEDLINE Ovid and Embase Ovid in May 2019. We searched Global Health, Ovid and PsycInfo in April 2016. In January 2020 we searched for grey literature in the Grey Literature Report and in OpenGrey, and for ongoing trials using the International Clinical Trials Registry Platform (ICTRP) and ClinicalTrials.gov. In October 2019 we also did a cited reference search using Web of Science, and a 'similar articles' search in PubMed.

SELECTION CRITERIA

Randomised and non-randomised trials, controlled before-after studies and time-series studies comparing routine health information system interventions, with controls, in primary, hospital or community health care settings. Participants included clinical staff and management, district management and community health workers using routine information systems.

DATA COLLECTION AND ANALYSIS

Two authors independently reviewed records to identify studies for inclusion, extracted data from the included studies and assessed the risk of bias. Interventions and outcomes were too varied across studies to allow for pooled risk analysis. We present a 'Summary of findings' table for each intervention comparisons broadly categorised into Technical and Organisational (or a combination), and report outcomes on data quality and service quality. We used the GRADE approach to assess the certainty of the evidence.

MAIN RESULTS

We included six studies: four cluster randomised trials and two controlled before-after studies, from Africa and South America. Three studies evaluated technical interventions, one study evaluated an organisational intervention, and two studies evaluated a combination of technical and organisational interventions. Four studies reported on data quality and six studies reported on service quality. In terms of data quality, a web-based electronic TB laboratory information system probably reduces the length of time to reporting of TB test results, and probably reduces the overall rate of recording errors of TB test results, compared to a paper-based system (moderate certainty evidence). We are uncertain about the effect of the electronic laboratory information system on the recording rate of serious (misidentification) errors for TB test results compared to a paper-based system (very low certainty evidence). Misidentification errors are inaccuracies in transferring test results between an electronic register and patients' clinical charts. We are also uncertain about the effect of the intervention on service quality (timeliness of starting or changing a patient's TB treatment) (very low certainty evidence). A hand-held electronic device probably improves the length of time to report TB test results, and probably reduces the total frequency of recording errors in TB test results between the laboratory notebook and the electronic information record system, compared to a paper-based system (moderate-certainty evidence). We are, however, uncertain about the effect of the intervention on the frequency of serious (misidentification) errors in recording between the laboratory notebook and the electronic information record, compared to a paper-based system (very low certainty evidence). We are uncertain about the effect of a hospital electronic health information system on service quality (length of time outpatients spend at hospital, length of hospital stay, and hospital revenue collection), compared to a paper-based system (very low certainty evidence). High-intensity brief text messaging (SMS) may make little or no difference to data quality (in terms of completeness of documentation of pregnancy outcomes), compared to low-intensity brief text messaging (low-certainty evidence). We are uncertain about the effect of electronic drug stock notification (with either data management support or product transfer support) on service quality (in terms of transporting stock and stock levels), compared to paper-based stock notification (very low certainty evidence). We are uncertain about the effect of health information strengthening (where it is part of comprehensive service quality improvement intervention) on service quality (health worker motivation, receipt of training by health workers, health information index scores, quality of clinical observation of children and adults) (very low certainty evidence).

AUTHORS' CONCLUSIONS

The review indicates mixed effects of mainly technical interventions to improve data quality, with gaps in evidence on interventions aimed at enhancing data-informed health system management. There is a gap in interventions studying information support beyond clinical management, such as for human resources, finances, drug supply and governance. We need to have a better understanding of the causal mechanisms by which information support may affect change in management decision-making, to inform robust intervention design and evaluation methods.

A, Salim N, Zaman SB, Shamba D, Blencowe H, Ruysen H, El Arifeen S, Boggs D, Gordeev VS, Rahman QSU, Hossain T, Joshi E, Thapa S, Poudel RP, Poudel D, Chaudhary P, Karki R, Chitrakar B, Mkopi N, Wisiko A, Kitende AP, Shirati MR, Chingalo C, Semhando AO, Mtei C, Mwenisongole V, Bakuza JM, Kombo J, Mbaruku G, Lawn JE. Labour and delivery ward register data availability, quality, and utility - Every Newborn - birth indicators research tracking in hospitals (EN-BIRTH) study baseline analysis in three countries

BACKGROUND

Countries with the highest burden of maternal and newborn deaths and stillbirths often have little information on these deaths. Since over 81% of births worldwide now occur in facilities, using routine facility data could reduce this data gap. We assessed the availability, quality, and utility of routine labour and delivery ward register data in five hospitals in Bangladesh, Nepal, and Tanzania. This paper forms the baseline register assessment for the Every Newborn-Birth Indicators Research Tracking in Hospitals (EN-BIRTH) study.

METHODS

We extracted 21 data elements from routine hospital labour ward registers, useful to calculate selected maternal and newborn health (MNH) indicators. The study sites were five public hospitals during a one-year period (2016-17). We measured 1) availability: completeness of data elements by register design, 2) data quality: implausibility, internal consistency, and heaping of birthweight and explored 3) utility by calculating selected MNH indicators using the available data.

RESULTS

Data were extracted for 20,075 births. Register design was different between the five hospitals with 10-17 of the 21 selected MNH data elements available. More data were available for health outcomes than interventions. Nearly all available data elements were > 95% complete in four of the five hospitals and implausible values were rare. Data elements captured in specific columns were 85.2% highly complete compared to 25.0% captured in non-specific columns. Birthweight data were less complete for stillbirths than live births at two hospitals, and significant heaping was found in all sites, especially at 2500g and 3000g. All five hospitals recorded count data required to calculate impact indicators including; stillbirth rate, low birthweight rate, Caesarean section rate, and mortality rates.

CONCLUSIONS

Data needed to calculate MNH indicators are mostly available and highly complete in EN-BIRTH study hospital routine labour ward registers in Bangladesh, Nepal and Tanzania. Register designs need to include interventions for coverage measurement. There is potential to improve data quality if Health Management Information Systems utilization with feedback loops can be strengthened. Routine health facility data could contribute to reduce the coverage and impact data gap around the time of birth.