Improving data quality across 3 sub-Saharan African countries using the Consolidated Framework for Implementation Research (CFIR): results from the African Health Initiative

Embedding research capacity strengthening in multi-country studies in low-and middle-income countries: learnings from sexual and reproductive health research

Research capacity strengthening (RCS) can empower individuals, institutions, networks, or countries to define and prioritize problems systematically; develop and scientifically evaluate appropriate solutions; and reinforce or improve capacities to translate knowledge into policy and practice. However, how to embed RCS into multi-country studies focusing on sexual and reproductive health and rights (SRHR) is largely undocumented. We used findings from a qualitative study, from a review of the literature, and from a validation exercise from a panel of experts from research institutions that work on SRHR RCS. We provide a framework for embedded RCS; suggest a set of seven concrete actions that research project planners, designers, implementers, and funders can utilise to guide embedded RCS activities in low- and middle-income countries; and present a practical checklist for planning and assessing embedded RCS in research projects.

What could improve surgical data system at health facilities with high surgical volume in Ethiopia?

BACKGROUND

The effective management of surgical and anesthesia care relies on quality data and its readily availability for both patient-centered decision-making and facility-level improvement efforts. Recognizing this critical need, the Strengthening Systems for Improved Surgical Outcomes (SSISO) project addressed surgical care data management and information use practices across 23 health facilities from October 2019 to September 2022. This study aimed to evaluate the effectiveness of SSISO interventions in enhancing practices related to surgical data capture, reporting, analysis, and visualization.

METHODS

This study employed a mixed method, pre- post intervention evaluation design to assess changes in data management and utilization practices at intervention facilities. The intervention packages included capacity building trainings, monthly mentorship visits facilitated by a hub-and-spoke approach, provision of data capture tools, and reinforcement of performance review teams. Data collection occurred at baseline (February - April 2020) and endline (April - June 2022). The evaluation focused on the availability and appropriate use of data capture tools, as well as changes in performance review practices. Appropriate use of registers was defined as filling all the necessary data onto the registers, and this was verified by completeness of selected key data elements in the registers.

RESULTS

The proportion of health facilities with Operation Room (OR) scheduling, referral, and surgical site infection registers significantly increased by 34.8%, 56.5% and 87%, respectively, at project endline compared to baseline. Availability of OR and Anesthesia registers remained high throughout the project, at 91.3% and 95.6%, respectively. Furthermore, the appropriate use of these registers improved, with statistically significant increases observed for OR scheduling registers (34.8% increase). Increases were also noted for OR register (9.5% increase) and anesthesia register (4.5% increase), although not statistically significant. Assessing the prior three months reports, the report submissions to the Ministry of Health/Regional Health Bureau (MOH/RHB) rose from 85 to 100%, reflecting complete reporting at endline period. Additionally, the proportion of surgical teams analyzing and displaying data for informed decision-making significantly increased from 30.4% at baseline to 60.8% at endline period.

CONCLUSION

The implemented interventions positively impacted surgical data management and utilization practice at intervention facilities. These positive changes were likely attributable to capacity building trainings and regular mentorship visits via hub-and-spoke approach. Hence, we recommend further investigation into the effectiveness of similar intervention packages in improving surgical data management, data analysis and visualization practices in low- and middle-income country settings.

Evaluation of the sentinel yellow fever surveillance system in Uganda, 2017-2022: strengths and weaknesses

BACKGROUND

Uganda has a sentinel surveillance system in seven high-risk sites to monitor yellow fever (YF) patterns and detect outbreaks. We evaluated the performance of this system from 2017 to 2022.

METHODS

We evaluated selected attributes, including timeliness (lags between different critical time points), external completeness (proportion of expected sentinel sites reporting ≥ 1 suspect case in the system annually), and internal completeness (proportion of reports with the minimum required data elements filled), using secondary data in the YF surveillance database from January 2017-July 2022. We conducted key informant interviews with stakeholders at health facility and national level to assess usefulness, flexibility, simplicity, and acceptability of the surveillance system.

RESULTS

In total, 3,073 suspected and 15 confirmed YF cases were reported. The median time lag from sample collection to laboratory shipment was 37 days (IQR:21-54). External completeness was 76%; internal completeness was 65%. Stakeholders felt that the surveillance system was simple and acceptable, but were uncertain about flexibility. Most (71%) YF cases in previous outbreaks were detected through the sentinel surveillance system; data were used to inform interventions such as intensified YF vaccination.

CONCLUSION

The YF sentinel surveillance system was useful in detecting outbreaks and informing public health action. Delays in case confirmation and incomplete data compromised its overall effectiveness and efficiency.

Introducing a standardised register for strengthening the inpatient management of newborns and sick children: Implementation research in selected health facilities of Bangladesh

Background

It is imperative to maintain accurate documentation of clinical interventions aimed at enhancing the quality of care for newborns and sick children. The National Newborn Health and IMCI programme of Bangladesh led the development of a standardised register for managing newborns and sick children under five years of age during inpatient care through stakeholder engagement. We aimed to assess the implementation outcomes of the standardised register in the inpatient department.

Methods

We conducted implementation research in two district hospitals and two sub-district hospitals of Kushtia and Dinajpur districts from November 2022 to January 2023 to assess the implementation outcomes of the standardised register. We assessed the following World Health Organization implementation outcome variables: usability, acceptability, adoption (actual use), fidelity (completeness and accuracy), and utility (quality of care) of the register against preset benchmarks. We collected data through structured interviews with health care providers; participant enrolment; and data extraction from inpatient registers and case record forms.

Results

The average usability and acceptability scores among health care providers were 73 (standard deviation (SD) = 14) and 82 (SD = 14) out of 100, respectively. The inpatient register recorded 96% (95% confidence interval (CI) = 95-97) of under-five children who were admitted to the inpatient department (adoption - actual use). The proportions of completed data elements in the inpatient register were above the preset benchmark of 70% for all the assessed data elements except 'investigation done' (24%; 95% CI = 23-26) (fidelity - completeness). The percentage agreements between government-appointed nurses posted and study-appointed nurses were above the preset benchmark of 70% for all the reported variables (fidelity - accuracy). The kappa coefficient for the overall level of agreement between these two groups regarding reported variables indicated moderate to substantial agreement. The proportion of newborns with sepsis receiving injectable antibiotics was 62% (95% CI = 47-75) (utility - quality of care). We observed some variability in the completeness and accuracy of the inpatient register by district and facility type.

Conclusions

The inpatient register was positively received by health care providers, with evaluations of implementation outcome variables showing encouraging results. Our findings could inform evidence-based decision-making on the implementation and scale-up of the inpatient register in Bangladesh, as well as other low- and middle-income countries.

Reducing the equity gap in under-5 mortality through an innovative community health program in Ethiopia: an implementation research study

BACKGROUND

The Ethiopian government implemented a national community health program, the Health Extension Program (HEP), to provide community-based health services to address persisting access-related barriers to care using health extension workers (HEWs). We used implementation research to understand how Ethiopia leveraged the HEP to widely implement evidence-based interventions (EBIs) known to reduce under-5 mortality (U5M) and address health inequities.

METHODS

This study was part of a six-country case study series using implementation research to understand how countries implemented EBIs between 2000-2015. Our mixed-methods research was informed by a hybrid implementation science framework using desk review of published and gray literature, analysis of existing data sources, and 11 key informant interviews. We used implementation of pneumococcal conjugate vaccine (PCV-10) and integrated community case management (iCCM) to illustrate Ethiopia's ability to rapidly integrate interventions into existing systems at a national level through leveraging the HEP and other implementation strategies and contextual factors which influenced implementation outcomes.

RESULTS

Ethiopia implemented numerous EBIs known to address leading causes of U5M, leveraging the HEP as a platform for delivery to successfully introduce and scale new EBIs nationally. By 2014/15, estimated coverage of three doses of PCV-10 was at 76%, with high acceptability (nearly 100%) of vaccines in the community. Between 2000 and 2015, we found evidence of improved care-seeking; coverage of oral rehydration solution for treatment of diarrhea, a service included in iCCM, doubled over this period. HEWs made health services more accessible to rural and pastoralist communities, which account for over 80% of the population, with previously low access, a contextual factor that had been a barrier to high coverage of interventions.

CONCLUSIONS

Leveraging the HEP as a platform for service delivery allowed Ethiopia to successfully introduce and scale existing and new EBIs nationally, improving feasibility and reach of introduction and scale-up of interventions. Additional efforts are required to reduce the equity gap in coverage of EBIs including PCV-10 and iCCM among pastoralist and rural communities. As other countries continue to work towards reducing U5M, Ethiopia's experience provides important lessons in effectively delivering key EBIs in the presence of challenging contextual factors.

Lessons from Peru to reduce under-5 mortality: understanding program implementation and context

BACKGROUND

Health policymakers aiming to reduce under-5 mortality (U5M) often lack data regarding how successful interventions in other countries were implemented. The Exemplars in U5M Study identified countries that achieved significant reductions in amenable U5M. This case study in Peru used implementation research to explore the contextual factors and strategies that contributed to the successful implementation of key evidence-based interventions (EBIs).

METHODS

This research utilized a hybrid implementation research framework and a mixed-methods approach to understand the factors associated with EBI implementation and the successful reduction of U5M between 2000-2015. A desk review of existing literature on EBIs and U5M in Peru was completed, and in-depth interviews were performed with key Peruvian informants to understand the implementation strategies employed and the contextual factors that facilitated or were barriers to success. For the purposes of this analysis, three EBIs were selected and evaluated: antenatal care visits (ANC), facility-based deliveries, and infant vaccination.

RESULTS

Between 2000-2015, the percent of mothers attending at least four antenatal care visits rose from 69% to 96.9%, and the percent of facility-based deliveries increased from 56 to 91%. Three doses of the tetanus/diphtheria/pertussis vaccine, widely acknowledged as a key global health indicator, reached 90% by 2015. Key informants noted that economic growth, financial reforms, strong national commitment to reduce poverty in Peru, and national prioritization of maternal and child health, were important contextual factors that contributed to the successful reduction of U5M. They noted key strategies that helped achieve success during the implementation of EBIs, including utilization of data for decision-making, adaptation driven by cultural sensitivity to address gaps in coverage, and a focus on equity and anti-poverty initiatives with the participation of government, civil society, and political parties to assure continuity of policies.

CONCLUSION

Several EBIs contributed to the successful reduction of U5M in Peru between 2000-2015. Strategies such as the focus on equity throughout the study period contributed to an increase in coverage of EBIs like ANC visits, facility-based deliveries and infant vaccination which worked to reduce U5M. Understanding how Peru successfully implemented programs that reduced preventable infant and child deaths could be useful to replicating this substantial public health success in other low- and middle-income countries.

Mobilizing faith-based COVID-19 health ambassadors to address COVID-19 health disparities among African American older adults in under-resourced communities: A hybrid, community-based participatory intervention

INTRODUCTION

The COVID-19 pandemic disproportionately affected older adults, particularly those with pre-existing chronic health conditions. To address the health disparity and challenges faced by under-resourced African American older adults in South Los Angeles during this period, we implemented a hybrid (virtual/in-person), pre-post, community-based participatory intervention research project utilizing a faith-based lay health advisor model (COVID-19 Health Ambassador Program (CHAP)). We recruited COVID-19 Health Ambassadors (CHAs) and African American older adults (participants) from faith-based organizations who partook in CHA-led meetings and follow-ups that educated and supported the participants. This paper seeks to evaluate this intervention's implementation using the Consolidated Framework for Implementation Research (CFIR) as a reporting tool with an emphasis on fidelity, challenges, and adaptations based on data collected via stakeholder interviews and surveys.

RESULTS

CHAP was delivered to 152 participants by 19 CHAs from 17 faith-based organizations. CHAs assisted with chronic disease management, resolved medication-related challenges, encouraged COVID-19 vaccination, reduced psychological stress and addressed healthcare avoidance behaviors such as COVID-19 vaccine hesitancy among the participants. Challenges encountered include ensuring participant engagement and retention in the virtual format and addressing technological barriers for CHAs and participants. Adaptations made to better suit the needs of participants included providing communication tools and additional training to CHAs to improve their proficiency in using virtual platforms in addition to adapting scientific/educational materials to suit our participants' diverse cultural and linguistic needs.

CONCLUSION

The community-centered hybrid approach in addition to our partnership with faith-based organizations and their respective COVID-19 health ambassadors proved to be essential in assisting underserved African American older adults manage chronic health conditions and address community-wide health disparities during the COVID-19 pandemic. Adaptability, cultural sensitivity, and teamwork are key to implementing health interventions especially in underserved populations.

Assessing drivers of implementing "Scaling-up the Systems Analysis and Improvement Approach" for Prevention of Mother-to-Child HIV Transmission in Mozambique (SAIA-SCALE) over implementation waves

BACKGROUND

The Systems Analysis and Improvement Approach (SAIA) is an evidence-based package of systems engineering tools originally designed to improve patient flow through the prevention of Mother-to-Child transmission of HIV (PMTCT) cascade. SAIA is a potentially scalable model for maximizing the benefits of universal antiretroviral therapy (ART) for mothers and their babies. SAIA-SCALE was a stepped wedge trial implemented in Manica Province, Mozambique, to evaluate SAIA's effectiveness when led by district health managers, rather than by study nurses. We present the results of a qualitative assessment of implementation determinants of the SAIA-SCALE strategy during two intensive and one maintenance phases.

METHODS

We used an extended case study design that embedded the Consolidated Framework for Implementation Research (CFIR) to guide data collection, analysis, and interpretation. From March 2019 to April 2020, we conducted in-depth individual interviews (IDIs) and focus group discussions (FGDs) with district managers, health facility maternal and child health (MCH) managers, and frontline nurses at 21 health facilities and seven districts of Manica Province (Chimoio, Báruè, Gondola, Macate, Manica, Sussundenga, and Vanduzi).

RESULTS

We included 85 participants: 50 through IDIs and 35 from three FGDs. Most study participants were women (98%), frontline nurses (49.4%), and MCH health facility managers (32.5%). An identified facilitator of successful intervention implementation (regardless of intervention phase) was related to SAIA's compatibility with organizational structures, processes, and priorities of Mozambique's health system at the district and health facility levels. Identified barriers to successful implementation included (a) inadequate health facility and road infrastructure preventing mothers from accessing MCH/PMTCT services at study health facilities and preventing nurses from dedicating time to improving service provision, and (b) challenges in managing intervention funds.

CONCLUSIONS

The SAIA-SCALE qualitative evaluation suggests that the scalability of SAIA for PMTCT is enhanced by its fit within organizational structures, processes, and priorities at the primary level of healthcare delivery and health system management in Mozambique. Barriers to implementation that impact the scalability of SAIA include district-level financial management capabilities and lack of infrastructure at the health facility level. SAIA cannot be successfully scaled up to adequately address PMTCT needs without leveraging central-level resources and priorities.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03425136 . Registered on 02/06/2018.

Barriers and Facilitators to Data Use for Decision Making: The Experience of the African Health Initiative Partnerships in Ethiopia, Ghana, and Mozambique

BACKGROUND

Three African Health Initiative (AHI) partnership projects in Ethiopia, Ghana, and Mozambique implemented strategies to improve the quality and evaluation of routinely collected data at the primary health care level and stimulate its use in evidence-based decision making. We compare how these programs designed and carried out data for decision-making (DDM) strategies, elaborate on barriers and facilitators to implementation success, and offer recommendations for future DDM programming.

METHODS

Researchers from each project collaboratively wrote a cross-country protocol based on these objectives. By adapting the Consolidated Framework for Implementation Research (CFIR) through a qualitative theme reduction process, they harmonized lines of inquiry on the design of the respective DDM strategies and the barriers and facilitators of effective implementation. We conducted in-depth interviews and focus group discussions with stakeholders from the primary health care systems in each country, and we carried out multistage, thematic analyses using a deductive lens.

RESULTS

Effective implementation of DDM depended on whether implementers felt that DDM was adaptable to context, feasible to trial, and easy to introduce and maintain. The prevailing policy and political environment in the wider health system, learning climate and absorptive capacity for evidence-based change in DDM settings, engagement of external change agents and internal change leaders, and promotion of opportunities and means for team-based reflection and evaluations of what works influenced the success or failure of DDM strategies.

CONCLUSION

Opportunities for team-based capacity building and individual mentorship led to effective DDM programming. External policies and associated incentives bolstered this but occasionally led to unintended consequences. Leadership engagement and availability of resources to act on recommendations; respond to capacity-building needs; and facilitate collaborations between peers, within hierarchies, and across the local health system proved crucial to DDM, as was encouraging adaptation and opportunities for iterative on-the-job learning.

Applying the Consolidated Framework for Implementation Research to Identify Implementation Determinants for the Integrated District Evidence-to-Action Program, Mozambique

INTRODUCTION

The Integrated District Evidence-to-Action program is an audit and feedback intervention introduced in 2017 in Manica and Sofala provinces, Mozambique, to reduce mortality in children younger than 5 years. We describe barriers and facilitators to early-stage effectiveness of that intervention.

METHOD

We embedded the Consolidated Framework for Implementation Research (CFIR) into an extended case study design to inform sampling, data collection, analysis, and interpretation. We collected data in 4 districts in Manica and Sofala Provinces in November 2018. Data collection included document review, 22 in-depth individual interviews, and 2 focus group discussions (FGDs) with 19 provincial, district, and facility managers and nurses. Most participants (70.2%) were nurses and facility managers and the majority were women (87.8%). We audio-recorded all but 2 interviews and FGDs and conducted a consensus-based iterative analysis.

RESULTS

Facilitators of effective intervention implementation included: implementation of the core intervention components of audit and feedback meetings, supportive supervision and mentorship, and small grants as originally planned; positive pressure from district managers and study nurses on health facility staff to strive for excellence; and easy access to knowledge and information about the intervention. Implementation barriers were the intervention's lack of compatibility in not addressing the scarcity of human and financial resources and inadequate infrastructures for maternal and child health services at district and facility levels and; the intervention's lack of adaptability in having little flexibility in the design and decision making about the use of intervention funds and data collection tools.

DISCUSSION

Our comprehensive and systematic use of the CFIR within an extended case study design generated granular evidence on CFIR's contribution to implementation science efforts to describe determinants of early-stage intervention implementation. It also provided baseline findings to assess subsequent implementation phases, considering similarities and differences in barriers and facilitators across study districts and facilities. Sharing preliminary findings with stakeholders promoted timely decision making about intervention implementation.

Learning Health Systems to Bridge the Evidence-Policy-Practice Gap in Primary Health Care: Lessons From the African Health Initiative

The compilation of lessons in this supplement on the Doris Duke Charitable Foundation’s African Health Initiative’s work in the application of implementation research in primary health care in sub-Saharan Africa reflects the evolution of the discipline that is now increasingly recognized as integral to health systems strengthening.

Did an Intervention Programme Aimed at Strengthening the Maternal and Child Health Services in Nigeria Improve the Completeness of Routine Health Data Within the Health Management Information System?

BACKGROUND

During 2012-2015, the Federal Government of Nigeria launched the Subsidy Reinvestment and Empowerment Programme, a health system strengthening (HSS) programme with a Maternal and Child Health component (Subsidy Reinvestment and Empowerment Programme [SURE-P]/MCH), which was monitored using the Health Management Information Systems (HMIS) data reporting tools. Good quality data is essential for health policy and planning decisions yet, little is known on whether and how broad health systems strengthening programmes affect quality of data. This paper explores the effects of the SURE-P/MCH on completeness of MCH data in the National HMIS.

METHODS

This mixed-methods study was undertaken in Anambra state, southeast Nigeria. A standardized proforma was used to collect facility-level data from the facility registers on MCH services to assess the completeness of data from 2 interventions and one control clusters. The facility data was collected to cover before, during, and after the SURE-P intervention activities. Qualitative in-depth interviews were conducted with purposefully-identified health facility workers to identify their views and experiences of changes in data quality throughout the above 3 periods.

RESULTS

Quantitative analysis of the facility data showed that data completeness improved substantially, starting before SURE-P and continuing during SURE-P but across all clusters (ie, including the control). Also health workers felt data completeness were improved during the SURE-P, but declined with the cessation of the programme. We also found that challenges to data completeness are dependent on many variables including a high burden on providers for data collection, many variables to be filled in the data collection tools, and lack of health worker incentives.

CONCLUSION

Quantitative analysis showed improved data completeness and health workers believed the SURE-P/MCH had contributed to the improvement. The functioning of national HMIS are inevitably linked with other health systems components. While health systems strengthening programmes have a great potential for improved overall systems performance, a more granular understanding of their implications on the specific components such as the resultant quality of HMIS data, is needed.

Creation of an Evidence-Based Implementation Framework for Digital Health Technology in the Intensive Care Unit: Qualitative Study

Background

Digital health technologies such as continuous remote monitoring and artificial intelligence–driven clinical decision support systems could improve clinical outcomes in intensive care medicine. However, comprehensive evidence and guidelines for the successful implementation of digital health technologies into specific clinical settings such as the intensive care unit (ICU) are scarce. We evaluated the implementation of a remote patient monitoring platform and derived a framework proposal for the implementation of digital health technology in an ICU.

Objective

This study aims to investigate barriers and facilitators to the implementation of a remote patient monitoring technology and to develop a proposal for an implementation framework for digital health technology in the ICU.

Methods

This study was conducted from May 2018 to March 2020 during the implementation of a tablet computer–based remote patient monitoring system. The system was installed in the ICU of a large German university hospital as a supplementary monitoring device. Following a hybrid qualitative approach with inductive and deductive elements, we used the Consolidated Framework for Implementation Research and the Expert Recommendations for Implementing Change to analyze the transcripts of 7 semistructured interviews with clinical ICU stakeholders and descriptive questionnaire data. The results of the qualitative analysis, together with the findings from informal meetings, field observations, and previous explorations, provided the basis for the derivation of the proposed framework.

Results

This study revealed an insufficient implementation process due to lack of staff engagement and few perceived benefits from the novel solution. Further implementation barriers were the high staff presence and monitoring coverage in the ICU. The implementation framework includes strategies to be applied before and during implementation, targeting the implementation setting by involving all ICU stakeholders, assessing the intervention’s adaptability, facilitating the implementation process, and maintaining a vital feedback culture. Setting up a unit responsible for implementation, considering the guidance of an implementation advisor, and building on existing institutional capacities could improve the institutional context of implementation projects in the ICU.

Conclusions

Implementation of digital health in the ICU should involve a thorough preimplementation assessment of the ICU’s need for innovation and its readiness to change, as well as an ongoing evaluation of the implementation conditions. Involvement of all stakeholders, transparent communication, and continuous feedback in an equal atmosphere are essential, but leadership roles must be clearly defined and competently filled. Our proposed framework may guide health care providers with concrete, evidence-based, and step-by-step recommendations for implementation practice, facilitating the introduction of digital health in intensive care.

Trial Registration

ClinicalTrials.gov NCT03514173; https://clinicaltrials.gov/ct2/show/NCT03514173

Utilisation of health management information and its determinant factors among health professionals working at public health facilities in North Wollo Zone, Northeast Ethiopia: a cross-sectional study

OBJECTIVE

The study aimed to assess health management information utilisation and associated factors among health professionals working at public health facilities in North Wollo Zone, Northeast Ethiopia.

SETTING

The study was conducted at public health facilities in the North Wollo Zone, Northeast Ethiopia.

PARTICIPANTS

A total of 664 (56.3% male and 43.7% female) health professionals participated in the study. All health professionals permanently working in North Wollo Zone were included in this study. However, health professionals who were not present during the data collection period by any means and who had less than 6 months of experience were not included in this study.

PRIMARY AND SECONDARY OUTCOME MEASURES

The main outcome measure was health management information utilisation.

RESULT

About 58.4% (n=388) (95% CI: 54.4% to 62.0%) of the study participants use health management information. The multivariable logistic regression model indicated that participants who had managerial positions are more likely to use health management information with an adjusted OR (AOR) of 3.11 and 95% CI 1.84 to 5.24. Similarly, having a good motivation level (AOR=4.42 (95% CI: 2.82 to 6.93)), perceived good culture of health information (AOR=6.17 (95% CI: 3.35 to 11.36)), a standard set of indicators (AOR=4.11 (95% CI: 2.65 to 6.38)), having good governance of health information system (AOR=1.75 (95% CI:1.13 to 2.72)) and health management information system (HMIS) training (AOR=3.10 (95% CI: 1.89 to 5.07)) were the predictors positively associated with higher utilisation of health management information.

CONCLUSION

This study revealed that utilisation of health management information was still inadequate. Enhancing motivation, building a culture of information use, having standardised indicators, strengthening the governance of health information systems and comprehensive HMIS training were measures to be taken to improve utilisation of health management information in this study setting.

Exploring data quality and use of the routine health information system in Ethiopia: a mixed-methods study

OBJECTIVE

A routine health information system (RHIS) enables decision making in the healthcare system. We aimed to analyse data quality at the district and regional level and explore factors and perceptions affecting the quality and use of routine data.

DESIGN

This was a mixed-methods study. We used the WHO toolkit for analysing data quality and interviewed staff at the point of data generation and along with the flow of data. Data were analysed using the Performance of Routine Information System Management framework.

SETTING

This study was performed in eight districts in four regions of Ethiopia. The study was nested within a 2-year programme of the Operational Research and Coaching for government Analysts.

PARTICIPANTS

We visited 45 health posts, 1 district hospital, 16 health centres and 8 district offices for analysis of routine RHIS data and interviewed 117 staff members for the qualitative assessment.

OUTCOME MEASURES

We assessed availability of source documents, completeness, timeliness and accuracy of reporting of routine data, and explored data quality and use perceptions.

RESULTS

There was variable quality of both indicator and data element. Data on maternal health and immunisation were of higher quality than data on child nutrition. Issues ranged from simple organisational factors, such as availability of register books, to intricate technical issues, like complexity of indicators and choice of denominators based on population estimates. Respondents showed knowledge of the reporting procedures, but also demonstrated limited skills, lack of supportive supervision and reporting to please the next level. We saw limited examples of the use of data by the staff who were responsible for data reporting.

CONCLUSION

We identified important organisational, technical, behavioural and process factors that need further attention to improve the quality and use of RHIS data in Ethiopia.

Development and application of a hybrid implementation research framework to understand success in reducing under-5 mortality in Rwanda

Background: We describe the development and testing of a hybrid implementation research (IR) framework to understand the pathways, successes, and challenges in addressing amenable under-5 mortality (U5M) - deaths preventable through health system-delivered evidence-based interventions (EBIs) - in low- and middle-income countries (LMICs). Methods: We reviewed existing IR frameworks to develop a hybrid framework designed to better understand U5M reduction in LMICs from identification of leading causes of amenable U5M, to EBI choice, identification, and testing of strategies, work to achieve sustainability at scale, and key contextual factors. We then conducted a mixed-methods case study of Rwanda using the framework to explore its utility in understanding the steps the country took in EBI-related decision-making and implementation between 2000-2015, key contextual factors which hindered or facilitated success, and to extract actionable knowledge for other countries working to reduce U5M. Results: While relevant frameworks were identified, none individually covered the scope needed to understand Rwanda's actions and success. Building on these frameworks, we combined and adapted relevant frameworks to capture exploration, planning, implementation, contextual factors in LMICs such as Rwanda, and outcomes beyond effectiveness and coverage. Utilizing our hybrid framework in Rwanda, we studied multiple EBIs and identified a common pathway and cross-cutting strategies and contextual factors that supported the country's success in reducing U5M through the health system EBIs. Using these findings, we identified transferable lessons for other countries working to accelerate reduction in U5M. Conclusions: We found that a hybrid framework building on and adapting existing frameworks was successful in guiding data collection and interpretation of results, emerging new insights into how and why Rwanda achieved equitable introduction and implementation of health system EBIs that contributed to the decline in U5M, and generated lessons for countries working to drop U5M.

Strengthening the national health information system through a capacity-building and mentorship partnership (CBMP) programme: a health system and university partnership initiative in Ethiopia

Background

A strong health information system (HIS) is one of the essential building blocks for a resilient health system. The Ministry of Health (MOH) of Ethiopia is working on different initiatives to strengthen the national HIS. Among these is the Capacity-Building and Mentorship Partnership (CBMP) Programme in collaboration with public universities in Ethiopia since November 2017. This study aims to evaluate the outcomes and share experiences of the country in working with universities to strengthen the national HIS.

Methods

The study employed a mixed-methods approach that included 247 health organizations (health offices and facilities) of CBMP-implementing woredas (districts) and 23 key informant interviews. The programme focused on capacity-building and mentoring facilities and woreda health offices. The status of HIS was measured using a connected woreda checklist before and after the intervention. The checklist consists of items related to HIS infrastructure, data quality and administrative use. The organizations were classified as emerging, candidate or model based on the score. The findings were triangulated with qualitative data collected through key informant interviews.

Results

The results showed that the overall score of the HIS implementation was 46.3 before and 74.2 after implementation of the programme. The proportion of model organizations increased from 1.2% before to 31.8% after the programme implementation. The health system–university partnership has provided an opportunity for higher education institutions to understand the health system and tune their curricula to address real-world challenges. The partnership brought opportunities to conduct and produce local- and national-level evidence to improve the HIS. Weak ownership, poor responsiveness and poor perceptions of the programme were mentioned as major challenges in programme implementation.

Conclusion

The overall HIS has shown substantial progress in CBMP implementation woredas. A number of facilities became models in a short period of time after the implementation of the programme. The health system–university partnership was found to be a promising approach to improve the national HIS and to share the on-the-ground experiences with the university academicians. However, weak ownership and poor responsiveness to feedback were the major challenges identified as needing more attention in future programme implementation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-021-00787-x.

Development and application of a hybrid implementation research framework to understand success in reducing under-5 mortality in Rwanda

Background: We describe the development and testing of a hybrid implementation research (IR) framework to understand the pathways, successes, and challenges in addressing amenable under-5 mortality (U5M) - deaths preventable through health system-delivered evidence-based interventions (EBIs) - in low- and middle-income countries (LMICs). Methods: We reviewed existing IR frameworks to develop a hybrid framework designed to better understand U5M reduction in LMICs from identification of leading causes of amenable U5M, to EBI choice, identification, and testing of strategies, work to achieve sustainability at scale, and key contextual factors. We then conducted a mixed-methods case study of Rwanda using the framework to explore its utility in understanding the steps the country took in EBI-related decision-making and implementation between 2000-2015, key contextual factors which hindered or facilitated success, and to extract actionable knowledge for other countries working to reduce U5M. Results: While relevant frameworks were identified, none individually covered the scope needed to understand Rwanda's actions and success. Building on these frameworks, we combined and adapted relevant frameworks to capture exploration, planning, implementation, contextual factors in LMICs such as Rwanda, and outcomes beyond effectiveness and coverage. Utilizing our hybrid framework in Rwanda, we studied multiple EBIs and identified a common pathway and cross-cutting strategies and contextual factors that supported the country's success in reducing U5M through the health system EBIs. Using these findings, we identified transferable lessons for other countries working to accelerate reduction in U5M. Conclusions: We found that a hybrid framework building on and adapting existing frameworks was successful in guiding data collection and interpretation of results, emerging new insights into how and why Rwanda achieved equitable introduction and implementation of health system EBIs that contributed to the decline in U5M, and generated lessons for countries working to drop U5M.

Continuous quality improvement (CQI) Institutionalization to reach 95:95:95 HIV targets: a multicountry experience from the Global South

BACKGROUND

Scaling up continuous quality improvement (CQI) processes could be key in achieving the 95:95:95 cascade and global HIV targets. This paper describes the experiences and outcomes related to implementing CQI processes to help reach these targets, with particular focus on clinical and programmatic settings in 6 countries from the global south.

METHODS

The HIV program at the University of Maryland, Baltimore (UMB) implemented an adapted CQI model in Kenya, Tanzania, Botswana, Zambia, Nigeria and Rwanda that included the following steps: (1) analysing the problem to identify goals and objectives for improvement; (2) developing individual changes or 'change packages', (3) developing a monitoring system to measure improvements; and (4) implementing and measuring changes through continuous 'plan-do-study-act' (PDSA) cycles. We describe country-level experiences related to implementing this adaptive design, a collaborative learning and scale-up/sustainability model that addresses the 95:95:95 global HIV targets via a CQI learning network, and mechanisms for fostering communication and the sharing of ideas and results; we describe trends both before and after model implementation.

RESULTS

Our selected country-level experiences based on implementing our CQI approach resulted in an increased partner testing acceptance rate from 21.7 to 48.2 % in Rwanda, which resulted in an increase in the HIV testing yield from 2.1 to 6.3 %. In Botswana, the overall linkage to treatment improved from 63 to 94 %, while in Kenya, the viral load testing uptake among paediatric and adolescent patients improved from 65 to 96 %, and the viral load suppression improved from 53 to 88 %.

CONCLUSIONS

Adopting CQI processes is a useful approach for accelerating progress towards the attainment of the global 95:95:95 HIV targets. This paper also highlights the value of institutionalizing CQI processes and building the capacity of Ministry of Health (MoH) personnel in sub-Saharan Africa for the effective quality improvement of HIV programs and subsequent sustainability efforts.

A realist systematic review of evidence from low- and middle-income countries of interventions to improve immunization data use

BACKGROUND

The use of routine immunization data by health care professionals in low- and middle-income countries remains an underutilized resource in decision-making. Despite the significant resources invested in developing national health information systems, systematic reviews of the effectiveness of data use interventions are lacking. Applying a realist review methodology, this study synthesized evidence of effective interventions for improving data use in decision-making.

METHODS

We searched PubMed, POPLINE, Centre for Agriculture and Biosciences International Global Health, and African Journals Online for published literature. Grey literature was obtained from conference, implementer, and technical agency websites and requested from implementing organizations. Articles were included if they reported on an intervention designed to improve routine data use or reported outcomes related to data use, and targeted health care professionals as the principal data users. We developed a theory of change a priori for how we expect data use interventions to influence data use. Evidence was then synthesized according to data use intervention type and level of the health system targeted by the intervention.

RESULTS

The searches yielded 549 articles, of which 102 met our inclusion criteria, including 49 from peer-reviewed journals and 53 from grey literature. A total of 66 articles reported on immunization data use interventions and 36 articles reported on data use interventions for other health sectors. We categorized 68 articles as research evidence and 34 articles as promising strategies. We identified ten primary intervention categories, including electronic immunization registries, which were the most reported intervention type (n = 14). Among the research evidence from the immunization sector, 32 articles reported intermediate outcomes related to data quality and availability, data analysis, synthesis, interpretation, and review. Seventeen articles reported data-informed decision-making as an intervention outcome, which could be explained by the lack of consensus around how to define and measure data use.

CONCLUSIONS

Few immunization data use interventions have been rigorously studied or evaluated. The review highlights gaps in the evidence base, which future research and better measures for assessing data use should attempt to address.

Mixed-method estimation of population-level HIV viral suppression rate in the Western Cape, South Africa

INTRODUCTION

There are few population-wide data on viral suppression (VS) that can be used to monitor programmatic targets in sub-Saharan Africa. We describe how routinely collected viral load (VL) data from antiretroviral therapy (ART) programmes can be extrapolated to estimate population VS and validate this using a combination of empiric and model-based estimates.

METHODS

VL test results from were matched using a record linkage algorithm to obtain linked results for individuals. Test-level and individual-level VS rates were based on test VL values <1000 cps/mL, and individual VL <1000 cps/mL in a calendar year, respectively. We calculated population VS among people living with HIV (PLWH) in the province by combining census-derived midyear population estimates, HIV prevalence estimates and individual level VS estimates from routine VL data.

RESULTS

Approximately 1.9 million VL test results between 2008 and 2018 were analysed. Among individuals in care, VS increased from 85.5% in 2008 to 90% in 2018. Population VS among all PLWH in the province increased from 12.2% in 2008 to 51.0% in 2017. The estimates derived from this method are comparable to those from other published studies. Sensitivity analyses showed that the results are robust to variations in linkage method, but sensitive to the extreme combinations of assumed VL testing coverage and population HIV prevalence.

CONCLUSION

While validation of this method in other settings is required, this approach provides a simple, robust method for estimating population VS using routine data from ART services that can be employed by national programmes in high-burden settings.

Surgical data strengthening in Ethiopia: results of a Kirkpatrick framework evaluation of a data quality intervention

Background: One key challenge in improving surgical care in resource-limited settings is the lack of high-quality and informative data. In Ethiopia, the Safe Surgery 2020 (SS2020) project developed surgical key performance indicators (KPIs) to evaluate surgical care within the country. New data collection methods were developed and piloted in 10 SS2020 intervention hospitals in the Amhara and Tigray regions of Ethiopia.

Objective: To assess the feasibility of collecting and reporting new surgical indicators and measure the impact of a surgical Data Quality Intervention (DQI) in rural Ethiopian hospitals.

Methods: An 8-week DQI was implemented to roll-out new data collection tools in SS2020 hospitals. The Kirkpatrick Method, a widely used mixed-method evaluation framework for training programs, was used to assess the impact of the DQI. Feedback surveys and focus groups at various timepoints evaluated the impact of the intervention on surgical data quality, the feasibility of a new data collection system, and the potential for national scale-up.

Results: Results of the evaluation are largely positive and promising. DQI participants reported knowledge gain, behavior change, and improved surgical data quality, as well as greater teamwork, communication, leadership, and accountability among surgical staff. Barriers remained in collection of high-quality data, such as lack of adequate human resources and electronic data reporting infrastructure.

Conclusions: Study results are largely positive and make evident that surgical data capture is feasible in low-resource settings and warrants more investment in global surgery efforts. This type of training and mentorship model can be successful in changing individual behavior and institutional culture regarding surgical data collection and reporting. Use of the Kirkpatrick Framework for evaluation of a surgical DQI is an innovative contribution to literature and can be easily adapted and expanded for use within global surgery.

Implementation research approaches to promoting universal health coverage in Africa: a scoping review

BACKGROUND

Implementation research has emerged as part of evidence-based decision-making efforts to plug current gaps in the translation of research evidence into health policy and practice. While there has been a growing number of initiatives promoting the uptake of implementation research in Africa, its role and effectiveness remain unclear, particularly in the context of universal health coverage (UHC). Hence, this scoping review aimed to identify and characterise the use of implementation research initiatives for assessing UHC-related interventions or programmes in Africa.

METHODS

The review protocol was developed based on the methodological framework proposed by Arksey and O'Malley, as enhanced by the Joanna Briggs Institute. The review is reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR). MEDLINE, Scopus and the Cochrane Library were searched. The search also included a hand search of relevant grey literature and reference lists. Literature sources involving the application of implementation research in the context of UHC in Africa were eligible for inclusion.

RESULTS

The database search yielded 2153 records. We identified 12 additional records from hand search of reference lists. After the removal of duplicates, we had 2051 unique records, of which 26 studies were included in the review. Implementation research was used within ten distinct UHC-related contexts, including HIV; maternal and child health; voluntary male medical circumcision; healthcare financing; immunisation; healthcare data quality; malaria diagnosis; primary healthcare quality improvement; surgery and typhoid fever control. The consolidated framework for implementation research (CFIR) was the most frequently used framework. Qualitative and mixed-methods study designs were the commonest methods used. Implementation research was mostly used to guide post-implementation evaluation of health programmes and the contextualisation of findings to improve future implementation outcomes. The most commonly reported contextual facilitators were political support, funding, sustained collaboration and effective programme leadership. Reported barriers included inadequate human and other resources; lack of incentives; perception of implementation as additional work burden; and socio-cultural barriers.

CONCLUSIONS

This review demonstrates that implementation research can be used to achieve UHC-related outcomes in Africa. It has identified important facilitators and barriers to the use of implementation research for promoting UHC in the region.

Development and application of a hybrid implementation research framework to understand success in reducing under-5 mortality in Rwanda

Background: We describe the development and testing of a hybrid implementation research (IR) framework to understand the pathways, successes, and challenges in addressing amenable under-5 mortality (U5M) - deaths preventable through health system-delivered evidence-based interventions (EBIs) - in low- and middle-income countries (LMICs). Methods: We reviewed existing IR frameworks to develop a hybrid framework designed to better understand U5M reduction in LMICs from identification of leading causes of amenable U5M, to EBI choice, identification and testing of strategies, work to achieve sustainability at scale and key contextual factors. We then conducted a mixed-methods case study of Rwanda using the framework to explore its utility in understanding the steps the country took in EBI-related decision-making and implementation between 2000-2015, key contextual factors which hindered or facilitated success, and extract actionable knowledge for other countries working to reduce U5M. Results: While relevant frameworks were identified, none individually covered the scope needed to understand Rwanda's actions and success. Building on these frameworks, we combined and adapted relevant frameworks to capture exploration, planning, implementation, contextual factors in LMICs such as Rwanda, and outcomes beyond effectiveness and coverage. Utilizing our hybrid framework in Rwanda, we studied multiple EBIs and identified a common pathway and cross-cutting strategies and contextual factors that supported the country's success in reducing U5M through the health system EBIs. Using these findings, we identified transferable lessons for other countries working to accelerate reduction in U5M. Conclusions: We found that a hybrid framework building on and adapting existing frameworks was successful in guiding data collection and interpretation of results, emerging new insights into how and why Rwanda achieved equitable introduction and implementation of health system EBIs that contributed to the decline in U5M, and generated lessons for countries working to drop U5M.

Uterotonics for prevention of postpartum haemorrhage: EN-BIRTH multi-country validation study

BACKGROUND

Postpartum haemorrhage (PPH) is a leading cause of preventable maternal mortality worldwide. The World Health Organization (WHO) recommends uterotonic administration for every woman after birth to prevent PPH. There are no standardised data collected in large-scale measurement platforms. The Every Newborn Birth Indicators Research Tracking in Hospitals (EN-BIRTH) is an observational study to assess the validity of measurement of maternal and newborn indicators, and this paper reports findings regarding measurement of coverage and quality for uterotonics.

METHODS

The EN-BIRTH study took place in five hospitals in Bangladesh, Nepal and Tanzania, from July 2017 to July 2018. Clinical observers collected tablet-based, time-stamped data. We compared observation data for uterotonics to routine hospital register-records and women's report at exit-interview survey. We analysed the coverage and quality gap for timing and dose of administration. The register design was evaluated against gap analyses and qualitative interview data assessing the barriers and enablers to data recording and use.

RESULTS

Observed uterotonic coverage was high in all five hospitals (> 99%, 95% CI 98.7-99.8%). Survey-report underestimated coverage (79.5 to 91.7%). "Don't know" replies varied (2.1 to 14.4%) and were higher after caesarean (3.7 to 59.3%). Overall, there was low accuracy in survey data for details of uterotonic administration (type and timing). Register-recorded coverage varied in four hospitals capturing uterotonics in a specific column (21.6, 64.5, 97.6, 99.4%). The average coverage measurement gap was 18.1% for register-recorded and 6.0% for survey-reported coverage. Uterotonics were given to 15.9% of women within the "right time" (1 min) and 69.8% within 3 min. Women's report of knowing the purpose of uterotonics after birth ranged from 0.4 to 64.9% between hospitals. Enabling register design and adequate staffing were reported to improve routine recording.

CONCLUSIONS

Routine registers have potential to track uterotonic coverage - register data were highly accurate in two EN-BIRTH hospitals, compared to consistently underestimated coverage by survey-report. Although uterotonic coverage was high, there were gaps in observed quality for timing and dose. Standardisation of register design and implementation could improve data quality and data flow from registers into health management information reporting systems, and requires further assessment.

'Do you trust those data?'-a mixed-methods study assessing the quality of data reported by community health workers in Kenya and Malawi

High-quality data are essential to monitor and evaluate community health worker (CHW) programmes in low- and middle-income countries striving towards universal health coverage. This mixed-methods study was conducted in two purposively selected districts in Kenya (where volunteers collect data) and two in Malawi (where health surveillance assistants are a paid cadre). We calculated data verification ratios to quantify reporting consistency for selected health indicators over 3 months across 339 registers and 72 summary reports. These indicators are related to antenatal care, skilled delivery, immunization, growth monitoring and nutrition in Kenya; new cases, danger signs, drug stock-outs and under-five mortality in Malawi. We used qualitative methods to explore perceptions of data quality with 52 CHWs in Kenya, 83 CHWs in Malawi and 36 key informants. We analysed these data using a framework approach assisted by NVivo11. We found that only 15% of data were reported consistently between CHWs and their supervisors in both contexts. We found remarkable similarities in our qualitative data in Kenya and Malawi. Barriers to data quality mirrored those previously reported elsewhere including unavailability of data collection and reporting tools; inadequate training and supervision; lack of quality control mechanisms; and inadequate register completion. In addition, we found that CHWs experienced tensions at the interface between the formal health system and the communities they served, mediated by the social and cultural expectations of their role. These issues affected data quality in both contexts with reports of difficulties in negotiating gender norms leading to skipping sensitive questions when completing registers; fabrication of data; lack of trust in the data; and limited use of data for decision-making. While routine systems need strengthening, these more nuanced issues also need addressing. This is backed up by our finding of the high value placed on supportive supervision as an enabler of data quality.

Improving the quality of routine maternal and newborn data captured in primary health facilities in Gombe State, Northeastern Nigeria: a before-and-after study

OBJECTIVES

Primary objective: to assess nine data quality metrics for 14 maternal and newborn health data elements, following implementation of an integrated, district-focused data quality intervention.

SECONDARY OBJECTIVE

to consider whether assessing the data quality metrics beyond completeness and accuracy of facility reporting offered new insight into reviewing routine data quality.

DESIGN

Before-and-after study design.

SETTING

Primary health facilities in Gombe State, Northeastern Nigeria.

PARTICIPANTS

Monitoring and evaluation officers and maternal, newborn and child health coordinators for state-level and all 11 local government areas (district-equivalent) overseeing 492 primary care facilities offering maternal and newborn care services.

INTERVENTION

Between April 2017 and December 2018, we implemented an integrated data quality intervention which included: introduction of job aids and regular self-assessment of data quality, peer-review and feedback, learning workshops, work planning for improvement, and ongoing support through social media.

OUTCOME MEASURES

9 metrics for the data quality dimensions of completeness and timeliness, internal consistency of reported data, and external consistency.

RESULTS

The data quality intervention was associated with improvements in seven of nine data quality metrics assessed including availability and timeliness of reporting, completeness of data elements, accuracy of facility reporting, consistency between related data elements, and frequency of outliers reported. Improvement differed by data element type, with content of care and commodity-related data improving more than contact-related data. Increases in the consistency between related data elements demonstrated improved internal consistency within and across facility documentation.

CONCLUSIONS

An integrated district-focused data quality intervention-including regular self-assessment of data quality, peer-review and feedback, learning workshops, work planning for improvement, and ongoing support through social media-can increase the completeness, accuracy and internal consistency of facility-based routine data.

How well are non-communicable disease services being integrated into primary health care in Africa: A review of progress against World Health Organization's African regional targets

OBJECTIVE

In Africa, mortality due to non-communicable diseases (NCDs) is projected to overtake the combined mortality from communicable, maternal, neonatal, and nutritional diseases by 2030. To address this growing NCD burden, primary health care (PHC) systems will require substantial re-orientation. In this study, we reviewed the progress of African countries towards integrating essential NCD services into PHC.

METHODS

A review of World Health Organization (WHO) reports was conducted for all 47 countries in the WHO African Region. To report each country's progress, we used an a priori framework developed by the WHO regional office for Africa (AFRO). Twelve indicators were used to measure countries' progress. The proportion of countries meeting each indicator was tabulated using a heat map. Correlation between country income status and attainment of each indicator was also assessed.

FINDINGS

No country met all the recommended indicators to integrate NCD services into PHC and seven countries met none of the indicators. Few countries (30%) had nationally approved guidelines for NCD management and very few reported availabilities of all essential NCD medicines (13%) and technologies (11%) in PHC facilities. There was no overall correlation between a country's GDP per capita and the aggregate of targets being met (rho = 0.23; P = .12). There was, however, a modestly negative correlation between out-of-pocket expenditure and overall country progress (rho = -0.58; P < .001).

CONCLUSION

Progress by AFRO Member States in integrating NCD care into PHC is variable across the region. Enhanced government commitment and judicious resource allocation to prioritize NCDs are needed. Particular areas of focus include increasing the uptake of simplified guidelines for NCDs; increasing workforce capacity to manage NCDs; and removing access barriers to essential medicines and basic diagnostic technologies.

Improving quality and use of routine health information system data in low- and middle-income countries: A scoping review

BACKGROUND

A routine health information system is one of the essential components of a health system. Interventions to improve routine health information system data quality and use for decision-making in low- and middle-income countries differ in design, methods, and scope. There have been limited efforts to synthesise the knowledge across the currently available intervention studies. Thus, this scoping review synthesised published results from interventions that aimed at improving data quality and use in routine health information systems in low- and middle-income countries.

METHOD

We included articles on intervention studies that aimed to improve data quality and use within routine health information systems in low- and middle-income countries, published in English from January 2008 to February 2020. We searched the literature in the databases Medline/PubMed, Web of Science, Embase, and Global Health. After a meticulous screening, we identified 20 articles on data quality and 16 on data use. We prepared and presented the results as a narrative.

RESULTS

Most of the studies were from Sub-Saharan Africa and designed as case studies. Interventions enhancing the quality of data targeted health facilities and staff within districts, and district health managers for improved data use. Combinations of technology enhancement along with capacity building activities, and data quality assessment and feedback system were found useful in improving data quality. Interventions facilitating data availability combined with technology enhancement increased the use of data for planning.

CONCLUSION

The studies in this scoping review showed that a combination of interventions, addressing both behavioural and technical factors, improved data quality and use. Interventions addressing organisational factors were non-existent, but these factors were reported to pose challenges to the implementation and performance of reported interventions.

Using routine health information data for research in low- and middle-income countries: a systematic review

BACKGROUND

Routine health information systems (RHISs) support resource allocation and management decisions at all levels of the health system, as well as strategy development and policy-making in many low- and middle-income countries (LMICs). Although RHIS data represent a rich source of information, such data are currently underused for research purposes, largely due to concerns over data quality. Given that substantial investments have been made in strengthening RHISs in LMICs in recent years, and that there is a growing demand for more real-time data from researchers, this systematic review builds upon the existing literature to summarize the extent to which RHIS data have been used in peer-reviewed research publications.

METHODS

Using terms 'routine health information system', 'health information system', or 'health management information system' and a list of LMICs, four electronic peer-review literature databases were searched from inception to February 202,019: PubMed, Scopus, EMBASE, and EconLit. Articles were assessed for inclusion based on pre-determined eligibility criteria and study characteristics were extracted from included articles using a piloted data extraction form.

RESULTS

We identified 132 studies that met our inclusion criteria, originating in 37 different countries. Overall, the majority of the studies identified were from Sub-Saharan Africa and were published within the last 5 years. Malaria and maternal health were the most commonly studied health conditions, although a number of other health conditions and health services were also explored.

CONCLUSIONS

Our study identified an increasing use of RHIS data for research purposes, with many studies applying rigorous study designs and analytic methods to advance program evaluation, monitoring and assessing services, and epidemiological studies in LMICs. RHIS data represent an underused source of data and should be made more available and further embraced by the research community in LMIC health systems.

Understanding the challenges associated with the use of data from routine health information systems in low- and middle-income countries: A systematic review

BACKGROUND

Routine health information systems (RHISs) are crucial to informing decision-making at all levels of the health system. However, the use of RHIS data in low- and middle-income countries (LMICs) is limited due to concerns regarding quality, accuracy, timeliness, completeness and representativeness.

OBJECTIVE

This study systematically reviewed technical, behavioural and organisational/environmental challenges that hinder the use of RHIS data in LMICs and strategies implemented to overcome these challenges.

METHOD

Four electronic databases were searched for studies describing challenges associated with the use of RHIS data and/or strategies implemented to circumvent these challenges in LMICs. Identified articles were screened against inclusion and exclusion criteria by two independent reviewers.

RESULTS

Sixty studies met the inclusion criteria and were included in this review, 55 of which described challenges in using RHIS data and 20 of which focused on strategies to address these challenges. Identified challenges and strategies were organised by their technical, behavioural and organisational/environmental determinants and by the core steps of the data process. Organisational/environmental challenges were the most commonly reported barriers to data use, while technical challenges were the most commonly addressed with strategies.

CONCLUSION

Despite the known benefits of RHIS data for health system strengthening, numerous challenges continue to impede their use in practice.

IMPLICATIONS

Additional research is needed to identify effective strategies for addressing the determinants of RHIS use, particularly given the disconnect identified between the type of challenge most commonly described in the literature and the type of challenge most commonly targeted for interventions.

Perceptions of factors influencing the introduction and adoption of electronic immunization registries in Tanzania and Zambia: a mixed methods study

BACKGROUND

As technology has become cheaper and more accessible, health programs are adopting digital health interventions (DHI) to improve the provision of and demand for health services. These interventions are complex and require strong coordination and support across different health system levels and government departments, and they need significant capacities in technology and information to be properly implemented. Electronic immunization registries (EIRs) are types of DHI used to capture, store, access, and share individual-level, longitudinal health information in digitized records. The BID Initiative worked in partnership with the governments of Tanzania and Zambia to introduce an EIR at the sub-national level in both countries within 5 years as part of a multi-component complex intervention package focusing on data use capacity-building.

METHODS

We aimed to gather and describe learnings from the BID experience by conducting a framework-based mixed methods study to describe perceptions of factors that influenced scale-up of the EIR. Data were collected through key informant interviews, a desk review, EIRs, and health management information systems. We described how implementation of the EIRs fulfilled domains described in our conceptual framework and used cases to illustrate the relationships and relative influence of domains for scale-up and adoption of the EIR.

RESULTS

We found that there was no single factor that seemed to influence the introduction or sustained adoption of the EIR as many of the factors were interrelated. For EIR introduction, strong strategic engagement among partners was important, while EIR adoption was influenced by adequate staffing at facilities, training, use of data for supervision, internet and electricity connectivity, and community sensitization.

CONCLUSIONS

Organizations deploying DHIs in the future should consider how best to adapt their intervention to the existing ecosystem, including human resources and organizational capacity, as well as the changing technological landscape during planning and implementation.

Evaluating and optimizing the consolidated framework for implementation research (CFIR) for use in low- and middle-income countries: a systematic review

BACKGROUND

The Consolidated Framework for Implementation Research (CFIR) is a determinants framework that may require adaptation or contextualization to fit the needs of implementation scientists in low- and middle-income countries (LMICs). The purpose of this review is to characterize how the CFIR has been applied in LMIC contexts, to evaluate the utility of specific constructs to global implementation science research, and to identify opportunities to refine the CFIR to optimize utility in LMIC settings.

METHODS

A systematic literature review was performed to evaluate the use of the CFIR in LMICs. Citation searches were conducted in Medline, CINAHL, PsycINFO, CINAHL, SCOPUS, and Web of Science. Data abstraction included study location, study design, phase of implementation, manner of implementation (ex., data analysis), domains and constructs used, and justifications for use, among other variables. A standardized questionnaire was sent to the corresponding authors of included studies to determine which CFIR domains and constructs authors found to be compatible with use in LMICs and to solicit feedback regarding ways in which CFIR performance could be improved for use in LMICs.

RESULTS

Our database search yielded 504 articles, of which 34 met final inclusion criteria. The studies took place across 21 countries and focused on 18 different health topics. The studies primarily used qualitative study designs (68%). Over half (59%) of the studies applied the CFIR at study endline, primarily to guide data analysis or to contextualize study findings. Nineteen (59%) of the contacted authors participated in the survey. Authors unanimously identified culture and engaging as compatible with use in global implementation research. Only two constructs, patient needs and resources and individual stages of change were commonly identified as incompatible with use. Author feedback centered on team level influences on implementation, as well as systems characteristics, such as health system architecture. We propose a "Characteristics of Systems" domain and eleven novel constructs be added to the CFIR to increase its compatibility for use in LMICs.

CONCLUSIONS

These additions provide global implementation science practitioners opportunities to account for systems-level determinants operating independently of the implementing organization. Newly proposed constructs require further reliability and validity assessments.

TRIAL REGISTRATION

PROSPERO, CRD42018095762.

Monitoring childbirth care in primary health facilities: a validity study in Gombe State, northeastern Nigeria

BACKGROUND

Improving the quality of facility-based births is a critical strategy for reducing the high burden of maternal and neonatal mortality and morbidity across all settings. Accurate data on childbirth care is essential for monitoring progress. In northeastern Nigeria, we assessed the validity of childbirth care indicators in a rural primary health care context, as documented by health workers and reported by women at different recall periods.

METHODS

We compared birth observations (gold standard) to: (i) facility exit interviews with observed women; (ii) household follow-up interviews 9-22 months after childbirth; and (iii) health worker documentation in the maternity register. We calculated sensitivity, specificity, and area under the receiver operating curve (AUC) to determine individual-level reporting accuracy. We calculated the inflation factor (IF) to determine population-level validity.

RESULTS

Twenty-five childbirth care indicators were assessed to validate health worker documentation and women's self-reports. During exit interviews, women's recall had high validity (AUC≥0.70 and 0.75

CONCLUSIONS

In addition to standard household surveys, monitoring of facility-based childbirth care should consider drawing from and linking multiple data sources, including routine health facility data and exit interviews with recently delivered women.

Collapse

Key Words Collapse

MESH Headings

• Adolescent
• Adult
• Delivery, Obstetric/standards
• Documentation/methods
• Female
• Humans
• Infant, Newborn
• Medical Records
• Middle Aged
• Nigeria
• Pregnancy
• Primary Health Care
• Reproducibility of Results
• Rural Health Services
• Self Report
• Young Adult

Collapse

Grants Collapse

Affiliation(s)

Antoinette Alas Bhattacharya Department of Disease Control, London School of Hygiene & Tropical Medicine, London, UK
Elizabeth Allen Department of Medical Statistics, London School of Hygiene & Tropical Medicine, London, UK
Nasir Umar Department of Disease Control, London School of Hygiene & Tropical Medicine, London, UK
Adamu Umar Usman Data Research and Mapping Consult, Abuja, Nigeria
Habila Felix State Primary Health Care Development Agency, Gombe, Nigeria
Ahmed Audu State Primary Health Care Development Agency, Gombe, Nigeria
Joanna Rm Schellenberg Department of Disease Control, London School of Hygiene & Tropical Medicine, London, UK
Tanya Marchant Department of Disease Control, London School of Hygiene & Tropical Medicine, London, UK

Collapse

Ageing, frailty and resilience in Botswana: rapid ageing, rapid change. Findings from a national working group meeting and literature review

Background

The demography of Botswana is rapidly changing. Successes in tackling communicable diseases and economic development increased life expectancy from 53.7 years in 2006 to 66.8 years in 2016. The prevalence of diseases associated with older age, especially chronic non-communicable diseases including diabetes, hypertension and cerebrovascular disease are suspected to have increased but accurate data are lacking. The country has high youth unemployment and national retirement is at the age of 60, which limits the opportunity to build pensions for prosperity in older age. Changes across health, social care and economic policy are needed to prepare for a future ageing population.

Methods

This article reports on the national working group meeting outputs on issues that face older people, their families, healthcare providers, and policy makers in Botswana. A collaborative working group meeting was convened in Gaborone, Botswana on 25th September 2018 by the University of Botswana and University of Birmingham, UK, to identify key challenges and opportunities for an ageing Batswana population.

Results

There was agreement across diverse stakeholders of a need for effective and rapid policy formation to prepare and protect the future health and economy of an ageing Batswana population with a high burden of NCDs. The main priorities for policy-makers must be social care, poverty reduction and healthcare provision, conducted in an evidence-based manner, as far as practicable. To effectively achieve this, research and high-quality data collection mechanisms are required.

Conclusions

Future policy in Botswana must focus on the challenges that an ageing population brings, and development of health and care system resilience for the demographic change could be a model for healthcare policy across Southern Africa.

Comparison of Micro-Census Results for Magarya Ward, Wurno Local Government Area of Sokoto State, Nigeria, with Other Sources of Denominator Data

Routine immunization coverage in Nigeria is suboptimal. In the northwestern state of Sokoto, an independent population-based survey for 2016 found immunization coverage with the third dose of Pentavalent vaccine to be 3%, whereas administrative coverage in 2016 was reported to be 69%. One possibility driving this large discrepancy is that administrative coverage is calculated using an under-estimated target population. Official population projections from the 2006 Census are based on state-specific standard population growth rates. Immunization target population estimates from other sources have not been independently validated. We conducted a micro-census in Magarya ward, Wurno Local Government Area of Sokoto state to obtain an accurate count of the total population living in the ward, and to compare these results with other sources of denominator data. We developed a precise micro-plan using satellite imagery, and used the navigation tool EpiSample v1 in the field to guide teams to each building, without duplications or omissions. The particular characteristics of the selected ward underscore the importance of using standardized shape files to draw precise boundaries for enumeration micro-plans. While the use of this methodology did not resolve the discrepancy between independent and administrative vaccination coverage rates, a simplified application can better define the target population for routine immunization services and estimate the number of children still unprotected from vaccine-preventable diseases.

Quality of routine facility data for monitoring priority maternal and newborn indicators in DHIS2: A case study from Gombe State, Nigeria

INTRODUCTION

Routine health information systems are critical for monitoring service delivery. District Heath Information System, version 2 (DHIS2) is an open source software platform used in more than 60 countries, on which global initiatives increasingly rely for such monitoring. We used facility-reported data in DHIS2 for Gombe State, north-eastern Nigeria, to present a case study of data quality to monitor priority maternal and neonatal health indicators.

METHODS

For all health facilities in DHIS2 offering antenatal and postnatal care services (n = 497) and labor and delivery services (n = 486), we assessed the quality of data for July 2016-June 2017 according to the World Health Organization data quality review guidance. Using data from DHIS2 as well as external facility-level and population-level household surveys, we reviewed three data quality dimensions-completeness and timeliness, internal consistency, and external consistency-and considered the opportunities for improvement.

RESULTS

Of 14 priority maternal and neonatal health indicators that could be tracked through facility-based data, 12 were included in Gombe's DHIS2. During July 2016-June 2017, facility-reported data in DHIS2 were incomplete at least 40% of the time, under-reported 10%-60% of the events documented in facility registers, and showed inconsistencies over time, between related indicators, and with an external data source. The best quality data elements were those that aligned with Gombe's health program priorities, particularly older health programs, and those that reflected contact indicators rather than indicators related to the provision of commodities or content of care.

CONCLUSION

This case study from Gombe State, Nigeria, demonstrates the high potential for effective monitoring of maternal and neonatal health using DHIS2. However, coordinated action at multiple levels of the health system is needed to maximize reporting of existing data; rationalize data flow; routinize data quality review, feedback, and supervision; and ensure ongoing maintenance of DHIS2.

Evidence for national universal eye health plans

Many low- and middle-income countries use national eye-care plans to guide efforts to strengthen eye-care services. The World Health Organization recognizes that evidence is essential to inform these plans. We assessed how evidence was incorporated in a sample of 28 national eye-care plans generated since the Universal eye health: a global action plan 2014-2019 was endorsed by the World Health Assembly in 2013. Most countries (26, 93%) cited estimates of the prevalence of blindness and 18 countries (64%) had set targets for the cataract surgical rate in their plan. Other evidence was rarely cited or used to set measurable targets. No country cited evidence from systematic reviews or solution-based research. This limited use of evidence reflects its low availability, but also highlights incomplete use of existing evidence. For example, despite sex-disaggregated data and cataract surgical coverage being available from surveys in 20 countries (71%), these data were reported in the eye health plans of only nine countries (32%). Only three countries established sex-disaggregated indicators and only one country had set a target for cataract surgical coverage for future monitoring. Countries almost universally recognized the need to strengthen health information systems and almost one-third planned to undertake operational or intervention research. Realistic strategies need to be identified and supported to translate these intentions into action. To gain insights into how a country can strengthen its evidence-informed approach to eye-care planning, we reflect on the process underway to develop Kenya's seventh national plan (2019-2023).

Toward communities as systems: a sequential mixed methods study to understand factors enabling implementation of a skilled birth attendance intervention in Nampula Province, Mozambique

BACKGROUND

Skilled birth attendance, institutional deliveries, and provision of quality, respectful care are key practices to improve maternal and neonatal health outcomes. In Mozambique, the government has prioritized improved service delivery and demand for these practices, alongside "humanization of the birth process." An intervention implemented in Nampula province beginning in 2009 saw marked improvement in institutional delivery rates. This study uses a sequential explanatory mixed methods case study design to explore the contextual factors that may have contributed to the observed increase in institutional deliveries.

METHODS

A descriptive time series analysis was conducted using clinic register data from 2009 to 2014 to assess institutional delivery coverage rates in two primary health care facilities, in two districts of Nampula province. Site selection was based on facilities exhibiting an initial increase in institutional deliveries from 2009 to 2011, similarity of health system attributes, and accessibility for study participation. Using a modified Delphi technique, two expert panels-each composed of ten stakeholders familiar with maternal health implementation at facility, district, provincial, and national levels-were convened to formulate the "story" of the implementation and to identify contextual factors to use in developing semi-structured interview guides. Thirty-four key informant interviews with facility MCH nurses, facility managers, traditional birth attendants, community leaders, and beneficiaries were then conducted and analyzed using the Consolidated Framework for Implementation Research through inductive and deductive coding.

RESULTS

The two sites' skilled birth attendance coverage of estimated live births reached 80 and 100%, respectively. Eight contextual and human factors were found as dominant themes. Though both sites achieved increases, implementation context differed significantly with compelling examples of both respectful and disrespectful care. In one site, facility and community actors worked together as complementary systems to sustain improved care and institutional deliveries. In the other, community actors sustained implementation and institutional deliveries largely in absence of health system counterparts.

CONCLUSION

Findings support global health recommendations for combined health system and community interventions for improved MNH outcomes including delivery of respectful care, and further suggest the capacity of communities to act as systems both in partnership to and independent of the formal health system.

Data-driven quality improvement in low-and middle-income country health systems: lessons from seven years of implementation experience across Mozambique, Rwanda, and Zambia

BACKGROUND

Well-functioning health systems need to utilize data at all levels, from the provider, to local and national-level decision makers, in order to make evidence-based and needed adjustments to improve the quality of care provided. Over the last 7 years, the Doris Duke Charitable Foundation's African Health Initiative funded health systems strengthening projects at the facility, district, and/or provincial level to improve population health. Increasing data-driven decision making was a common strategy in Mozambique, Rwanda and Zambia. This paper describes the similar and divergent approaches to increase data-driven quality of care improvements (QI) and implementation challenge and opportunities encountered in these three countries.

METHODS

Eight semi-structured in-depth interviews (IDIs) were administered to program staff working in each country. IDIs for this paper included principal investigators of each project, key program implementers (medically-trained support staff, data managers and statisticians, and country directors), as well as Ministry of Health counterparts. IDI data were collected through field notes; interviews were not audio recorded. Data were analyzed using thematic analysis but no systematic coding was conducted. IDIs were supplemented through donor report abstractions, a structured questionnaire, one-on-one phone calls, and email exchanges with country program leaders to clarify and expand on key themes emerging from IDIs.

RESULTS

Project successes ranged from over 450 collaborative action-plans developed, implemented, and evaluated in Mozambique, to an increase from <10% to >80% of basic clinical protocols followed in intervention facilities in rural Zambia, and a shift from a lack of awareness of health data among health system staff to collaborative ownership of data and using data to drive change in Rwanda.

CONCLUSION

Based on common successes across the country experiences, we recommend future data-driven QI interventions begin with data quality assessments to promote that rapid health system improvement is possible, ensure confidence in available data, serve as the first step in data-driven targeted improvements, and improve staff data analysis and visualization skills. Explicit Ministry of Health collaborative engagement can ensure performance review is collaborative and internally-driven rather than viewed as an external "audit."

Bridging the intervention-implementation gap in primary health care delivery: the critical role of integrated implementation research

For national and local leaders to achieve universal health coverage, a new approach or technique to gathering evidence and understanding the contexts that influence the outcome of a study and goes beyond the quantitative results of clinical trials and pilot projects is important. The Doris Duke Charitable Foundation's African Health Initiative (AHI) was designed to produce this type of knowledge through embedding implementation research into Population Health Implementation and Training (PHIT) partnership projects in five countries (Ghana, Mozambique, Rwanda, Tanzania, and Zambia) with the goal of improving primary health care and population health. In Ghana, this integration of research into implementation has contributed to the successful testing, adaptation and implementation of the Community-based Health Planning and Services (CHPS) model (The Navrongo Pilot Project), with results from the AHI-funded work informing national scale-up of effective practices. Further application of implementation science methods and frameworks to study cross-project lessons also produced the evidence needed by national and local decision makers on how and why different intervention components were successful and where and how local context drove implementation and adaptation. Cross-project research also identified effective approaches across diverse settings for building capacity for data-driven improvement, coaching and mentoring clinicians and researchers, developing locally appropriate interventions to reduce neonatal mortality, and integrating implementation research to inform local implementers and researchers in more effective strategies to strengthen health systems and improve health services and population health. Evidence has already shown the potential for this type of work to accelerate regional learning and spread of successful interventions to achieve targeted health goals more efficiently, better enabling countries to achieve the ambitious, but important, U.N. Sustainable Development Goals.