Data-driven quality improvement in low-and middle-income country health systems: lessons from seven years of implementation experience across Mozambique, Rwanda, and Zambia

Routine Health Information System (RHIS) improvements for strengthened health system management

BACKGROUND

A well-functioning routine health information system (RHIS) can provide the information needed for health system management, for governance, accountability, planning, policy making, surveillance and quality improvement, but poor information support has been identified as a major obstacle for improving health system management.

OBJECTIVES

To assess the effects of interventions to improve routine health information systems in terms of RHIS performance, and also, in terms of improved health system management performance, and improved patient and population health outcomes.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library, MEDLINE Ovid and Embase Ovid in May 2019. We searched Global Health, Ovid and PsycInfo in April 2016. In January 2020 we searched for grey literature in the Grey Literature Report and in OpenGrey, and for ongoing trials using the International Clinical Trials Registry Platform (ICTRP) and ClinicalTrials.gov. In October 2019 we also did a cited reference search using Web of Science, and a 'similar articles' search in PubMed.

SELECTION CRITERIA

Randomised and non-randomised trials, controlled before-after studies and time-series studies comparing routine health information system interventions, with controls, in primary, hospital or community health care settings. Participants included clinical staff and management, district management and community health workers using routine information systems.

DATA COLLECTION AND ANALYSIS

Two authors independently reviewed records to identify studies for inclusion, extracted data from the included studies and assessed the risk of bias. Interventions and outcomes were too varied across studies to allow for pooled risk analysis. We present a 'Summary of findings' table for each intervention comparisons broadly categorised into Technical and Organisational (or a combination), and report outcomes on data quality and service quality. We used the GRADE approach to assess the certainty of the evidence.

MAIN RESULTS

We included six studies: four cluster randomised trials and two controlled before-after studies, from Africa and South America. Three studies evaluated technical interventions, one study evaluated an organisational intervention, and two studies evaluated a combination of technical and organisational interventions. Four studies reported on data quality and six studies reported on service quality. In terms of data quality, a web-based electronic TB laboratory information system probably reduces the length of time to reporting of TB test results, and probably reduces the overall rate of recording errors of TB test results, compared to a paper-based system (moderate certainty evidence). We are uncertain about the effect of the electronic laboratory information system on the recording rate of serious (misidentification) errors for TB test results compared to a paper-based system (very low certainty evidence). Misidentification errors are inaccuracies in transferring test results between an electronic register and patients' clinical charts. We are also uncertain about the effect of the intervention on service quality (timeliness of starting or changing a patient's TB treatment) (very low certainty evidence). A hand-held electronic device probably improves the length of time to report TB test results, and probably reduces the total frequency of recording errors in TB test results between the laboratory notebook and the electronic information record system, compared to a paper-based system (moderate-certainty evidence). We are, however, uncertain about the effect of the intervention on the frequency of serious (misidentification) errors in recording between the laboratory notebook and the electronic information record, compared to a paper-based system (very low certainty evidence). We are uncertain about the effect of a hospital electronic health information system on service quality (length of time outpatients spend at hospital, length of hospital stay, and hospital revenue collection), compared to a paper-based system (very low certainty evidence). High-intensity brief text messaging (SMS) may make little or no difference to data quality (in terms of completeness of documentation of pregnancy outcomes), compared to low-intensity brief text messaging (low-certainty evidence). We are uncertain about the effect of electronic drug stock notification (with either data management support or product transfer support) on service quality (in terms of transporting stock and stock levels), compared to paper-based stock notification (very low certainty evidence). We are uncertain about the effect of health information strengthening (where it is part of comprehensive service quality improvement intervention) on service quality (health worker motivation, receipt of training by health workers, health information index scores, quality of clinical observation of children and adults) (very low certainty evidence).

AUTHORS' CONCLUSIONS

The review indicates mixed effects of mainly technical interventions to improve data quality, with gaps in evidence on interventions aimed at enhancing data-informed health system management. There is a gap in interventions studying information support beyond clinical management, such as for human resources, finances, drug supply and governance. We need to have a better understanding of the causal mechanisms by which information support may affect change in management decision-making, to inform robust intervention design and evaluation methods.

A, Salim N, Zaman SB, Shamba D, Blencowe H, Ruysen H, El Arifeen S, Boggs D, Gordeev VS, Rahman QSU, Hossain T, Joshi E, Thapa S, Poudel RP, Poudel D, Chaudhary P, Karki R, Chitrakar B, Mkopi N, Wisiko A, Kitende AP, Shirati MR, Chingalo C, Semhando AO, Mtei C, Mwenisongole V, Bakuza JM, Kombo J, Mbaruku G, Lawn JE. Labour and delivery ward register data availability, quality, and utility - Every Newborn - birth indicators research tracking in hospitals (EN-BIRTH) study baseline analysis in three countries

BACKGROUND

Countries with the highest burden of maternal and newborn deaths and stillbirths often have little information on these deaths. Since over 81% of births worldwide now occur in facilities, using routine facility data could reduce this data gap. We assessed the availability, quality, and utility of routine labour and delivery ward register data in five hospitals in Bangladesh, Nepal, and Tanzania. This paper forms the baseline register assessment for the Every Newborn-Birth Indicators Research Tracking in Hospitals (EN-BIRTH) study.

METHODS

We extracted 21 data elements from routine hospital labour ward registers, useful to calculate selected maternal and newborn health (MNH) indicators. The study sites were five public hospitals during a one-year period (2016-17). We measured 1) availability: completeness of data elements by register design, 2) data quality: implausibility, internal consistency, and heaping of birthweight and explored 3) utility by calculating selected MNH indicators using the available data.

RESULTS

Data were extracted for 20,075 births. Register design was different between the five hospitals with 10-17 of the 21 selected MNH data elements available. More data were available for health outcomes than interventions. Nearly all available data elements were > 95% complete in four of the five hospitals and implausible values were rare. Data elements captured in specific columns were 85.2% highly complete compared to 25.0% captured in non-specific columns. Birthweight data were less complete for stillbirths than live births at two hospitals, and significant heaping was found in all sites, especially at 2500g and 3000g. All five hospitals recorded count data required to calculate impact indicators including; stillbirth rate, low birthweight rate, Caesarean section rate, and mortality rates.

CONCLUSIONS

Data needed to calculate MNH indicators are mostly available and highly complete in EN-BIRTH study hospital routine labour ward registers in Bangladesh, Nepal and Tanzania. Register designs need to include interventions for coverage measurement. There is potential to improve data quality if Health Management Information Systems utilization with feedback loops can be strengthened. Routine health facility data could contribute to reduce the coverage and impact data gap around the time of birth.

Decentralizing evidence-based decision-making in resource limited setting: A case of SNNP region, Ethiopia

BACKGROUND

Access to and the use of accurate, valid, reliable, timely, relevant, legible and complete information is vital for safe and reliable healthcare. Though the study area has been implementing standardized Health Management Information System (HMIS), there was a need for information on how well structures were utilizing information and this study was designed to assess HMIS data utilization.

METHODS

Facility based retrospective study was conducted in Southern Nations Nationalities and People's Region (SNNPR) in April, 2017. We included data from 163 sample facilities. Data use was evaluated by reviewing eight items from performance monitoring system that included activities from problem identification to monitoring of proposed action plans. Each item reviewed was recoded to yes or no and summed to judge overall performance.

RESULTS

About half (52%) of woredas, 26.2% health centers (HCs), 25% hospitals and 6.2% health posts (HPs) reviewed their performance monthly but only 20% woredas, 6.2% HCs, 1.5% HPs and no hospital prepared action plans after reviewing performance. Summary of 8 items assessed showed that majority of facilities (87.5% hospitals, 81.5% HPs and 70.8% HCs) were poor in data utilization.

CONCLUSIONS

Only about half of woredas and below one-fifth of health facilities were utilizing HMIS data and a lot to move to catch-up country's information revolution plan. Lower health care systems should be supported in evidence-based decision-making and progress should be monitored routinely quantitatively and/or qualitatively.

Challenges and facilitators to evidence-based decision-making for maternal and child health in Mozambique: district, municipal and national case studies

BACKGROUND

The need for evidence-based decision-making in the health sector is well understood in the global health community. Yet, gaps persist between the availability of evidence and the use of that evidence. Most research on evidence-based decision-making has been carried out in higher-income countries, and most studies look at policy-making rather than decision-making more broadly. We conducted this study to address these gaps and to identify challenges and facilitators to evidence-based decision-making in Maternal, Newborn and Child Health and Nutrition (MNCH&N) at the municipality, district, and national levels in Mozambique.

METHODS

We used a case study design to capture the experiences of decision-makers and analysts (n = 24) who participated in evidence-based decision-making processes related to health policies and interventions to improve MNCH&N in diverse decision-making contexts (district, municipality, and national levels) in 2014-2017, in Mozambique. We examined six case studies, at the national level, in Maputo City and in two districts of Sofala Province and two of Zambézia Province, using individual in-depth interviews with key informants and a document review, for three weeks, in July 2018.

RESULTS

Our analysis highlighted various challenges for evidence-based decision-making for MNCH&N, at national, district, and municipality levels in Mozambique, including limited demand for evidence, limited capacity to use evidence, and lack of trust in the available evidence. By contrast, access to evidence, and availability of evidence were viewed positively and seen as potential facilitators. Organizational capacity for the demand and use of evidence appears to be the greatest challenge; while individual capacity is also a barrier.

CONCLUSION

Evidence-based decision-making requires that actors have access to evidence and are empowered to act on that evidence. This, in turn, requires alignment between those who collect data, those who analyze and interpret data, and those who make and implement decisions. Investments in individual, organizational, and systems capacity to use evidence are needed to foster practices of evidence-based decision-making for improved maternal and child health in Mozambique.

Improving the cascade of global tuberculosis care: moving from the "what" to the "how" of quality improvement

Tuberculosis is preventable, treatable, and curable, yet it has the highest mortality rate of infectious diseases worldwide. Over the past decade, services to prevent, screen, diagnose, and treat tuberculosis have been developed and scaled up globally, but progress to end the disease as a public health threat has been slow, particularly in low-income and middle-income countries. In these settings, low-quality tuberculosis prevention, diagnostic, and treatment services frustrate efforts to translate use of existing tools, approaches, and treatment regimens into improved individual and public health outcomes. Increasingly sophisticated methods have been used to identify gaps in quality of tuberculosis care, but inadequate work has been done to apply these findings to activities that generate population-level improvements. In this Personal View, we contend that shifting the focus from the "what" to the "how" of quality improvement will require National Tuberculosis Programmes to change the way they organise, use data, implement, and respond to the needs and preferences of people with tuberculosis and at-risk communities.

How community and healthcare provider perceptions, practices and experiences influence reporting, disclosure and data collection on stillbirth: Findings of a qualitative study in Afghanistan

Quality concerns exist with stillbirth data from low- and middle-income countries including under-reporting and misclassification which affect the reliability of burden estimates. This is particularly problematic for household survey data. Disclosure and reporting of stillbirths are affected by the socio-cultural context in which they occur and societal perceptions around pregnancy loss. In this qualitative study, we aimed to understand how community and healthcare providers' perceptions and practices around stillbirth influence stillbirth data quality in Afghanistan. We collected data through 55 in-depth interviews with women and men that recently experienced a stillbirth, female elders, community health workers, healthcare providers, and government officials in Kabul province, Afghanistan between October-November 2017. The results showed that at the community-level, there was variation in local terminology and interpretation of stillbirth which did not align with the biomedical categories of stillbirth and miscarriage and could lead to misclassification. Specific birth attendant practices such as avoiding showing mothers their stillborn baby had implications for women's ability to recall skin appearance and determine stillbirth timing; however, parents who did see their baby, had a detailed recollection of these characteristics. Birth attendants also unintentionally misclassified birth outcomes. We found several practices that could potentially reduce under-reporting and misclassification of stillbirth; these included the cultural significance of ascertaining signs of life after birth (which meant families distinguished between stillbirths and early neonatal deaths); the perceived value and social recognition of a stillborn; and openness of families to disclose and discuss stillbirths. At the facility-level, we identified that healthcare provider's practices driven by institutional culture and demands, family pressure, and socio-cultural influences, could contribute to under-reporting or misclassification of stillbirths. Data collection methodologies need to take into consideration the socio-cultural context and investigate thoroughly how perceptions and practices might facilitate or impede stillbirth reporting in order to make progress on data quality improvements for stillbirth.

The At Risk Child Clinic (ARCC): 3 Years of Health Activities in Support of the Most Vulnerable Children in Beira, Mozambique

The concept of “children at risk” changes worldwide according to each specific context. Africa has a large burden of overall risk factors related to childhood health and development, most of which are of an infective or social origin. The aim of this study was to report and analyze the volumes of activities of fifteen At Risk Child Clinics (ARCCs) within the Beira District (Mozambique) over a 3 year-period in order to define the health profile of children accessing such health services. We retrospectively analyzed the data from all of the children accessing one of the 15 Beira ARCCs from January 2015 to December 2017. From this, 17,657 first consultations were registered. The motivations for accessing the services were in order of relevance: HIV exposure (n. 12,300; 69.7%), other risk conditions (n. 2542; 14.4%), Moderate Acute Malnutrition (MAM) (n. 1664; 9.4%), Severe Acute Malnutrition (SAM) (n. 772; 4.4%), and TB exposure (n. 542; 3.1%). During the first consultations, 16,865 children were screened for HIV (95.5%), and 7.89% tested HIV-positive. In our three years of experience, HIV exposure was the main indication for children to access the ARCCs in Mozambique. ARCCs could represent a strategic point to better understand health demands and to monitor the quality of care provided to this vulnerable population group, however significant effort is needed to improve the quality of the data collection.

Health system strengthening: a qualitative evaluation of implementation experience and lessons learned across five African countries

BACKGROUND

Achieving the United Nations Sustainable Development Goals in sub-Saharan Africa will require substantial improvements in the coverage and performance of primary health care delivery systems. Projects supported by the Doris Duke Charitable Foundation's (DDCF) African Health Initiative (AHI) created public-private-academic and community partnerships in five African countries to implement and evaluate district-level health system strengthening interventions. In this study, we captured common implementation experiences and lessons learned to understand core elements of successful health systems interventions.

METHODS

We used qualitative data from key informant interviews and annual progress reports from the five Population Health Implementation and Training (PHIT) partnership projects funded through AHI in Ghana, Mozambique, Rwanda, Tanzania, and Zambia.

RESULTS

Four major overarching lessons were highlighted. First, variety and inclusiveness of concerned key players (public, academic and private) are necessary to address complex health system issues at all levels. Second, a learning culture that promotes evidence creation and ability to efficiently adapt were key in order to meet changing contextual needs. Third, inclusion of strong implementation science tools and strategies allowed informed and measured learning processes and efficient dissemination of best practices. Fourth, five to seven years was the minimum time frame necessary to effectively implement complex health system strengthening interventions and generate the evidence base needed to advocate for sustainable change for the PHIT partnership projects.

CONCLUSION

The AHI experience has raised remaining, if not overlooked, challenges and potential solutions to address complex health systems strengthening intervention designs and implementation issues, while aiming to measurably accomplish sustainable positive change in dynamic, learning, and varied contexts.

Bridging the intervention-implementation gap in primary health care delivery: the critical role of integrated implementation research

For national and local leaders to achieve universal health coverage, a new approach or technique to gathering evidence and understanding the contexts that influence the outcome of a study and goes beyond the quantitative results of clinical trials and pilot projects is important. The Doris Duke Charitable Foundation's African Health Initiative (AHI) was designed to produce this type of knowledge through embedding implementation research into Population Health Implementation and Training (PHIT) partnership projects in five countries (Ghana, Mozambique, Rwanda, Tanzania, and Zambia) with the goal of improving primary health care and population health. In Ghana, this integration of research into implementation has contributed to the successful testing, adaptation and implementation of the Community-based Health Planning and Services (CHPS) model (The Navrongo Pilot Project), with results from the AHI-funded work informing national scale-up of effective practices. Further application of implementation science methods and frameworks to study cross-project lessons also produced the evidence needed by national and local decision makers on how and why different intervention components were successful and where and how local context drove implementation and adaptation. Cross-project research also identified effective approaches across diverse settings for building capacity for data-driven improvement, coaching and mentoring clinicians and researchers, developing locally appropriate interventions to reduce neonatal mortality, and integrating implementation research to inform local implementers and researchers in more effective strategies to strengthen health systems and improve health services and population health. Evidence has already shown the potential for this type of work to accelerate regional learning and spread of successful interventions to achieve targeted health goals more efficiently, better enabling countries to achieve the ambitious, but important, U.N. Sustainable Development Goals.

Mentorship and coaching to support strengthening healthcare systems: lessons learned across the five Population Health Implementation and Training partnership projects in sub-Saharan Africa

Background

Despite global efforts to increase health workforce capacity through training and guidelines, challenges remain in bridging the gap between knowledge and quality clinical practice and addressing health system deficiencies preventing health workers from providing high quality care. In many developing countries, supervision activities focus on data collection, auditing and report completion rather than catalyzing learning and supporting system quality improvement. To address this gap, mentorship and coaching interventions were implemented in projects in five African countries (Ghana, Mozambique, Rwanda, Tanzania, and Zambia) as components of health systems strengthening (HSS) strategies funded through the Doris Duke Charitable Foundation’s African Health Initiative. We report on lessons learned from a cross-country evaluation.

Methods

The evaluation was designed based on a conceptual model derived from the project-specific interventions. Semi-structured interviews were administered to key informants to capture data in six categories: 1) mentorship and coaching goals, 2) selection and training of mentors and coaches, 3) integration with the existing systems, 4) monitoring and evaluation, 5) reported outcomes, and 6) challenges and successes. A review of project-published articles and technical reports from the individual projects supplemented interview information.

Results

Although there was heterogeneity in the approaches to mentorship and coaching and targeted areas of the country projects, all led to improvements in core health system areas, including quality of clinical care, data-driven decision making, leadership and accountability, and staff satisfaction. Adaptation of approaches to reflect local context encouraged their adoption and improved their effectiveness and sustainability.

Conclusion

We found that incorporating mentorship and coaching activities into HSS strategies was associated with improvements in quality of care and health systems, and mentorship and coaching represents an important component of HSS activities designed to improve not just coverage, but even further effective coverage, in achieving Universal Health Care.

Improving data quality across 3 sub-Saharan African countries using the Consolidated Framework for Implementation Research (CFIR): results from the African Health Initiative

Background

High-quality data are critical to inform, monitor and manage health programs. Over the seven-year African Health Initiative of the Doris Duke Charitable Foundation, three of the five Population Health Implementation and Training (PHIT) partnership projects in Mozambique, Rwanda, and Zambia introduced strategies to improve the quality and evaluation of routinely-collected data at the primary health care level, and stimulate its use in evidence-based decision-making. Using the Consolidated Framework for Implementation Research (CFIR) as a guide, this paper: 1) describes and categorizes data quality assessment and improvement activities of the projects, and 2) identifies core intervention components and implementation strategy adaptations introduced to improve data quality in each setting.

Methods

The CFIR was adapted through a qualitative theme reduction process involving discussions with key informants from each project, who identified two domains and ten constructs most relevant to the study aim of describing and comparing each country’s data quality assessment approach and implementation process. Data were collected on each project’s data quality improvement strategies, activities implemented, and results via a semi-structured questionnaire with closed and open-ended items administered to health management information systems leads in each country, with complementary data abstraction from project reports.

Results

Across the three projects, intervention components that aligned with user priorities and government systems were perceived to be relatively advantageous, and more readily adapted and adopted. Activities that both assessed and improved data quality (including data quality assessments, mentorship and supportive supervision, establishment and/or strengthening of electronic medical record systems), received higher ranking scores from respondents.

Conclusion

Our findings suggest that, at a minimum, successful data quality improvement efforts should include routine audits linked to ongoing, on-the-job mentoring at the point of service. This pairing of interventions engages health workers in data collection, cleaning, and analysis of real-world data, and thus provides important skills building with on-site mentoring. The effect of these core components is strengthened by performance review meetings that unify multiple health system levels (provincial, district, facility, and community) to assess data quality, highlight areas of weakness, and plan improvements.