Effect of data quality improvement intervention on health management information system data accuracy: An interrupted time series analysis

'The objective was about not blaming one another': a qualitative study to explore how collaboration is experienced within quality improvement collaboratives in Ethiopia

BACKGROUND

Quality improvement collaboratives are a common approach to improving quality of care. They rely on collaboration across and within health facilities to enable and accelerate quality improvement. Originating in high-income settings, little is known about how collaboration transfers to low-income settings, despite the widespread use of these collaboratives.

METHOD

We explored collaboration within quality improvement collaboratives in Ethiopia through 42 in-depth interviews with staff of two hospitals and four health centers and three with quality improvement mentors. Data were analysed thematically using a deductive and inductive approach.

RESULTS

There was collaboration at learning sessions though experience sharing, co-learning and peer pressure. Respondents were used to a blaming environment, which they contrasted to the open and non-blaming environment at the learning sessions. Respondents formed new relationships that led to across facility practical support. Within facilities, those in the quality improvement team continued to collaborate through the plan-do-study-act cycles, although this required high engagement and support from mentors. Few staff were able to attend learning sessions and within facility transfer of quality improvement knowledge was rare. This affected broader participation and led to some resentment and resistance. Improved teamwork skills and behaviors occurred at individual rather than facility or systems level, with implications for sustainability. Challenges to collaboration included unequal participation, lack of knowledge transfer, high workloads, staff turnover and a culture of dependency.

CONCLUSION

We conclude that collaboration can occur and is valued within a traditionally hierarchical system, but may require explicit support at learning sessions and by mentors. More emphasis is needed on ensuring quality improvement knowledge transfer, buy-in and system level change. This could include a modified collaborative design to provide facility-level support for spread.

Quality of routine health data at the onset of the COVID-19 pandemic in Ethiopia, Haiti, Laos, Nepal, and South Africa

BACKGROUND

During the COVID-19 pandemic, governments and researchers have used routine health data to estimate potential declines in the delivery and uptake of essential health services. This research relies on the data being high quality and, crucially, on the data quality not changing because of the pandemic. In this paper, we investigated those assumptions and assessed data quality before and during COVID-19.

METHODS

We obtained routine health data from the DHIS2 platforms in Ethiopia, Haiti, Lao People's Democratic Republic, Nepal, and South Africa (KwaZulu-Natal province) for a range of 40 indicators on essential health services and institutional deaths. We extracted data over 24 months (January 2019-December 2020) including pre-pandemic data and the first 9 months of the pandemic. We assessed four dimensions of data quality: reporting completeness, presence of outliers, internal consistency, and external consistency.

RESULTS

We found high reporting completeness across countries and services and few declines in reporting at the onset of the pandemic. Positive outliers represented fewer than 1% of facility-month observations across services. Assessment of internal consistency across vaccine indicators found similar reporting of vaccines in all countries. Comparing cesarean section rates in the HMIS to those from population-representative surveys, we found high external consistency in all countries analyzed.

CONCLUSIONS

While efforts remain to improve the quality of these data, our results show that several indicators in the HMIS can be reliably used to monitor service provision over time in these five countries.

Improving birth weight measurement and recording practices in Kenya and Tanzania: a prospective intervention study with historical controls

BACKGROUND

Low birth weight (LBW) is a significant public health concern given its association with early-life mortality and other adverse health consequences that can impact the entire life cycle. In many countries, accurate estimates of LBW prevalence are lacking due to inaccuracies in collection and gaps in available data. Our study aimed to determine LBW prevalence among facility-born infants in selected areas of Kenya and Tanzania and to assess whether the introduction of an intervention to improve the accuracy of birth weight measurement would result in a meaningfully different estimate of LBW prevalence than current practice.

METHODS

We carried out a historically controlled intervention study in 22 health facilities in Kenya and three health facilities in Tanzania. The intervention included: provision of high-quality digital scales, training of nursing staff on accurate birth weight measurement, recording and scale calibration practices, and quality maintenance support that consisted of enhanced supervision and feedback (prospective arm). The historically controlled data were birth weights from the same facilities recorded in maternity registers for the same calendar months from the previous year measured using routine practices and manual scales. We calculated mean birth weight (95% confidence interval CI), mean difference in LBW prevalence, and respective risk ratio (95% CI) between study arms.

RESULTS

Between October 2019 and February 2020, we prospectively collected birth weights from 8441 newborns in Kenya and 4294 in Tanzania. Historical data were available from 9318 newborns in Kenya and 12,007 in Tanzania. In the prospective sample, the prevalence of LBW was 12.6% (95% confidence intervals [CI]: 10.9%-14.4%) in Kenya and 18.2% (12.2%-24.2%) in Tanzania. In the historical sample, the corresponding prevalence estimates were 7.8% (6.5%-9.2%) and 10.0% (8.6%-11.4%). Compared to the retrospective sample, the LBW prevalence in the prospective sample was 4.8% points (3.2%-6.4%) higher in Kenya and 8.2% points (2.3%-14.0%) higher in Tanzania, corresponding to a risk ratio of 1.61 (1.38-1.88) in Kenya and 1.81 (1.30-2.52) in Tanzania.

CONCLUSION

Routine birth weight records underestimate the risk of LBW among facility-born infants in Kenya and Tanzania. The quality of birth weight data can be improved by a simple intervention consisting of provision of digital scales and supportive training.

'The false reporter will get a praise and the one who reported truth will be discouraged': a qualitative study on intentional data falsification by frontline maternal and newborn healthcare workers in two regions in Ethiopia

INTRODUCTION

Health Management Information Systems (HMIS) are vital to ensure accountability and for making decisions including for tracking the Sustainable Development Goals. The Ethiopia Health Sector Transformation Plan II includes preventing data falsification as a major strategic initiative and our study aimed to explore the reasons why healthcare providers intentionally falsify maternal and newborn health (MNH) data in two regions of Ethiopia.

METHODS

We conducted a qualitative study in two hospitals, four health centres and their associated health posts in Oromia and Amhara regions. We conducted 45 in-depth interviews with health facility managers, quality improvement (QI) focal persons, health information technicians, MNH care providers, Health Extension Workers and QI mentors. Data were collected in local languages and transcribed in English. During analysis we repeatedly read the transcripts, coded them inductively using NVivo V.12, and categorised the codes into themes.

RESULTS

Participants were hesitant to report personal data falsification but many reported that falsification is common and that they had experienced it in other facilities or had been told about it by other health workers. Falsification was mostly inflating the number of services provided (such as deliveries). Decreasing the number of deaths or reclassifying neonatal death into stillbirths was also reported. An overarching theme was that the health system focuses on, and rewards, the number of services provided over any other metric. This focus led to both system and individual level incentives for falsification and disincentives for accurate reporting.

CONCLUSION

Our finding suggests that to reduce facility level data falsification policy makers might consider disentangling reward and punishments from the performance reports based on the routine HMIS data. Further studies examining the high-level drivers of falsification at regional, national and global levels and effective interventions to address the drivers of data falsification are needed.

"Even though they insult us, the delivery they give us is the greatest thing": a qualitative study contextualizing women's experiences with facility-based maternal health care in Ethiopia

BACKGROUND

Globally, amidst increased utilization of facility-based maternal care services, there is continued need to better understand women's experience of care in places of birth. Quantitative surveys may not sufficiently characterize satisfaction with maternal healthcare (MHC) in local context, limiting their interpretation and applicability. The purpose of this study is to untangle how contextual and cultural expectations shape women's care experience and what women mean by satisfaction in two Ethiopian regions.

METHODS

Health center and hospital childbirth care registries were used to identify and interview 41 women who had delivered a live newborn within a six-month period. We used a semi-structured interview guide informed by the Donabedian framework to elicit women's experiences with MHC and delivery, any prior delivery experiences, and recommendations to improve MHC. We used an inductive analytical approach to compare and contrast MHC processes, experiences, and satisfaction.

RESULTS

Maternal and newborn survival and safety were central to women's descriptions of their MHC experiences. Women nearly exclusively described healthy and safe deliveries with healthy outcomes as 'satisfactory'. The texture behind this 'satisfaction', however, was shaped by what mothers bring to their delivery experiences, creating expectations from events including past births, experiences with antenatal care, and social and community influences. Secondary to the absence of adverse outcomes, health provider's interpersonal behaviors (e.g., supportive communication and behavioral demonstrations of commitment to their births) and the facility's amenities (e.g., bathing, cleaning, water, coffee, etc) enhanced women's experiences. Finally, at the social and community levels, we found that family support and material resources may significantly buffer against negative experiences and facilitate women's overall satisfaction, even in the context of poor-quality facilities and limited resources.

CONCLUSION

Our findings highlight the importance of understanding contextual factors including past experiences, expectations, and social support that influence perceived quality of MHC and the agency a woman has to negotiate her care experience. Our finding that newborn and maternal survival primarily drove women's satisfaction suggests that quantitative assessments conducted shortly following delivery may be overly influenced by these outcomes and not fully capture the complexity of women's care experience.

Assessing routine health information system performance during the tenth outbreak of Ebola virus disease (2018-2020) in the Democratic Republic of the Congo: A qualitative study in North Kivu

The Democratic Republic of Congo has implemented reforms to its national routine health information system (RHIS) to improve timeliness, completeness, and use of quality data. However, outbreaks can undermine efforts to strengthen it. We assessed the functioning of the RHIS during the 2018-2020 outbreak of Ebola Virus Disease (EVD) to identify opportunities for future development. We conducted a qualitative study in North Kivu, from March to May 2020. Semi-structured interviews were conducted with 34 key informants purposively selected from among the personnel involved in the production of RHIS data. The topics discussed included RHIS functioning, tools, compilation, validation, quality, sharing, and the use of data. Audio recordings were transcribed verbatim and thematic analysis was used to study the interviewees' lived experience. The RHIS retained its structure, tools, and flow during the outbreak. The need for other types of data to inform the EVD response created other parallel systems to the RHIS. This included data from Ebola treatment centers, vaccination against Ebola, points of entry surveillance, and safe and dignified burial. The informants indicated that the availability of weekly surveillance data had improved, while timeliness and quality of monthly RHIS reporting declined. The compilation of data was late and validation meetings were irregular. The upsurge of patients following the implementation of the free care policy, the departure of healthcare workers for better-paid jobs, and the high prioritization of the outbreak response over routine activities led to RHIS disruptions. Delays in decision-making were one of the consequences of the decline in data timeliness. Adequate allocation of human resources, equitable salary policy, coordination, and integration of the response with local structures are necessary to ensure optimal functioning of the RHIS during an outbreak. Future research should assess the scale of data quality changes during outbreaks.

Using health management information system data: case study and verification of institutional deliveries in Ethiopia

Health management information systems (HMIS) are a crucial source of timely health statistics and have the potential to improve reporting in low-income countries. However, concerns about data quality have hampered their widespread adoption in research and policy decisions. This article presents results from a data verification study undertaken to gain insights into the quality of HMIS data in Ethiopia. We also provide recommendations for working with HMIS data for research and policy translation. We linked the HMIS to the 2016 Emergency Obstetric and Newborn Care Assessment, a national census of all health facilities that provided maternal and newborn health services in Ethiopia. We compared the number of visits for deliveries and caesarean sections (C-sections) reported in the HMIS in 2015 (January–December) to those found in source documents (paper-based labour and delivery and operating theatre registers) in 2425 facilities across Ethiopia. We found that two-thirds of facilities had ‘good’ HMIS reporting for deliveries (defined as reporting within 10% of source documents) and half had ‘very good’ reporting (within 5% of source documents). Results were similar for reporting on C-section deliveries. We found that good reporting was more common in urban areas (OR: 1.30, 95% CI 1.06 to 1.59), public facilities (OR: 2.95, 95% CI 1.38 to 6.29) and in hospitals compared with health centres (OR: 1.71, 95% CI 1.13 to 2.61). Facilities in the Somali and Afar regions had the lowest odds of good reporting compared with Addis Ababa and were more likely to over-report deliveries in the HMIS. Further work remains to address remaining discrepancies in the Ethiopian HMIS. Nonetheless, our findings corroborate previous data verification exercises in Ethiopia and support greater use and uptake of HMIS data for research and policy decisions (particularly, greater use of HMIS data elements (eg, absolute number of services provided each month) rather than coverage indicators). Increased use of these data, combined with feedback mechanisms, is necessary to maintain data quality.

A Conceptual Framework of Data Readiness: The Contextual Intersection of Quality, Availability, Interoperability, and Provenance

BACKGROUND

Data readiness is a concept often used when referring to health information technology applications in the informatics disciplines, but it is not clearly defined in the literature. To avoid misinterpretations in research and implementation, a formal definition should be developed.

OBJECTIVES

The objective of this research is to provide a conceptual definition and framework for the term data readiness that can be used to guide research and development related to data-based applications in health care.

METHODS

PubMed, the National Institutes of Health RePORTER, Scopus, the Cochrane Library, and Duke University Library databases for business and information sciences were queried for formal mentions of the term "data readiness." Manuscripts found in the search were reviewed, and relevant information was extracted, evaluated, and assimilated into a framework for data readiness.

RESULTS

Of the 264 manuscripts found in the database searches, 20 were included in the final synthesis to define data readiness. In these 20 manuscripts, the term data readiness was revealed to encompass the constructs of data quality, data availability, interoperability, and data provenance.

DISCUSSION

Based upon our review of the literature, we define data readiness as the application-specific intersection of data quality, data availability, interoperability, and data provenance. While these concepts are not new, the combination of these factors in a novel data readiness model may help guide future informatics research and implementation science.

CONCLUSION

This analysis provides a definition to guide research and development related to data-based applications in health care. Future work should be done to validate this definition, and to apply the components of data readiness to real-world applications so that specific metrics may be developed and disseminated.

Routine health management information system data in Ethiopia: consistency, trends, and challenges

Background: Ethiopia is investing in the routine Health Management Information System. Improved routine data are needed for decision-making in the health sector. Objective: To analyse the quality of the routine Health Management Information System data and triangulate with other sources, such as the Demographic and Health Surveys. Methods: We analysed national Health Management Information System data on 19 indicators of maternal health, neonatal survival, immunization, child nutrition, malaria, and tuberculosis over the 2012-2018 time period. The analyses were conducted by 38 analysts from the Ministry of Health, Ethiopia, and two government agencies who participated in the Operational Research and Coaching for Analysts (ORCA) project between June 2018 and June 2020. Using a World Health Organization Data Quality Review toolkit, we assessed indicator definitions, completeness, internal consistency over time and between related indicators, and external consistency compared with other data sources. Results: Several services reported coverage of above 100%. For many indicators, denominators were based on poor-quality population data estimates. Data on individual vaccinations had relatively good internal consistency. In contrast, there was low external consistency for data on fully vaccinated children, with the routine Health Management Information System showing 89% coverage but the Demographic and Health Survey estimate at 39%. Maternal health indicators displayed increasing coverage over time. Indicators on child nutrition, malaria, and tuberculosis were less consistent. Data on neonatal mortality were incomplete and operationalised as mortality on day 0-6. Our comparisons with survey and population projections indicated that one in eight early neonatal deaths were reported in the routine Health Management Information System. Data quality varied between regions. Conclusions: The quality of routine data gathered in the health system needs further attention. We suggest regular triangulation with data from other sources. We recommend addressing the denominator issues, reducing the complexity of indicators, and aligning indicators to international definitions.