Using health management information system data: case study and verification of institutional deliveries in Ethiopia

Effect of Performance-Based Nonfinancial Incentives on Data Quality in Individual Medical Records of Institutional Births: Quasi-Experimental Study

BACKGROUND

Despite the potential of routine health information systems in tackling persistent maternal deaths stemming from poor service quality at health facilities during and around childbirth, research has demonstrated their suboptimal performance, evident from the incomplete and inaccurate data unfit for practical use. There is a consensus that nonfinancial incentives can enhance health care providers' commitment toward achieving the desired health care quality. However, there is limited evidence regarding the effectiveness of nonfinancial incentives in improving the data quality of institutional birth services in Ethiopia.

OBJECTIVE

This study aimed to evaluate the effect of performance-based nonfinancial incentives on the completeness and consistency of data in the individual medical records of women who availed institutional birth services in northwest Ethiopia.

METHODS

We used a quasi-experimental design with a comparator group in the pre-post period, using a sample of 1969 women's medical records. The study was conducted in the "Wegera" and "Tach-armacheho" districts, which served as the intervention and comparator districts, respectively. The intervention comprised a multicomponent nonfinancial incentive, including smartphones, flash disks, power banks, certificates, and scholarships. Personal records of women who gave birth within 6 months before (April to September 2020) and after (February to July 2021) the intervention were included. Three distinct women's birth records were examined: the integrated card, integrated individual folder, and delivery register. The completeness of the data was determined by examining the presence of data elements, whereas the consistency check involved evaluating the agreement of data elements among women's birth records. The average treatment effect on the treated (ATET), with 95% CIs, was computed using a difference-in-differences model.

RESULTS

In the intervention district, data completeness in women's personal records was nearly 4 times higher (ATET 3.8, 95% CI 2.2-5.5; P=.02), and consistency was approximately 12 times more likely (ATET 11.6, 95% CI 4.18-19; P=.03) than in the comparator district.

CONCLUSIONS

This study indicates that performance-based nonfinancial incentives enhance data quality in the personal records of institutional births. Health care planners can adapt these incentives to improve the data quality of comparable medical records, particularly pregnancy-related data within health care facilities. Future research is needed to assess the effectiveness of nonfinancial incentives across diverse contexts to support successful scale-up.

Effect of the COVID-19 pandemic on HIV, malaria and tuberculosis indicators in Togo: an interrupted time series analysis

BACKGROUND

Limited data are available on the effects of the COVID-19 pandemic on health-related indicators in sub-Saharan Africa. This study aimed to estimate the effect of the COVID-19 pandemic on nine indicators of HIV, malaria and tuberculosis (TB) in Togo.

METHODS

For this interrupted time series analysis, national health information system data from January 2019 to December 2021 and TB programmatic data from the first quarter of 2018 to the fourth quarter of 2022 were analysed. Nine indicators were included. We used Poisson segmented regression to estimate the immediate impact of the pandemic and per-pandemic period trends through incidence rate ratios (IRRs) with 95% CIs.

RESULTS

Overall, there was a decrease in six of the nine indicators, ranging from 19.3% (IRR 0.807, 95% CI 0.682 to 0.955, p=0.024) for the hospitalisation of patients for malaria to 36.9% (IRR 0.631, 95% CI 0.457 to 0.871, p=0.013) for TB diagnosis by Mycobacterium tuberculosis Xpert immediately after the declaration of the COVID-19 pandemic. A comparison of the observed and predicted trends showed that the trend remained constant between the prepandemic and pandemic periods of COVID-19 for all malaria indicators. A significant downward monthly trend was observed in antiretroviral therapy initiation (IRR 0.909, 95% CI 0.892 to 0.926, p<0.001) and positive TB microscopy (IRR 0.919, 95% CI 0.880 to 0.960, p=0.002).

CONCLUSION

HIV, malaria and TB services were generally maintained over time in Togo despite the COVID-19 pandemic. However, given the decline in levels immediately after the onset of the pandemic, there is an urgent need to improve the preparedness of the healthcare system.

Data completeness and consistency in individual medical records of institutional births: retrospective crossectional study from Northwest Ethiopia, 2022

BACKGROUND

Ensuring the data quality of Individual Medical Records becomes a crucial strategy in mitigating maternal and newborn morbidity and mortality during and around childbirth. However, previous research in Ethiopia primarily focused on studying data quality of institutional birth at the facility level, overlooking the data quality within Individual Medical Records. This study examined the data completeness and consistency within Individual Medical Records of the institutional birth service and associated factors.

METHODS

An institution-based retrospective cross-sectional study was conducted in two districts of Northwest Ethiopia. Data were obtained by reviewing three sets of Individual Medical Records of 651 women: the delivery register, Integrated Individual Folder, and integrated card. The proportions of completeness and consistency were computed. A multilevel binary logistic regression was used to identify factors of completeness and consistency. An odds ratio with a 95% confidence interval was used to assess the level of significance.

RESULTS

Overall, 74.0% of women's Individual Medical Records demonstrated good data completeness ( > = 70%), 95%CI (70.5, 77.3), while 26% exhibited good consistency, 95%CI (22.9, 29.7). The presence of trained providers in data quality (AOR = 2.9, 95%CI: (1.5, 5.7)) and supportive supervision (AOR = 11.5, 95%CI: (4.8, 27.2)) were found to be associated with completeness. Health facilities' practice of root cause analysis on data quality gaps (AOR = 8.7, 9%CI: (1.5, 50.9)) was statistically significantly associated with the consistency.

CONCLUSIONS

Most medical records were found to have good completeness, but nearly only a quarter of them found to contain consistent data. Completeness and consistency varied on the type of medical record. Health facility's root cause analysis of data quality gaps, the presence of trained providers in data quality, and supportive supervision from higher officials were identified as factors affecting data quality in institutional birth service. These results emphasize the importance of focused efforts to enhance data completeness and consistency within Individual Medical Records, particularly through consideration of Individual Medical Records in future provider training, supervision, and the implementation of root cause analysis practices.

Quality of routine health data at the onset of the COVID-19 pandemic in Ethiopia, Haiti, Laos, Nepal, and South Africa

BACKGROUND

During the COVID-19 pandemic, governments and researchers have used routine health data to estimate potential declines in the delivery and uptake of essential health services. This research relies on the data being high quality and, crucially, on the data quality not changing because of the pandemic. In this paper, we investigated those assumptions and assessed data quality before and during COVID-19.

METHODS

We obtained routine health data from the DHIS2 platforms in Ethiopia, Haiti, Lao People's Democratic Republic, Nepal, and South Africa (KwaZulu-Natal province) for a range of 40 indicators on essential health services and institutional deaths. We extracted data over 24 months (January 2019-December 2020) including pre-pandemic data and the first 9 months of the pandemic. We assessed four dimensions of data quality: reporting completeness, presence of outliers, internal consistency, and external consistency.

RESULTS

We found high reporting completeness across countries and services and few declines in reporting at the onset of the pandemic. Positive outliers represented fewer than 1% of facility-month observations across services. Assessment of internal consistency across vaccine indicators found similar reporting of vaccines in all countries. Comparing cesarean section rates in the HMIS to those from population-representative surveys, we found high external consistency in all countries analyzed.

CONCLUSIONS

While efforts remain to improve the quality of these data, our results show that several indicators in the HMIS can be reliably used to monitor service provision over time in these five countries.

Tracking health system performance in times of crisis using routine health data: lessons learned from a multicountry consortium

COVID-19 has prompted the use of readily available administrative data to track health system performance in times of crisis and to monitor disruptions in essential healthcare services. In this commentary we describe our experience working with these data and lessons learned across countries. Since April 2020, the Quality Evidence for Health System Transformation (QuEST) network has used administrative data and routine health information systems (RHIS) to assess health system performance during COVID-19 in Chile, Ethiopia, Ghana, Haiti, Lao People's Democratic Republic, Mexico, Nepal, South Africa, Republic of Korea and Thailand. We compiled a large set of indicators related to common health conditions for the purpose of multicountry comparisons. The study compiled 73 indicators. A total of 43% of the indicators compiled pertained to reproductive, maternal, newborn and child health (RMNCH). Only 12% of the indicators were related to hypertension, diabetes or cancer care. We also found few indicators related to mental health services and outcomes within these data systems. Moreover, 72% of the indicators compiled were related to volume of services delivered, 18% to health outcomes and only 10% to the quality of processes of care. While several datasets were complete or near-complete censuses of all health facilities in the country, others excluded some facility types or population groups. In some countries, RHIS did not capture services delivered through non-visit or nonconventional care during COVID-19, such as telemedicine. We propose the following recommendations to improve the analysis of administrative and RHIS data to track health system performance in times of crisis: ensure the scope of health conditions covered is aligned with the burden of disease, increase the number of indicators related to quality of care and health outcomes; incorporate data on nonconventional care such as telehealth; continue improving data quality and expand reporting from private sector facilities; move towards collecting patient-level data through electronic health records to facilitate quality-of-care assessment and equity analyses; implement more resilient and standardized health information technologies; reduce delays and loosen restrictions for researchers to access the data; complement routine data with patient-reported data; and employ mixed methods to better understand the underlying causes of service disruptions.

Development and Testing Requirements for an Integrated Maternal and Child Health Information System in Iran: A Design Thinking Case Study

BACKGROUND

 Management of child health care can be negatively affected by incomplete recording, low data quality, and lack of data integration of health management information systems to support decision making and public health program needs. Given the importance of identifying key determinants of child health via capturing and integrating accurate and high-quality information, we aim to address this gap through the development and testing requirements for an integrated child health information system.

SUBJECTS AND METHODS

 A five-phase design thinking approach including empathizing, defining, ideation, prototyping, and testing was applied. We employed observations and interviews with the health workers at the primary health care network to identify end-users' challenges and needs using tools in human-centered design and focus group discussion. Then, a potential solution to the identified problems was developed as an integrated maternal and child health information system (IMCHIS) prototype and tested using Software Quality Requirements and Evaluation Model (SQuaRE) ISO/IEC 25000.

RESULTS

 IMCHIS was developed as a web-based system with 74 data elements and seven maternal and child health care requirements. The requirements of "child disease" with weight (0.26), "child nutrition" with weight (0.20), and "prenatal care" with weight (0.16) acquired the maximum weight coefficient. In the testing phase, the highest score with the weight coefficient of 0.48 and 0.73 was attributed to efficiency and functionality characteristics, focusing on software capability to fulfill the tasks that meet users' needs.

CONCLUSION

 Implementing a successful child health care system integrates both maternal and child health care information systems to track the effect of maternal conditions on child health and support managing performance and optimizing service delivery. The highest quality score of IMCHIS in efficiency and functionality characteristics confirms that it owns the capability to identify key determinants of child health.

Healthcare utilization and maternal and child mortality during the COVID-19 pandemic in 18 low- and middle-income countries: An interrupted time-series analysis with mathematical modeling of administrative data

BACKGROUND

The Coronavirus Disease 2019 (COVID-19) pandemic has had wide-reaching direct and indirect impacts on population health. In low- and middle-income countries, these impacts can halt progress toward reducing maternal and child mortality. This study estimates changes in health services utilization during the pandemic and the associated consequences for maternal, neonatal, and child mortality.

METHODS AND FINDINGS

Data on service utilization from January 2018 to June 2021 were extracted from health management information systems of 18 low- and lower-middle-income countries (Afghanistan, Bangladesh, Cameroon, Democratic Republic of the Congo (DRC), Ethiopia, Ghana, Guinea, Haiti, Kenya, Liberia, Madagascar, Malawi, Mali, Nigeria, Senegal, Sierra Leone, Somalia, and Uganda). An interrupted time-series design was used to estimate the percent change in the volumes of outpatient consultations and maternal and child health services delivered during the pandemic compared to projected volumes based on prepandemic trends. The Lives Saved Tool mathematical model was used to project the impact of the service utilization disruptions on child and maternal mortality. In addition, the estimated monthly disruptions were also correlated to the monthly number of COVID-19 deaths officially reported, time since the start of the pandemic, and relative severity of mobility restrictions. Across the 18 countries, we estimate an average decline in OPD volume of 13.1% and average declines of 2.6% to 4.6% for maternal and child services. We projected that decreases in essential health service utilization between March 2020 and June 2021 were associated with 113,962 excess deaths (110,686 children under 5, and 3,276 mothers), representing 3.6% and 1.5% increases in child and maternal mortality, respectively. This excess mortality is associated with the decline in utilization of the essential health services included in the analysis, but the utilization shortfalls vary substantially between countries, health services, and over time. The largest disruptions, associated with 27.5% of the excess deaths, occurred during the second quarter of 2020, regardless of whether countries reported the highest rate of COVID-19-related mortality during the same months. There is a significant relationship between the magnitude of service disruptions and the stringency of mobility restrictions. The study is limited by the extent to which administrative data, which varies in quality across countries, can accurately capture the changes in service coverage in the population.

CONCLUSIONS

Declines in healthcare utilization during the COVID-19 pandemic amplified the pandemic's harmful impacts on health outcomes and threaten to reverse gains in reducing maternal and child mortality. As efforts and resource allocation toward prevention and treatment of COVID-19 continue, essential health services must be maintained, particularly in low- and middle-income countries.

Quality of medical records in public health facilities: A case of Southern Ethiopia, resource limited setting

Facility based retrospective study was conducted in three regions in southern Ethiopia to assess quality of medical records. A total of 2,145 medical records were reviewed from 73 public health facilities. Minimum standards of medical records were considered to assess completeness and legibility of records. The completeness of medical records were judged systematically according to national HMIS formats. From total of 2,145 medical cards reviewed, only 394 (18.4%) records had all complete and readable data. Gaps observed include 29.0% missed at least one of identification data, 14.3% lack chief compliant, 20.1% lack diagnosis, 12.5% lack medication and 60.3% records had no date and/or signature. Moreover, 9.5% cards had at least one non-readable component. Records at health centers were 56.8% less likely to be quality record as compared to records in hospitals. Even though completeness of every single record is must, only less than one fifth of records met quality of national medical record standard. Ministry of health should consider rules and regulation to maintain data quality and switching to electronic record, and finally progress in data quality should be monitored routinely.

COVID-19 and resilience of healthcare systems in ten countries

Declines in health service use during the Coronavirus Disease 2019 (COVID-19) pandemic could have important effects on population health. In this study, we used an interrupted time series design to assess the immediate effect of the pandemic on 31 health services in two low-income (Ethiopia and Haiti), six middle-income (Ghana, Lao People's Democratic Republic, Mexico, Nepal, South Africa and Thailand) and high-income (Chile and South Korea) countries. Despite efforts to maintain health services, disruptions of varying magnitude and duration were found in every country, with no clear patterns by country income group or pandemic intensity. Disruptions in health services often preceded COVID-19 waves. Cancer screenings, TB screening and detection and HIV testing were most affected (26-96% declines). Total outpatient visits declined by 9-40% at national levels and remained lower than predicted by the end of 2020. Maternal health services were disrupted in approximately half of the countries, with declines ranging from 5% to 33%. Child vaccinations were disrupted for shorter periods, but we estimate that catch-up campaigns might not have reached all children missed. By contrast, provision of antiretrovirals for HIV was not affected. By the end of 2020, substantial disruptions remained in half of the countries. Preliminary data for 2021 indicate that disruptions likely persisted. Although a portion of the declines observed might result from decreased needs during lockdowns (from fewer infectious illnesses or injuries), a larger share likely reflects a shortfall of health system resilience. Countries must plan to compensate for missed healthcare during the current pandemic and invest in strategies for better health system resilience for future emergencies.

'The false reporter will get a praise and the one who reported truth will be discouraged': a qualitative study on intentional data falsification by frontline maternal and newborn healthcare workers in two regions in Ethiopia

INTRODUCTION

Health Management Information Systems (HMIS) are vital to ensure accountability and for making decisions including for tracking the Sustainable Development Goals. The Ethiopia Health Sector Transformation Plan II includes preventing data falsification as a major strategic initiative and our study aimed to explore the reasons why healthcare providers intentionally falsify maternal and newborn health (MNH) data in two regions of Ethiopia.

METHODS

We conducted a qualitative study in two hospitals, four health centres and their associated health posts in Oromia and Amhara regions. We conducted 45 in-depth interviews with health facility managers, quality improvement (QI) focal persons, health information technicians, MNH care providers, Health Extension Workers and QI mentors. Data were collected in local languages and transcribed in English. During analysis we repeatedly read the transcripts, coded them inductively using NVivo V.12, and categorised the codes into themes.

RESULTS

Participants were hesitant to report personal data falsification but many reported that falsification is common and that they had experienced it in other facilities or had been told about it by other health workers. Falsification was mostly inflating the number of services provided (such as deliveries). Decreasing the number of deaths or reclassifying neonatal death into stillbirths was also reported. An overarching theme was that the health system focuses on, and rewards, the number of services provided over any other metric. This focus led to both system and individual level incentives for falsification and disincentives for accurate reporting.

CONCLUSION

Our finding suggests that to reduce facility level data falsification policy makers might consider disentangling reward and punishments from the performance reports based on the routine HMIS data. Further studies examining the high-level drivers of falsification at regional, national and global levels and effective interventions to address the drivers of data falsification are needed.

Effect of the COVID-19 pandemic on health service utilization across regions of Ethiopia: An interrupted time series analysis of health information system data from 2019-2020

The spread of COVID-19 and associated deaths have remained low in Ethiopia. However, the pandemic could pose a public health crisis indirectly through disruptions in essential health services. The aim of this study was to examine disruptions in health service utilization during the first nine months of the COVID-19 pandemic across 10 regions in Ethiopia. We analyzed utilization of 21 different health services across all of Ethiopia (except the Tigray region) for the period of January 2019 to December 2020. Data were extracted from the Ethiopian district health information system (DHIS2). Monthly visits in 2020 were graphed relative to the same months in 2019. Interrupted time series analysis was used to estimate the effect of the pandemic on service utilization in each region. We found that disruptions in health services were generally higher in urban regions which were most affected by COVID. Outpatient visits declined by 52%, 54%, and 58%, specifically in Dire Dawa, Addis Ababa and Harari, the three urban regions. Similarly, there was a 47% reduction in inpatient admissions in Addis Ababa. In agrarian regions, the pandemic caused an 11% to 17% reduction in outpatient visits and a 10% to 27% decline in inpatient admissions. Visits for children with diarrhea, pneumonia and malnutrition also declined substantially while maternal health services were less affected. Our study indicates that disruptions in health services were more pronounced in areas that were relatively harder hit by the pandemic. Our results show that the Ethiopian health system has a limited capacity to absorb shocks. During future waves of COVID or future pandemics, the Ethiopian health system must be better prepared to maintain essential services and mitigate the indirect impact of the pandemic on public health, particularly in urban areas.