Understanding the challenges associated with the use of data from routine health information systems in low- and middle-income countries: A systematic review

Accessing the Impact of TikTok's Algorithm on Regional Inequality in Health Information

This study aims to audit the potential algorithmic bias in TikTok's health-related video recommendation toward geographically diverse groups in China. We employed 120 cloud phones and conducted two agent-based testing experiments simulating users' geographical locations and online behaviors. The results indicated significant regional inequality in video sources recommended by the TikTok algorithm, t(118) = 3.02, p = .003, with users from developed cities encountering a higher proportion of professional videos than those from underdeveloped cities. However, when users from both regions expressed a similar preference for the same type of information, an equal proportion of professional videos was recommended. Our findings suggest that widely used algorithms may covertly perpetuate social inequities and reinforce preexisting class-based inequalities, particularly affecting vulnerable population from low-income regions. This study also highlights the importance of enhancing eHealth literacy among disadvantaged users to mitigate problematic outcomes in the AI-based communication landscape.

What could improve surgical data system at health facilities with high surgical volume in Ethiopia?

BACKGROUND

The effective management of surgical and anesthesia care relies on quality data and its readily availability for both patient-centered decision-making and facility-level improvement efforts. Recognizing this critical need, the Strengthening Systems for Improved Surgical Outcomes (SSISO) project addressed surgical care data management and information use practices across 23 health facilities from October 2019 to September 2022. This study aimed to evaluate the effectiveness of SSISO interventions in enhancing practices related to surgical data capture, reporting, analysis, and visualization.

METHODS

This study employed a mixed method, pre- post intervention evaluation design to assess changes in data management and utilization practices at intervention facilities. The intervention packages included capacity building trainings, monthly mentorship visits facilitated by a hub-and-spoke approach, provision of data capture tools, and reinforcement of performance review teams. Data collection occurred at baseline (February - April 2020) and endline (April - June 2022). The evaluation focused on the availability and appropriate use of data capture tools, as well as changes in performance review practices. Appropriate use of registers was defined as filling all the necessary data onto the registers, and this was verified by completeness of selected key data elements in the registers.

RESULTS

The proportion of health facilities with Operation Room (OR) scheduling, referral, and surgical site infection registers significantly increased by 34.8%, 56.5% and 87%, respectively, at project endline compared to baseline. Availability of OR and Anesthesia registers remained high throughout the project, at 91.3% and 95.6%, respectively. Furthermore, the appropriate use of these registers improved, with statistically significant increases observed for OR scheduling registers (34.8% increase). Increases were also noted for OR register (9.5% increase) and anesthesia register (4.5% increase), although not statistically significant. Assessing the prior three months reports, the report submissions to the Ministry of Health/Regional Health Bureau (MOH/RHB) rose from 85 to 100%, reflecting complete reporting at endline period. Additionally, the proportion of surgical teams analyzing and displaying data for informed decision-making significantly increased from 30.4% at baseline to 60.8% at endline period.

CONCLUSION

The implemented interventions positively impacted surgical data management and utilization practice at intervention facilities. These positive changes were likely attributable to capacity building trainings and regular mentorship visits via hub-and-spoke approach. Hence, we recommend further investigation into the effectiveness of similar intervention packages in improving surgical data management, data analysis and visualization practices in low- and middle-income country settings.

Measuring progress on health and well-being in the Eastern Mediterranean Region via voluntary national reviews, 2016-2021: What does the data reveal?

BACKGROUND

Country submission of Voluntary National Reviews is the formal mechanism to report on progress of the Sustainable Development Goals (SDGs). Despite strong political commitment to strong information systems, large data gaps exist in the Eastern Mediterranean Region.

METHODS

This study aims to review reports submitted by countries in the region to assess the comprehensiveness of reporting on the health-reported SDG targets and indicators. We conducted a content analysis of reports submitted between 2016 and 2021 of 18 countries of the region. The review focused on progress on the SDGs by assessing i) the reporting on the 50 health-related targets and indicators ii) data availability using the WHO reporting framework, and iii) data availability based on source of information. Spreadsheets were developed and used to extract data and facilitate content analysis.

RESULTS

All reports confirmed that SDG monitoring and reporting mechanisms have been established, however, only 11 reported on all 17 SDGs and 8 explicitly mentioned country specific 2030 targets. Many reports identified data availability as a key challenge to SDG monitoring; for the health SDG, data availability ranged from 48% to 93% among the five countries reporting this figure. Comprehensiveness of reporting varied by type of indicator (maternal, child and infant mortality were the most common) and by country income level (greater reporting by high income countries).

CONCLUSIONS

Significant work remains to enhance information systems across the region to monitor progress and guide actions to achieve the health-related SDGs. Strengthening health information systems regulatory frameworks, data collection capacities including strengthening civil registration and vital statistics and population-based surveys are key steps to enhancing access to quality data which in turn can contribute to achieving the health-related SDGs.

Cross-border data sharing through the lens of research ethics committee members in sub-Saharan Africa

BACKGROUND

Several factors thwart successful data sharing-ambiguous or fragmented regulatory landscapes, conflicting institutional/researcher interests and varying levels of data science-related expertise are among these. Traditional ethics oversight mechanisms and practices may not be well placed to guarantee adequate research oversight given the unique challenges presented by digital technologies and artificial intelligence (AI). Data-intensive research has raised new, contextual ethics and legal challenges that are particularly relevant in an African research setting. Yet, no empirical research has been conducted to explore these challenges.

MATERIALS AND METHODS

We explored REC members' views and experiences on data sharing by conducting 20 semi-structured interviews online between June 2022 and February 2023. Using purposive sampling and snowballing, we recruited representatives across sub-Saharan Africa (SSA). We transcribed verbatim and thematically analysed the data with Atlas.ti V22.

RESULTS

Three dominant themes were identified: (i) experiences in reviewing data sharing protocols, (ii) perceptions of data transfer tools and (iii) ethical, legal and social challenges of data sharing. Several sub-themes emerged as: (i.a) frequency of and approaches used in reviewing data sharing protocols, (i.b) practical/technical challenges, (i.c) training, (ii.a) ideal structure of data transfer tools, (ii.b) key elements of data transfer tools, (ii.c) implementation level, (ii.d) key stakeholders in developing and reviewing a data transfer agreement (DTA), (iii.a) confidentiality and anonymity, (iii.b) consent, (iii.c) regulatory frameworks, and (iii.d) stigmatisation and discrimination.

CONCLUSIONS

Our results indicated variability in REC members' perceptions, suboptimal awareness of the existence of data protection laws and a unanimously expressed need for REC member training. To promote efficient data sharing within and across SSA, guidelines that incorporate ethical, legal and social elements need to be developed in consultation with relevant stakeholders and field experts, along with the training accreditation of REC members in the review of data-intensive protocols.

How Can Community Data Be Leveraged to Advance Primary Health Care? A Scoping Review of Community-Based Health Information Systems

BACKGROUND

Community-based health information systems (CBISs) can provide critical insights into how community health systems function, and digitized CBISs may improve the quality of community-level data and facilitate integration and use of CBISs within the broader health system. This scoping review aims to understand how CBISs have been implemented, integrated, and used to support community health outcomes in low- and middle-income countries (LMICs).

METHODS

Both peer-reviewed and gray literature were included; relevant articles were identified using key terms and controlled vocabulary related to community/primary health care, health information systems, digital health, and LMICs. A total of 11,611 total records were identified from 5 databases and the gray literature. After deduplication, 6,985 peer-reviewed/gray literature were screened, and 95 articles/reports were included, reporting on 105 CBIS implementations across 38 countries.

RESULTS

Findings show that 55% of CBISs included some level of digitization, with just 28% being fully digitized (for data collection and reporting). Data flow from the community level into the health system varied, with digitized CBISs more likely to reach national-level integration. National-level integration was primarily seen among vertical CBISs. Data quality challenges were present in both paper-based and digitized CBISs, exacerbated by fragmentation of the community health landscape with often parallel reporting systems. CBIS data use was constrained to mostly vertical and digitized (partially or fully) CBISs at national/subnational levels.

CONCLUSION

Digitization can play a pivotal role in strengthening CBIS use, but findings demonstrate that CBISs are only as effective as the community health systems they are embedded within. Community-level data are often not being integrated into national/subnational health information systems, undermining the ability to understand what the community health needs are. Furthermore, stronger investments within community health systems need to be in place broadly to reduce fragmentation and provide stronger infrastructural and systemic support to the community health workforce.

Effect of Performance-Based Nonfinancial Incentives on Data Quality in Individual Medical Records of Institutional Births: Quasi-Experimental Study

BACKGROUND

Despite the potential of routine health information systems in tackling persistent maternal deaths stemming from poor service quality at health facilities during and around childbirth, research has demonstrated their suboptimal performance, evident from the incomplete and inaccurate data unfit for practical use. There is a consensus that nonfinancial incentives can enhance health care providers' commitment toward achieving the desired health care quality. However, there is limited evidence regarding the effectiveness of nonfinancial incentives in improving the data quality of institutional birth services in Ethiopia.

OBJECTIVE

This study aimed to evaluate the effect of performance-based nonfinancial incentives on the completeness and consistency of data in the individual medical records of women who availed institutional birth services in northwest Ethiopia.

METHODS

We used a quasi-experimental design with a comparator group in the pre-post period, using a sample of 1969 women's medical records. The study was conducted in the "Wegera" and "Tach-armacheho" districts, which served as the intervention and comparator districts, respectively. The intervention comprised a multicomponent nonfinancial incentive, including smartphones, flash disks, power banks, certificates, and scholarships. Personal records of women who gave birth within 6 months before (April to September 2020) and after (February to July 2021) the intervention were included. Three distinct women's birth records were examined: the integrated card, integrated individual folder, and delivery register. The completeness of the data was determined by examining the presence of data elements, whereas the consistency check involved evaluating the agreement of data elements among women's birth records. The average treatment effect on the treated (ATET), with 95% CIs, was computed using a difference-in-differences model.

RESULTS

In the intervention district, data completeness in women's personal records was nearly 4 times higher (ATET 3.8, 95% CI 2.2-5.5; P=.02), and consistency was approximately 12 times more likely (ATET 11.6, 95% CI 4.18-19; P=.03) than in the comparator district.

CONCLUSIONS

This study indicates that performance-based nonfinancial incentives enhance data quality in the personal records of institutional births. Health care planners can adapt these incentives to improve the data quality of comparable medical records, particularly pregnancy-related data within health care facilities. Future research is needed to assess the effectiveness of nonfinancial incentives across diverse contexts to support successful scale-up.

COVID-19 vaccine hesitancy and conspiracy beliefs in Togo: Findings from two cross-sectional surveys

Togo is a low-income country in West Africa. Estimates from Our World in Data suggest that only 25% of the Togolese population have received at least one dose of any COVID-19 vaccine by June 2023. Whilst the early phase of the pandemic vaccine rollout across 2021 was dominated by higher-income countries taking much of the available supply, there have long been sufficient supplies for all nations. Thus, there remains a need to understand reasons for low uptake in countries such as Togo, here focusing on population confidence and trust, essentially characteristics that could potentially be addressed within health promotion strategies. Two cross-sectional telephone surveys of Togo residents were conducted in December 2020 and January 2022. These surveys asked questions around perceptions of COVID-19, trust in public health messaging, belief in conspiracy theories, and hesitancy around COVID-19 vaccination. Analyses here focus on unvaccinated respondents. Across Survey 1 (N = 1430) and Survey 2 (N = 212), 65% of respondents were men, and 47% lived in Lomé (capital city of Togo). Between Surveys 1 and 2, overall hesitancy (33.0% to 58.0% respectively) and beliefs in conspiracy theories (29% to 65%) significantly increased. Using logistics regression, governmental mistrust was the strongest significant predictor of hesitancy (OR: 2.90). Participants who indicated agreement or uncertainty with at least one conspiracy belief also predicted greater vaccine hesitancy (OR: 1.36). Proactive approaches to public health messaging, that better understand reasons for hesitancy across different demographics, can support uptake of COVID-19 vaccinations within Togo. This includes health promotion campaigns that use locally and nationally trusted knowledge providers (e.g. the health service or religious leaders) for greatest effectiveness at reducing impact of misinformation. Key future research should focus around knowledge gaps and areas of mistrust created by the pandemic, such as the impact of misinformation upon routine immunisation uptake.

Challenges and successes in implementing an integrated electronic patient record (HIVE) at the Manchester University National Health Service Foundation Trust, England: 1000+ legacy systems, 10 hospitals, one electronic patient record

BACKGROUND

The Manchester University National Health Service (NHS) Foundation Trust (MFT) is one of the largest NHS Trusts in England. Historically, the Trust has had very mixed clinical record keeping, including over 1000 individual information systems. None of these health information technology systems had the full functionality of an integrated electronic patient record (EPR). MFT evolved to its current size and complexity with a vision to improve patient care in Greater Manchester by adopting a Trust-wide EPR. The EPR "Go Live" occurred in September 2022.

AIM

To describe the process of EPR integration as it reflected and impacted upon MFT's health information management (HIM) teams.

METHOD

MFT worked through a 2-year readiness program of work. This included technical readiness, software development and migration planning. Migration of data from the approximately 1000 systems was a major undertaking, during which access to the clinical history and ongoing operational reporting needed to be maintained. Pre-implementation requirements were outlined, a change management program was implemented, and the overall implementation was managed to tight timelines.

DISCUSSION

"Go Live" was achieved for the EPIC EPR product (HIVE) within MFT. Legacy systems are still in the process of being decommissioned and staff are transacting within HIVE. Significant changes in processes and reporting continue to be made, despite some challenges.

CONCLUSION

The Trust delivered the single largest EPIC European "Go live." Lessons learnt continue to be identified. The impact of what the EPR means for the HIM function is described.

Effectiveness of hospital-to-home transitional care interventions and consultation for implementation in Sudan: a scoping review of systematic reviews

Background

Hospital discharge is often associated with a lack of continuity resulting in fragmented care, particularly in low-income countries. As there is limited information about interventions in these countries and no study evaluating the effectiveness of hospital discharge interventions, we conducted a scoping review to identify effective hospital-to-home transitional care interventions and explore their applicability in a low-income country (Sudan).

Methods

Our scoping review of systematic reviews and meta-analyses classed interventions as effective, ineffective, undesirable, or uncertain, based on the quality of their evidence and their estimated effects on the following outcomes: readmission rates, mortality, costs, quality of life, and adverse outcomes) and certainty of evidence. Our authors from Sudan used the SUPPORT summary tool to determine if three effective interventions could be implemented in Sudan.

Results

Out of 3,276 articles that were identified, and 72 articles were reviewed, 10 articles has been included in the review. Seven interventions were classified as effective, one as ineffective, and none with undesirable effects. Eight interventions were classified as having an uncertain effect. The effective interventions were composed of home visits, information and communication technology (ICT), case manager models, multidisciplinary teams, and self-management support.

Conclusions

The finding of this study suggested that a combining two to four interventions can improve enhance hospital-to-home transitional care. Effective interventions are composed of home visits, ICT, case manager models, multidisciplinary teams, and self-management support. The implementation of these interventions in Sudan was found to be undermined by contextual factors such as inadequate human resources, telecommunication instability, and inequality in accessibility. These interventions could be tailored based on an in-depth understanding of the contextual factors in low-income countries that influence implementation.

Systematic Review Registration

https://osf.io/9eqvr/, doi: 10.17605/OSF.IO/9EQVR.

The HIV/AIDS responses pre and during the COVID-19 pandemic in sub-Saharan Africa: A basis for sustainable health system strengthening post-COVID-19

In May 2023, the World Health Organization announced that COVID-19 was no longer a global emergency. The impact of COVID-19 on the provision of HIV/AIDS services was not that severe due to the reprogramming of key resources. For health systems in sub-Saharan Africa to become resilient to future pandemics, lessons should be learned from the successful HIV/AIDS response in the region, and how the HIV/AIDS services were successfully adapted to the COVID-19 pandemic. In this article, we reviewed the 6 World Health Organization health system building blocks on how the best practices from the provision of HIV/AIDS services and the services' response to the COVID-19 pandemic can be used as a basis for restoring and strengthening health systems to offer universal access to quality essential health services. The success of the leadership and governance for universal access to anti-retroviral therapy can be a blueprint for the realization of universal health coverage. Significant efficiencies that resulted in the reduction in anti-retroviral therapy costs can be leveraged to ensure cheaper essential drugs while differentiated service delivery models can be used to improve health service accessibility. New technologies that have proven to be successful in HIV/AIDS care can also be used in the care of other diseases, including disease outbreaks. The strong health information systems developed for HIV programs can be used as a foundation for developing health information systems for the whole health sector while the healthcare professionals trained for the provision of HIV/AIDS services can be trained to provide services for a variety of other conditions.

Identifying barriers to the production and use of routine health information in Western Province, Zambia

Recent decades of improvements to routine health information systems in low- and middle-income countries (LMICs) have increased the volume of health data collected. However, countries continue to face several challenges with quality production and use of information for decision-making at sub-national levels, limiting the value of health information for policy, planning and research. Improving the quality of data production and information use is thus a priority in many LMICs to improve decision-making and health outcomes. This qualitative study identified the challenges of producing and using routine health information in Western Province, Zambia. We analysed the interview responses from 37 health and social sector professionals at the national, provincial, district and facility levels to understand the barriers to using data from the Zambian health management information system (HMIS). Respondents raised several challenges that we categorized into four themes: governance and health system organization, geographic barriers, technical and procedural barriers, and challenges with human resource capacity and staff training. Staff at the facility and district levels were arguably the most impacted by these barriers as they are responsible for much of the labour to collect and report routine data. However, facility and district staff had the least authority and ability to mitigate the barriers to data production and information use. Expectations for information use should therefore be clearly outlined for each level of the health system. Further research is needed to understand to what extent the available HMIS data address the needs and purposes of the staff at facilities and districts.

Regular measurement is essential but insufficient to improve quality of healthcare

Ambrose Agweyu and colleagues argue that large scale improvements in quality of healthcare require strong change management as well as health information systems that can provide continuous and rapid feedback

Routine use of DHIS2 data: a scoping review

Background

In regard to health service planning and delivery, the use of information at different levels in the health system is vital, ranging from the influencing of policy to the programming of action to the ensuring of evidence-informed practices. However, neither ownership of, nor access to, good quality data guarantees actual use of these data. For information to be used, relevant data need to be collected, processed and analysed in an accessible format. This problem of underused data, and indeed the absence of data use entirely, is widespread and has been evident for decades.

The DHIS2 software platform supports routine health management for an estimated 2.4 billion people, in over 70 countries worldwide. It is by far the largest and most widespread software for this purpose and adopts a holistic, socio-technical approach to development and implementation. Given this approach, and the rapid and extensive scaling of DHIS2, we questioned whether or not there has been a parallel increase in the scaling of improved information use. To date, there has been no rigorous review of the documentation on how exactly DHIS2 data is routinely being used for decision-making and subsequent programming of action. This scoping review addresses this review gap.

Methods

The five-stage approach of Arksey and O’Malley progressed by Levac et al. and Peters was followed. Three databases (PubMed, Web of Science and Embase) were searched, along with relevant conference proceedings and postgraduate theses. In total, over 500 documents were reviewed and data from 19 documents were extracted.

Results

Overall, DHIS2 data are being used but there are few detailed descriptions of this usage in peer reviewed or grey literature. We find that, commonly, there exists a centralised versus decentralised pattern of use in terms of access to data and the reporting of data ‘up’ in the system. We also find that the different conceptualisations of data use and how data use is conceptualised are not made explicit.

Conclusions

We conclude with some suggestions for a way forward, namely: i) the need to document in more detail and share how data are being used, ii) the need to investigate how data were created and who uses such data, iii) the need to design systems based on work practices, and in tandem develop and promote forums in which ‘conversations’ around data can take place.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08598-8.

Assessing the Use of Hospital Information Systems (HIS) to Support Decision-Making: A Cross-Sectional Study in Public Hospitals in the Huíla Health Region of Southern Angola

Hospital information systems could be relevant tools to inform hospital managers, support better management decisions in healthcare, and increase efficiency. Nonetheless, hospital managers’ effective use of these systems to support decision-making in Angola is unknown. Our study aimed to analyse the use of hospital information systems as a tool to support decision-making by hospital managers in Huíla, Angola. It was a descriptive, cross-sectional study inducted between July and September 2017 in seven hospitals in Huíla Province, Angola, specifically in the cities of Lubango and Matala. Thirty-six members of the hospital boards filled out a self-questionnaire that consisted of twenty questions based on the following issues: Characterisation of the interviewee’s profile; availability of information in the institution; and quality and usefulness of the available operational information. At least two thirds of the participants reported being unsatisfied or relatively satisfied with each assessed hospital information systems-specific feature. More than 50% have rarely or never used the health information system to support decision-making. Most managers do not use hospital information systems to support management-related decision-making in Angola. Improving the ability of hospital information systems to compute adequate indicators and training for hospital managers could be targets for future interventions to support better management-related decision-making in Angolan healthcare.

Behavioral Determinants of Routine Health Information System Data Use in Senegal: A Qualitative Inquiry Based on the Integrated Behavioral Model

Although behavioral factors are thought to be important barriers to routine data use, they remain understudied particularly in low-income country settings. We show that the integrated behavior model can be a valuable theoretical framework for targeted communication strategies and capacity-building interventions aimed at promoting a culture of data use.

Routine health information system (RHIS) data are essential in driving decision making and planning in health systems as well as health programs. However, despite their importance, these data are underutilized, and the underlying individual-level facilitators and barriers to use remain understudied. In this research, we applied the Integrated Behavior Model (IBM) to examine how attitudes toward RHIS data, perceived norms concerning RHIS data use, and the ability to use RHIS data influence the demand and use of RHIS data among stakeholders in Senegal. Using data from interviews with respondents working at national levels of malaria, HIV, and TB control programs in Senegal, we used a framework analysis approach to apply the IBM behavioral constructs and identify their linkages to RHIS data use. We found that attitudes about the quality, availability, and relevance of RHIS data for decision making were important in driving data use among respondents. Institutional expectations, organizational protocols, policies, and practices around RHIS data ultimately shape social norms around the use of the data. Although we found that perceived ability and self-efficacy to use RHIS data were not barriers to RHIS data use among stakeholders at the strategic levels of their respective organizations, these were reported to be barriers at lower levels of the health system. Low perceived control of the RHIS data production process ultimately reduced RHIS data use for decision making among the strategic-level respondents. We recommend context-specific reexamination of existing RHIS interventions with a renewed emphasis on behavioral aspects of data use. The IBM can help guide practitioners, policy makers, and academics to address multiple socioecological factors that influence data use behavior when recommending RHIS and data use solutions.

Framework synthesis to inform the ideation and design of a paper-based health information system (PHISICC)

Background

Health information systems (HIS) are meant to support decision‐making at all levels of the system, including frontline health workers. In field studies in Côte d’Ivoire, Mozambique and Nigeria, we observed health workers' interactions with the HIS and identified twelve decision‐making components of HIS. The objective of this framework synthesis is to portray these components in HIS research, in order to inform the ideation of a paper‐based HIS intervention (PHISICC).

Methods

We searched studies in the Cochrane Central Register of Controlled Trials (CENTRAL), Embase, Epistemonikos, Medline, in‐Process on the Ovid platform, OpenGrey, PDQ  Evidence (“pretty darnd quick” Evidence), the World Health Organization (WHO) Global Health Library and included studies focussing on HIS interventions, data quality, information support tools and data use for decision‐making in the context of the governmental health care sector. We assessed the methodological quality of studies using the Critical Appraisal Skills Programme tool. We synthesised the findings based on the decision‐making components of HIS and thematic areas.

Results

The search identified 6784 studies; 50 were included. Most of the 50 studies had quality concerns. All studies included at least one of the decision‐making components: the most prominent were the technical aspects of ‘recording’ and ‘reporting’. Data use for decision‐making was much less represented.

Conclusion

HIS research focuses on the more technical aspects of HIS. Further research on HIS, given the strong push towards HIS digitalisation, should consider putting at the centre the human experience of decision‐making and data use, in order to make HIS relevant for quality of care.

Across the health system, Health Information Systems (HIS) are meant to support data management and use, which are key activities of health care delivery.

Informed by existing frameworks and by fieldwork in three African countries, we identified twelve decision‐making components of HIS.

In this framework synthesis, we found that the most common decision‐making components in the HIS literature were those related to the technical aspects of data ‘recording’ and ‘reporting’; data use for decision‐making was much less represented.

For HIS instruments to facilitate improved health care delivery, clinical and public health decision‐making components of HIS should be put at the centre for HIS research, ideation and implementation.

The Effectiveness of the Capacity Building and Mentorship Program in Improving Evidence-Based Decision-making in the Amhara Region, Northwest Ethiopia: Difference-in-Differences Study

Background

Weak health information systems (HISs) hobble countries’ abilities to effectively manage and distribute their resources to match the burden of disease. The Capacity Building and Mentorship Program (CBMP) was implemented in select districts of the Amhara region of Ethiopia to improve HIS performance; however, evidence about the effectiveness of the intervention was meager.

Objective

This study aimed to determine the effectiveness of routine health information use for evidence-based decision-making among health facility and department heads in the Amhara region, Northwest Ethiopia.

Methods

The study was conducted in 10 districts of the Amhara region: five were in the intervention group and five were in the comparison group. We employed a quasi-experimental study design in the form of a pretest-posttest comparison group. Data were collected from June to July 2020 from the heads of departments and facilities in 36 intervention and 43 comparison facilities. The sample size was calculated using the double population formula, and we recruited 172 participants from each group. We applied a difference-in-differences analysis approach to determine the effectiveness of the intervention. Heterogeneity of program effect among subgroups was assessed using a triple differences method (ie, difference-in-difference-in-differences [DIDID] method). Thus, the β coefficients, 95% CIs, and P values were calculated for each parameter, and we determined that the program was effective if the interaction term was significant at P<.05.

Results

Data were collected using the endpoint survey from 155 out of 172 (90.1%) participants in the intervention group and 166 out of 172 (96.5%) participants in the comparison group. The average level of information use for the comparison group was 37.3% (95% CI 31.1%-43.6%) at baseline and 43.7% (95% CI 37.9%-49.5%) at study endpoint. The average level of information use for the intervention group was 52.2% (95% CI 46.2%-58.3%) at baseline and 75.8% (95% CI 71.6%-80.0%) at study endpoint. The study indicated that the net program change over time was 17% (95% CI 5%-28%; P=.003). The subgroup analysis also indicated that location showed significant program effect heterogeneity, with a DIDID estimate equal to 0.16 (95% CI 0.026-0.29; P=.02). However, sex, age, educational level, salary, and experience did not show significant heterogeneity in program effect, with DIDID estimates of 0.046 (95% CI –0.089 to 0.182), –0.002 (95% CI –0.015 to 0.009), –0.055 (95% CI –0.190 to 0.079), –1.63 (95% CI –5.22 to 1.95), and –0.006 (95% CI –0.017 to 0.005), respectively.

Conclusions

The CBMP was effective at enhancing the capacity of study participants in using the routine HIS for decision-making. We noted that urban facilities had benefited more than their counterparts. The intervention has been shown to produce positive outcomes and should be scaled up to be used in other districts. Moreover, the mentorship modalities for rural facilities should be redesigned to maximize the benefits.

Trial Registration

Pan African Clinical Trials Registry PACTR202001559723931; https://tinyurl.com/3j7e5ka5

Building Resilient Health Systems Intelligence: Adapting Indicators of Compromise for the Cyber-Bionexus

The health sector is an underutilized source of actionable health intelligence for responding to threats across the "cyber-bionexus," defined as the convergence of threats from the biological and cybersecurity domains to produce harms with widespread societal consequences. The escalation of concerns about such threats-related to misinformation and disinformation; chemical, biological, radiological, and nuclear events; cyberattacks; natural disease outbreaks; and disasters of various kinds-places health system concerns squarely at the forefront of national critical systems and broader security imperatives. Events such as the COVID-19 pandemic have highlighted the dearth of systems available for generating real-time intelligence in relation to critical functions of health sector operations amidst an unfolding crisis. Drawing on principles from the field of cyberthreat intelligence, and building on existing scholarship in health security intelligence, we propose a model for applying health system indicators of compromise for cyberbio events. We further discuss the relevance of this approach within the broader landscape of the cyber-bionexus to signal new pathways for research, practice, and policy engagement.

Addressing missing values in routine health information system data: an evaluation of imputation methods using data from the Democratic Republic of the Congo during the COVID-19 pandemic

BACKGROUND

Poor data quality is limiting the use of data sourced from routine health information systems (RHIS), especially in low- and middle-income countries. An important component of this data quality issue comes from missing values, where health facilities, for a variety of reasons, fail to report to the central system.

METHODS

Using data from the health management information system in the Democratic Republic of the Congo and the advent of COVID-19 pandemic as an illustrative case study, we implemented seven commonly used imputation methods and evaluated their performance in terms of minimizing bias in imputed values and parameter estimates generated through subsequent analytical techniques, namely segmented regression, which is widely used in interrupted time series studies, and pre-post-comparisons through paired Wilcoxon rank-sum tests. We also examined the performance of these imputation methods under different missing mechanisms and tested their stability to changes in the data.

RESULTS

For regression analyses, there were no substantial differences found in the coefficient estimates generated from all methods except mean imputation and exclusion and interpolation when the data contained less than 20% missing values. However, as the missing proportion grew, k-NN started to produce biased estimates. Machine learning algorithms, i.e. missForest and k-NN, were also found to lack robustness to small changes in the data or consecutive missingness. On the other hand, multiple imputation methods generated the overall most unbiased estimates and were the most robust to all changes in data. They also produced smaller standard errors than single imputations. For pre-post-comparisons, all methods produced p values less than 0.01, regardless of the amount of missingness introduced, suggesting low sensitivity of Wilcoxon rank-sum tests to the imputation method used.

CONCLUSIONS

We recommend the use of multiple imputation in addressing missing values in RHIS datasets and appropriate handling of data structure to minimize imputation standard errors. In cases where necessary computing resources are unavailable for multiple imputation, one may consider seasonal decomposition as the next best method. Mean imputation and exclusion and interpolation, however, always produced biased and misleading results in the subsequent analyses, and thus, their use in the handling of missing values should be discouraged.

Assessment of RHIS Quality Assurance Practices in Tarkwa Submunicipal Health Directorate, Ghana

Background. Routine health information system (RHIS) quality assurance has become an important issue, not only because of its significance in promoting high standard of patient care, but also because of its impact on government budgets for the maintenance of health services. Routine health information system comprises healthcare data collection, compilation, storage, analysis, report generation, and dissemination on routine basis at the various healthcare settings. The data from RHIS give a representation of health status, health services, and health resources. The sources of RHIS data are normally individual health records, records of services delivered, and records of health resources. Using reliable information from routine health information systems is fundamental in the healthcare delivery system. Quality assurance practices are measures that are put in places to ensure the health data that are collected meet required quality standards. Routine health information system quality assurance practices ensure that data that are generated from the system are fit for use. This study considered quality assurance practices in the RHIS processes. Methods. A cross-sectional study was conducted in eight health facilities in Tarkwa Submunicipal health service in the western region of Ghana. The study involved routine quality assurance practices among the 90-health staff and management selected from facilities in Tarkwa Submunicipal who collect or use data routinely from 24th December, 2019, to 20th January, 2020. Results. Generally, Tarkwa Submunicipal health service appears to practice quality assurance during data collection, compilation, storage, analysis, and dissemination. The results show some achievement in quality control performance in report dissemination (77.6%), data analysis (68.0%), data compilation (67.4%), report compilation (66.3%), data storage (66.3%), and collection (61.1%). Conclusions. Even though Tarkwa Submunicipal health directorate engages some control measures to ensure data quality, there is the need to strengthen the process to achieve the targeted percentage of performance (90.0%). There was significant shortfall in quality assurance practices performance especially during data collection, with respect to the expected performance.

A call for standardised age-disaggregated health data

The 2030 Sustainable Development Goals agenda calls for health data to be disaggregated by age. However, age groupings used to record and report health data vary greatly, hindering the harmonisation, comparability, and usefulness of these data, within and across countries. This variability has become especially evident during the COVID-19 pandemic, when there was an urgent need for rapid cross-country analyses of epidemiological patterns by age to direct public health action, but such analyses were limited by the lack of standard age categories. In this Personal View, we propose a recommended set of age groupings to address this issue. These groupings are informed by age-specific patterns of morbidity, mortality, and health risks, and by opportunities for prevention and disease intervention. We recommend age groupings of 5 years for all health data, except for those younger than 5 years, during which time there are rapid biological and physiological changes that justify a finer disaggregation. Although the focus of this Personal View is on the standardisation of the analysis and display of age groups, we also outline the challenges faced in collecting data on exact age, especially for health facilities and surveillance data. The proposed age disaggregation should facilitate targeted, age-specific policies and actions for health care and disease management.

Routine Health Information Systems in the European Context: A Systematic Review of Systematic Reviews

(1) Background: The aim of this study is to provide a better understanding of the requirements to improve routine health information systems (RHISs) for the management of health systems, including the identification of best practices, opportunities, and challenges in the 53 countries and territories of the WHO European region. (2) Methods: We conducted an overview of systematics reviews and searched the literature in the databases MEDLINE/PubMed, Cochrane, EMBASE, and Web of Science electronic databases. After a meticulous screening, we identified 20 that met the inclusion criteria, and RHIS evaluation results were presented according to the Performance of Routine Information System Management (PRISM) framework. (3) Results: The reviews were published between 2007 and 2020, focusing on the use of different systems or technologies and aimed to analyze interventions on professionals, centers, or patients' outcomes. All reviews examined showed variability in results in accordance with the variability of interventions and target populations. We have found different areas for improvement for RHISs according to the three determinants of the PRISM framework that influence the configuration of RHISs: technical, organizational, or behavioral elements. (4) Conclusions: RHIS interventions in the European region are promising. However, new global and international strategies and the development of tools and mechanisms should be promoted to highly integrate platforms among European countries.

Optimizing Pandemic Preparedness and Response Through Health Information Systems: Lessons Learned From Ebola to COVID-19

Strengthening health systems and maintaining essential service delivery during health emergencies response is critical for early detection and diagnosis, prompt treatment, and effective control of pandemics, including the novel coronavirus disease 2019 (COVID-19). Health information systems (HIS) developed during recent Ebola outbreaks in West Africa and the Democratic Republic of the Congo (DRC) provided opportunities to collect, analyze, and distribute data to inform both day-to-day and long-term policy decisions on outbreak preparedness. As COVID-19 continues to sweep across the globe, HIS and related technological advancements remain vital for effective and sustained data sharing, contact tracing, mapping and monitoring, community risk sensitization and engagement, preventive education, and timely preparedness and response activities. In reviewing literature of how HIS could have further supported mitigation of these Ebola outbreaks and the ongoing COVID-19 pandemic, 3 key areas were identified: governance and coordination, health systems infrastructure and resources, and community engagement. In this concept study, we outline scalable HIS lessons from recent Ebola outbreaks and early COVID-19 responses along these 3 domains, synthesizing recommendations to offer clear, evidence-based approaches on how to leverage HIS to strengthen the current pandemic response and foster community health systems resilience moving forward.