Determinants of Facility-Level Use of Electronic Immunization Registries in Tanzania and Zambia: An Observational Analysis

Evaluating the "Zindagi Mehfooz" Electronic Immunization Registry and Suite of Digital Health Interventions to Improve the Coverage and Timeliness of Immunization Services in Sindh, Pakistan: Mixed Methods Study

BACKGROUND

The Zindagi Mehfooz (safe life; ZM) electronic immunization registry (EIR) is a comprehensive suite of digital health interventions that aims to improve equitable access, timeliness, and coverage of child immunizations through a smartphone-based app for vaccinators, web-based dashboards for supervisors and managers, text message alerts and reminders for caregivers, and a call center. It has been implemented at scale in Sindh Province, Pakistan.

OBJECTIVE

This study aimed to present findings from an evaluation of the ZM-EIR suite of digital health interventions in order to improve data availability and use as a contribution, among other immunization program interventions, to enhanced immunization outcomes for children aged 12-23 months in Sindh Province.

METHODS

The mixed methods study included (1) analysis of ZM-EIR system data to identify high-, moderate-, and low-adoption and compliance sites; (2) in-depth interviews with caregivers, vaccinators, supervisors, and managers in the Expanded Program for Immunization (EPI); and (3) pre-post outcome evaluation using vaccine coverage from the Multiple Indicator Cluster Surveys (MICS) 2014 and 2018-2019. Key outcomes of interest were improved data availability, use and contribution to immunization outcomes, including receipt of individual antigens (Bacillus Calmette-Guérin [BCG], pentavalent [Penta] 1-3, measles), full immunization (all antigens), and zero-dose children defined as children aged 6-23 months who have not received the first dosage of the diphtheria-pertussis-tetanus 1/Penta vaccine.

RESULTS

By registering newborns, providing alerts and reminders, and tracking their immunization completion, the ZM-EIR improved data availability and use in the EPI. The ZM-EIR was well received by EPI administrators, supervisors, vaccinators, and caregivers. The key benefit highlighted by ZM-EIR users was a list of children who missed scheduled vaccines (defaulters). Through greater availability and use of data, the ZM-EIR implementation, as part of a broader package of immunization program-strengthening activities in Sindh Province, may have contributed to an increase in immunization coverage and timeliness for BCG vaccinations and a decrease in zero-dose children in 2018-2019 from 2014. Additional findings from the study included the dual burden of reporting on paper and gender-related considerations of female caregivers not wanting to provide their phone numbers to male vaccinators, creating barriers to greater uptake of the ZM-EIR.

CONCLUSIONS

The ZM-EIR is a promising technology platform that has increased the availability and use of immunization data, which may have contributed, along with other intensive immunization program interventions, to improvements in immunization outcomes through systematic registration of children, alerts and reminders, and increased use of data for planning and monitoring by the EPI.

TRIAL REGISTRATION

ISRCTN Registry ISRCTN23078223; https://doi.org/10.1186/ISRCTN23078223.

Electronic Immunization Registry in Rwanda: Qualitative Study of Health Worker Experiences

BACKGROUND

Monitoring childhood immunization programs is essential for health systems. Despite the introduction of an electronic immunization registry called e-Tracker in Rwanda, challenges such as lacking population denominators persist, leading to implausible reports of coverage rates of more than 100%.

OBJECTIVE

This study aimed to assess the extent to which the immunization e-Tracker responds to stakeholders' needs and identify key areas for improvement.

METHODS

In-depth interviews were conducted with all levels of e-Tracker users including immunization nurses, data managers, and supervisors from health facilities in 5 districts of Rwanda. We used an interview guide based on the constructs of the Human, Organization, and Technology-Fit (HOT-Fit) framework, and we analyzed and summarized our findings using the framework.

RESULTS

Immunization nurses reported using the e-Tracker as a secondary data entry tool in addition to paper-based forms, which resulted in considerable dissatisfaction among nurses. While users acknowledged the potential of a digital tool compared to paper-based systems, they also reported the need for improvement of functionalities to support their work, such as digital client appointment lists, lists of defaulters, search and register functions, automated monthly reports, and linkages to birth notifications and the national identity system.

CONCLUSIONS

Reducing dual documentation for users can improve e-Tracker use and user satisfaction. Our findings can help identify additional digital health interventions to support and strengthen the health information system for the immunization program.

Outcomes and Costs of the Transition From a Paper-Based Immunization System to a Digital Immunization System in Vietnam: Mixed Methods Study

BACKGROUND

The electronic National Immunization Information System (NIIS) was introduced nationwide in Vietnam in 2017. Health workers were expected to use the NIIS alongside the legacy paper-based system. Starting in 2018, Hanoi and Son La provinces transitioned to paperless reporting. Interventions to support this transition included data guidelines and training, internet-based data review meetings, and additional supportive supervision visits.

OBJECTIVE

This study aims to assess (1) changes in NIIS data quality and use, (2) changes in immunization program outcomes, and (3) the economic costs of using the NIIS versus the traditional paper system.

METHODS

This mixed methods study took place in Hanoi and Son La provinces. It aimed to analyses pre- and postintervention data from various sources including the NIIS; household and health facility surveys; and interviews to measure NIIS data quality, data use, and immunization program outcomes. Financial data were collected at the national, provincial, district, and health facility levels through record review and interviews. An activity-based costing approach was conducted from a health system perspective.

RESULTS

NIIS data timeliness significantly improved from pre- to postintervention in both provinces. For example, the mean number of days from birth date to NIIS registration before and after intervention dropped from 18.6 (SD 65.5) to 5.7 (SD 31.4) days in Hanoi (P<.001) and from 36.1 (SD 94.2) to 11.7 (40.1) days in Son La (P<.001). Data from Son La showed that the completeness and accuracy improved, while Hanoi exhibited mixed results, possibly influenced by the COVID-19 pandemic. Data use improved; at postintervention, 100% (667/667) of facilities in both provinces used NIIS data for activities beyond monthly reporting compared with 34.8% (202/580) in Hanoi and 29.4% (55/187) in Son La at preintervention. Across nearly all antigens, the percentage of children who received the vaccine on time was higher in the postintervention cohort compared with the preintervention cohort. Up-front costs associated with developing and deploying the NIIS were estimated at US $0.48 per child in the study provinces. The commune health center level showed cost savings from changing from the paper system to the NIIS, mainly driven by human resource time savings. At the administrative level, incremental costs resulted from changing from the paper system to the NIIS, as some costs increased, such as labor costs for supportive supervision and additional capital costs for equipment associated with the NIIS.

CONCLUSIONS

The Hanoi and Son La provinces successfully transitioned to paperless reporting while maintaining or improving NIIS data quality and data use. However, improvements in data quality were not associated with improvements in the immunization program outcomes in both provinces. The COVID-19 pandemic likely had a negative influence on immunization program outcomes, particularly in Hanoi. These improvements entail up-front financial costs.

Design, Development, and Deployment of an Electronic Immunization Registry: Experiences From Vietnam, Tanzania, and Zambia

INTRODUCTION

There is growing interest among low- and middle-income countries to introduce electronic immunization registries (EIRs) that capture individual-level vaccine data. We compare the design, development, and deployment of EIRs in Vietnam, Tanzania, and Zambia. Through desk review and the authors' firsthand implementation experiences, we describe experiences related to timeline, partnerships, financial costs, and technology and infrastructure.

IMPLEMENTATION EXPERIENCE

The country cases highlight the multi-year timeline required to implement an EIR at scale and the benefit of multiple iterative cycles to pilot and redesign the system before achieving scale. Of the 3 countries, only Vietnam has achieved nationwide scale of the EIR, which took 7 years. In all 3 countries, national government leadership as part of an interdisciplinary team (with experience in leadership, technology, and immunization) was important to ensure country ownership and sustainability. Where international software developers were contracted, partnering with a local software company helped improve responsiveness and sustainability. Across all 3 countries, governments contributed significant in-kind time in addition to investments from donors. Cost savings were observed in Tanzania and Zambia, largely driven by health worker time savings from using the EIR. All 3 case countries underscore the need to understand the local technology and infrastructure context and design the EIR to fit the context. In Vietnam, an initial landscape assessment was conducted to assess technology and infrastructure, whereas in Tanzania and Zambia, user advisory groups provided insights. Existing infrastructure informed EIR design decisions, such as choosing a system with offline functionality in Tanzania and Zambia. All 3 countries have a local partner to provide ongoing technical support.

CONCLUSION

Comparing implementation factors across these cases highlights practical experience and recommendations that complement existing EIR guidance documents. The findings and recommendations from this study can inform other countries considering or in the process of implementing an EIR.

What Factors Explain Low Adoption of Digital Technologies for Health Financing in an Insurance Setting? Novel Evidence From a Quantitative Panel Study on IMIS in Tanzania

BACKGROUND

Digital information management systems for health financing are implemented on the assumption thatdigitalization, among other things, enables strategic purchasing. However, little is known about the extent to which thesesystems are adopted as planned to achieve desired results. This study assesses the levels of, and the factors associated withthe adoption of the Insurance Management Information System (IMIS) by healthcare providers in Tanzania.

METHODS

Combining multiple data sources, we estimated IMIS adoption levels for 365 first-line health facilities in2017 by comparing IMIS claim data (verified claims) with the number of expected claims. We defined adoption as abinary outcome capturing underreporting (verified

RESULTS

We found a median (interquartile range [IQR]) difference of 77.8% (32.7-100) between expected and verifiedclaims, showing a consistent pattern of underreporting across districts, regions, and months. Levels of underreportingvaried across regions (ANOVA: F=7.24, P<.001) and districts (ANOVA: F=4.65, P<.001). Logistic regression resultsshowed that higher service volume, share of people insured, and greater distance to district headquarter were associatedwith a higher probability of underreporting.

CONCLUSION

Our study shows that the adoption of IMIS in Tanzania may be sub-optimal and far from policy-makers'expectations, limiting its capacity to provide the necessary information to enhance strategic purchasing in the healthsector. Countries and agencies adopting digital interventions such as openIMIS to foster health financing reform areadvised to closely track their implementation efforts to make sure the data they rely on is accurate. Further, our studysuggests organizational and infrastructural barriers beyond the software itself hamper effective adoption.

Collapse

Key Words

• Adoption
• Digital Health Intervention
• Health Financing
• Health Insurance
• Strategic Purchasing
• Tanzania

Collapse

MESH Headings

• Humans
• Tanzania
• Healthcare Financing
• Digital Technology
• Insurance, Health
• Universal Health Insurance

Collapse

Grants Collapse

Affiliation(s)

Leon Schuetze Heidelberg Institute of Global Health, Medical Faculty and University Hospital, University of Heidelberg, Heidelberg, Germany
Siddharth Srivastava Swiss Tropical and Public Health Institute (Swiss TPH), Basel, Switzerland University of Basel, Basel, Switzerland
Naasegnibe Kuunibe Heidelberg Institute of Global Health, Medical Faculty and University Hospital, University of Heidelberg, Heidelberg, Germany Faculty of Integrated Development Studies, University for Development Studies, Wa, Ghana
Elizeus Josephat Rwezaula Health Promotion and System Strengthening Project (HPSS), Dodoma, Tanzania
Abdallah Missenye Kongwa District Council, Dodoma, Tanzania
Manfred Stoermer Swiss Tropical and Public Health Institute (Swiss TPH), Basel, Switzerland University of Basel, Basel, Switzerland
Manuela De Allegri Heidelberg Institute of Global Health, Medical Faculty and University Hospital, University of Heidelberg, Heidelberg, Germany

Collapse

Factors Affecting the Successful Implementation of a Digital Intervention for Health Financing in a Low-Resource Setting at Scale: Semistructured Interview Study With Health Care Workers and Management Staff

BACKGROUND

Digital interventions for health financing, if implemented at scale, have the potential to improve health system performance by reducing transaction costs and improving data-driven decision-making. However, many interventions never reach sustainability, and evidence on success factors for scale is scarce. The Insurance Management Information System (IMIS) is a digital intervention for health financing, designed to manage an insurance scheme and already implemented on a national scale in Tanzania. A previous study found that the IMIS claim function was poorly adopted by health care workers (HCWs), questioning its potential to enable strategic purchasing and succeed at scale.

OBJECTIVE

This study aimed to understand why the adoption of the IMIS claim function by HCWs remained low in Tanzania and to assess implications for use at scale.

METHODS

We conducted 21 semistructured interviews with HCWs and management staff in 4 districts where IMIS was first implemented. We sampled respondents by using a maximum variation strategy. We used the framework method for data analysis, applying a combination of inductive and deductive coding to organize codes in a socioecological model. Finally, we related emerging themes to a framework for digital health interventions for scale.

RESULTS

Respondents appreciated IMIS's intrinsic software characteristics and technical factors and acknowledged IMIS as a valuable tool to simplify claim management. Human factors, extrinsic ecosystem, and health care ecosystem were considered as barriers to widespread adoption.

CONCLUSIONS

Digital interventions for health financing, such as IMIS, may have the potential for scale if careful consideration is given to the environment in which they are placed. Without a sustainable health financing environment, sufficient infrastructure, and human capacity, they cannot unfold their full potential to improve health financing functions and ultimately contribute to universal health coverage.

Changes in on-time vaccination following the introduction of an electronic immunization registry, Tanzania 2016-2018: interrupted time-series analysis

BACKGROUND

Digital health interventions (DHI) have the potential to improve the management and utilization of health information to optimize health care worker performance and provision of care. Despite the proliferation of DHI projects in low-and middle-income countries, few have been evaluated in an effort to understand their impact on health systems and health-related outcomes. Although more evidence is needed on their impact and effectiveness, the use of DHIs among immunization programs has become more widespread and shows promise for improving vaccination uptake and adherence to immunization schedules.

METHODS

Our aim was to assess the impact of an electronic immunization registry (EIR) using an interrupted time-series analysis to analyze the effect on proportion of on-time vaccinations following introduction of an EIR in Tanzania. We hypothesized that the introduction of the EIR would lead to statistically significant changes in vaccination timeliness at 3, 6, and > 6 months post-introduction.

RESULTS

For our primary analysis, we observed a decrease in the proportion of on-time vaccinations following EIR introduction. In contrast, our sensitivity analysis estimated improvements in timeliness among those children with complete vaccination records. However, we must emphasize caution interpreting these findings as they are likely affected by implementation challenges.

CONCLUSIONS

This study highlights the complexities of using digitized individual-level routine health information system data for evaluation and research purposes. EIRs have the potential to improve vaccination timeliness, but analyses using EIR data can be complicated by data quality issues and inconsistent data entry leading to difficulties interpreting findings.

Behavioral Determinants of Routine Health Information System Data Use in Senegal: A Qualitative Inquiry Based on the Integrated Behavioral Model

Although behavioral factors are thought to be important barriers to routine data use, they remain understudied particularly in low-income country settings. We show that the integrated behavior model can be a valuable theoretical framework for targeted communication strategies and capacity-building interventions aimed at promoting a culture of data use.

Routine health information system (RHIS) data are essential in driving decision making and planning in health systems as well as health programs. However, despite their importance, these data are underutilized, and the underlying individual-level facilitators and barriers to use remain understudied. In this research, we applied the Integrated Behavior Model (IBM) to examine how attitudes toward RHIS data, perceived norms concerning RHIS data use, and the ability to use RHIS data influence the demand and use of RHIS data among stakeholders in Senegal. Using data from interviews with respondents working at national levels of malaria, HIV, and TB control programs in Senegal, we used a framework analysis approach to apply the IBM behavioral constructs and identify their linkages to RHIS data use. We found that attitudes about the quality, availability, and relevance of RHIS data for decision making were important in driving data use among respondents. Institutional expectations, organizational protocols, policies, and practices around RHIS data ultimately shape social norms around the use of the data. Although we found that perceived ability and self-efficacy to use RHIS data were not barriers to RHIS data use among stakeholders at the strategic levels of their respective organizations, these were reported to be barriers at lower levels of the health system. Low perceived control of the RHIS data production process ultimately reduced RHIS data use for decision making among the strategic-level respondents. We recommend context-specific reexamination of existing RHIS interventions with a renewed emphasis on behavioral aspects of data use. The IBM can help guide practitioners, policy makers, and academics to address multiple socioecological factors that influence data use behavior when recommending RHIS and data use solutions.

Added Value of Electronic Immunization Registries in Low- and Middle-Income Countries: Observational Case Study in Tanzania

Background

There is growing interest and investment in electronic immunization registries (EIRs) in low- and middle-income countries. EIRs provide ready access to patient- and aggregate-level service delivery data that can be used to improve patient care, identify spatiotemporal trends in vaccination coverage and dropout, inform resource allocation and program operations, and target quality improvement measures. The Government of Tanzania introduced the Tanzania Immunization Registry (TImR) in 2017, and the system has since been rolled out in 3736 facilities in 15 regions.

Objective

The aims of this study are to conceptualize the additional ways in which EIRs can add value to immunization programs (beyond measuring vaccine coverage) and assess the potential value-add using EIR data from Tanzania as a case study.

Methods

This study comprised 2 sequential phases. First, a comprehensive list of ways EIRs can potentially add value to immunization programs was developed through stakeholder interviews. Second, the added value was evaluated using descriptive and regression analyses of TImR data for a prioritized subset of program needs.

Results

The analysis areas prioritized through stakeholder interviews were population movement, missed opportunities for vaccination (MOVs), continuum of care, and continuous quality improvement. The included TImR data comprised 958,870 visits for 559,542 patients from 2359 health facilities. Our analyses revealed that few patients sought care outside their assigned facility (44,733/810,568, 5.52% of applicable visits); however, this varied by region; facility urbanicity, type, ownership, patient volume, and duration of TImR system use; density of facilities in the immediate area; and patient age. Analyses further showed that MOVs were highest among children aged <12 months (215,576/831,018, 25.94% of visits included an MOV and were applicable visits); however, there were few significant differences based on other individual or facility characteristics. Nearly half (133,337/294,464, 45.28%) of the children aged 12 to 35 months were fully vaccinated or had received all doses except measles-containing vaccine–1 of the 14-dose under-12-month schedule (ie, through measles-containing vaccine–1), and facility and patient characteristics associated with dropout varied by vaccine. The continuous quality improvement analysis showed that most quality issues (eg, MOVs) were concentrated in <10% of facilities, indicating the potential for EIRs to target quality improvement efforts.

Conclusions

EIRs have the potential to add value to immunization stakeholders at all levels of the health system. Individual-level electronic data can enable new analyses to understand service delivery or care-seeking patterns, potential risk factors for underimmunization, and where challenges occur. However, to achieve this potential, country programs need to leverage and strengthen the capacity to collect, analyze, interpret, and act on the data. As EIRs are introduced and scaled in low- and middle-income countries, implementers and researchers should continue to share real-world examples and build an evidence base for how EIRs can add value to immunization programs, particularly for innovative uses.

The Transition to an Entirely Digital Immunization Registry in Ha Noi Province and Son La Province, Vietnam: Readiness Assessment Study

Background

Vietnam is one of the first low- to middle-income countries to develop and implement a national-scale electronic immunization registry. This system was finalized into the National Immunization Information System (NIIS) and scaled up to a national-level system in 2017. As a result, immunization coverage and the timeliness of vaccinations have drastically improved. The time spent on planning and reporting vaccinations has drastically reduced; as a result, vaccination planning and reporting has become more accurate and effective. However, to date, end users have been tasked with managing both the NIIS and paper-based systems in parallel until a formal assessment of the readiness to fully transition to the NIIS is conducted.

Objective

This study aims to evaluate the readiness to move to an entirely digital NIIS in 2 provinces of Vietnam—Ha Noi and Son La.

Methods

All health facilities were surveyed to assess their infrastructure, capacity, and need for human resources. NIIS end users were observed and interviewed to evaluate their NIIS knowledge and skill sets. Data from immunization cards and facility paper-based logbooks were compared with data from the NIIS, and vaccine stocks at selected facilities were tallied and compared with data from the NIIS.

Results

Of the 990 health facilities evaluated, most used the NIIS to enter and track immunizations (987/990, 99.7%) and vaccine stocks (889/990, 90.8%). Most had stable electricity (971/990, 98.1%), at least 1 computer (986/990, 99.6%), and ≥2 trained NIIS end users (825/990, 83.3%). End users reported that the NIIS supported them in managing and reporting immunization data and saving them time (725/767, 94.5%). Although many end users were able to perform basic skills, almost half struggled with performing more complex tasks. Immunization data were compiled from the NIIS and immunization cards (338/378, 89.4%) and paper-based logbooks (254/269, 94.4%). However, only 54.5% (206/378) of immunization IDs matched, 57% (13/23) of Bacillus Calmette-Guérin vaccination records were accurate, and 70% (21/30) of the facilities had consistent physical vaccine stock balances. The feedback received from NIIS end users suggests that more supportive supervision, frequent refresher training for strengthening their skill sets, and detailed standardized guides for improving data quality are needed.

Conclusions

The readiness to transition to a digital system is promising; however, additional resources are required to address the timeliness, completeness, and accuracy of the data.