Evaluation of a mHealth Data Quality Intervention to Improve Documentation of Pregnancy Outcomes by Health Surveillance Assistants in Malawi: A Cluster Randomized Trial

'[We] learned how to speak with love': a qualitative exploration of accredited social health activist (ASHA) community health worker experiences of the Mobile Academy refresher training in Rajasthan, India

INTRODUCTION

Mobile Academy is a mobile-based training course for India's accredited social health activist (ASHA) community health workers (CHW). The course, which ASHAs access by dialling a number from their phones, totals 4 hours of audio content. It consists of 11 chapters, each with their own quiz, and provides a cumulative pass or fail score at the end. This qualitative study of Mobile Academy explores how the programme was accessed and experienced by CHWs, and how they perceive it to have influenced their work.

METHODS

We conducted in-depth interviews (n=25) and focus group discussions (n=5) with ASHAs and other health system actors. Open-ended questions explored ASHA perspectives on Mobile Academy, the course's perceived influence on ASHAs and preferences for future training programmes. After applying a priori codes to the transcripts, we identified emergent themes and grouped them according to our CHW mLearning framework.

RESULTS

ASHAs reported enjoying Mobile Academy, specifically praising its friendly tone and the ability to repeat content. They, and higher level health systems actors, conceived it to primarily be a test not a training. ASHAs reported that they found the quizzes easy but generally did not consider the course overly simplistic. ASHAs considered Mobile Academy's content to be a useful knowledge refresher but said its primary benefit was in modelling a positive communications approach, which inspired them to adopt a kinder, more 'loving' communication style when speaking to beneficiaries. ASHAs and health system actors wanted follow-on mLearning courses that would continue to compliment but not replace face-to-face training.

CONCLUSION

This mLearning programme for CHWs in India was well received by ASHAs across a wide range of education levels and experience. Dial-in audio training has the potential to reinforce topical knowledge and showcase positive ways to communicate.

Digital Solutions for Community and Primary Health Workers: Lessons From Implementations in Africa

The agenda for Universal Health Coverage has driven the exploration of various innovative approaches to expanding health services to the general population. As more African countries have adopted digital health tools as part of the strategic approach to expanding health services, there is a need for defining a standard framework for implementation across board. Therefore, there is a need to review and employ an evidence-based approach to inform managing challenges, adopting best approaches, and implement informed recommendations. We reviewed a variety of digital health tools applied to different health conditions in primary care settings and highlighted the challenges faced, approaches that worked and relevant recommendations. These include limited coverage and network connectivity, lack of technological competence, lack of power supply, limited mobile phone usage and application design challenges. Despite these challenges, this review suggests that mHealth solutions could attain effective usage when healthcare workers receive adequate onsite training, deploying applications designed in an intuitive and easy to understand approach in a manner that fits into the users existing workflows, and involvement of the stakeholders at all levels in the design, planning, and implementation stages of the interventions.

Does an innovative paper-based health information system (PHISICC) improve data quality and use in primary healthcare? Protocol of a multicountry, cluster randomised controlled trial in sub-Saharan African rural settings

INTRODUCTION

Front-line health workers in remote health facilities are the first contact of the formal health sector and are confronted with life-saving decisions. Health information systems (HIS) support the collection and use of health related data. However, HIS focus on reporting and are unfit to support decisions. Since data tools are paper-based in most primary healthcare settings, we have produced an innovative Paper-based Health Information System in Comprehensive Care (PHISICC) using a human-centred design approach. We are carrying out a cluster randomised controlled trial in three African countries to assess the effects of PHISICC compared with the current systems.

METHODS AND ANALYSIS

Study areas are in rural zones of Côte d'Ivoire, Mozambique and Nigeria. Seventy health facilities in each country have been randomly allocated to using PHISICC tools or to continuing to use the regular HIS tools. We have randomly selected households in the catchment areas of each health facility to collect outcomes' data (household surveys have been carried out in two of the three countries and the end-line data collection is planned for mid-2021). Primary outcomes include data quality and use, coverage of health services and health workers satisfaction; secondary outcomes are additional data quality and use parameters, childhood mortality and additional health workers and clients experience with the system. Just prior to the implementation of the trial, we had to relocate the study site in Mozambique due to unforeseen logistical issues. The effects of the intervention will be estimated using regression models and accounting for clustering using random effects.

ETHICS AND DISSEMINATION

Ethics committees in Côte d'Ivoire, Mozambique and Nigeria approved the trials. We plan to disseminate our findings, data and research materials among researchers and policy-makers. We aim at having our findings included in systematic reviews on health systems interventions and future guidance development on HIS.

TRIAL REGISTRATION NUMBER

PACTR201904664660639; Pre-results.

'Do you trust those data?'-a mixed-methods study assessing the quality of data reported by community health workers in Kenya and Malawi

High-quality data are essential to monitor and evaluate community health worker (CHW) programmes in low- and middle-income countries striving towards universal health coverage. This mixed-methods study was conducted in two purposively selected districts in Kenya (where volunteers collect data) and two in Malawi (where health surveillance assistants are a paid cadre). We calculated data verification ratios to quantify reporting consistency for selected health indicators over 3 months across 339 registers and 72 summary reports. These indicators are related to antenatal care, skilled delivery, immunization, growth monitoring and nutrition in Kenya; new cases, danger signs, drug stock-outs and under-five mortality in Malawi. We used qualitative methods to explore perceptions of data quality with 52 CHWs in Kenya, 83 CHWs in Malawi and 36 key informants. We analysed these data using a framework approach assisted by NVivo11. We found that only 15% of data were reported consistently between CHWs and their supervisors in both contexts. We found remarkable similarities in our qualitative data in Kenya and Malawi. Barriers to data quality mirrored those previously reported elsewhere including unavailability of data collection and reporting tools; inadequate training and supervision; lack of quality control mechanisms; and inadequate register completion. In addition, we found that CHWs experienced tensions at the interface between the formal health system and the communities they served, mediated by the social and cultural expectations of their role. These issues affected data quality in both contexts with reports of difficulties in negotiating gender norms leading to skipping sensitive questions when completing registers; fabrication of data; lack of trust in the data; and limited use of data for decision-making. While routine systems need strengthening, these more nuanced issues also need addressing. This is backed up by our finding of the high value placed on supportive supervision as an enabler of data quality.

Routine Health Information System (RHIS) improvements for strengthened health system management

BACKGROUND

A well-functioning routine health information system (RHIS) can provide the information needed for health system management, for governance, accountability, planning, policy making, surveillance and quality improvement, but poor information support has been identified as a major obstacle for improving health system management.

OBJECTIVES

To assess the effects of interventions to improve routine health information systems in terms of RHIS performance, and also, in terms of improved health system management performance, and improved patient and population health outcomes.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library, MEDLINE Ovid and Embase Ovid in May 2019. We searched Global Health, Ovid and PsycInfo in April 2016. In January 2020 we searched for grey literature in the Grey Literature Report and in OpenGrey, and for ongoing trials using the International Clinical Trials Registry Platform (ICTRP) and ClinicalTrials.gov. In October 2019 we also did a cited reference search using Web of Science, and a 'similar articles' search in PubMed.

SELECTION CRITERIA

Randomised and non-randomised trials, controlled before-after studies and time-series studies comparing routine health information system interventions, with controls, in primary, hospital or community health care settings. Participants included clinical staff and management, district management and community health workers using routine information systems.

DATA COLLECTION AND ANALYSIS

Two authors independently reviewed records to identify studies for inclusion, extracted data from the included studies and assessed the risk of bias. Interventions and outcomes were too varied across studies to allow for pooled risk analysis. We present a 'Summary of findings' table for each intervention comparisons broadly categorised into Technical and Organisational (or a combination), and report outcomes on data quality and service quality. We used the GRADE approach to assess the certainty of the evidence.

MAIN RESULTS

We included six studies: four cluster randomised trials and two controlled before-after studies, from Africa and South America. Three studies evaluated technical interventions, one study evaluated an organisational intervention, and two studies evaluated a combination of technical and organisational interventions. Four studies reported on data quality and six studies reported on service quality. In terms of data quality, a web-based electronic TB laboratory information system probably reduces the length of time to reporting of TB test results, and probably reduces the overall rate of recording errors of TB test results, compared to a paper-based system (moderate certainty evidence). We are uncertain about the effect of the electronic laboratory information system on the recording rate of serious (misidentification) errors for TB test results compared to a paper-based system (very low certainty evidence). Misidentification errors are inaccuracies in transferring test results between an electronic register and patients' clinical charts. We are also uncertain about the effect of the intervention on service quality (timeliness of starting or changing a patient's TB treatment) (very low certainty evidence). A hand-held electronic device probably improves the length of time to report TB test results, and probably reduces the total frequency of recording errors in TB test results between the laboratory notebook and the electronic information record system, compared to a paper-based system (moderate-certainty evidence). We are, however, uncertain about the effect of the intervention on the frequency of serious (misidentification) errors in recording between the laboratory notebook and the electronic information record, compared to a paper-based system (very low certainty evidence). We are uncertain about the effect of a hospital electronic health information system on service quality (length of time outpatients spend at hospital, length of hospital stay, and hospital revenue collection), compared to a paper-based system (very low certainty evidence). High-intensity brief text messaging (SMS) may make little or no difference to data quality (in terms of completeness of documentation of pregnancy outcomes), compared to low-intensity brief text messaging (low-certainty evidence). We are uncertain about the effect of electronic drug stock notification (with either data management support or product transfer support) on service quality (in terms of transporting stock and stock levels), compared to paper-based stock notification (very low certainty evidence). We are uncertain about the effect of health information strengthening (where it is part of comprehensive service quality improvement intervention) on service quality (health worker motivation, receipt of training by health workers, health information index scores, quality of clinical observation of children and adults) (very low certainty evidence).

AUTHORS' CONCLUSIONS

The review indicates mixed effects of mainly technical interventions to improve data quality, with gaps in evidence on interventions aimed at enhancing data-informed health system management. There is a gap in interventions studying information support beyond clinical management, such as for human resources, finances, drug supply and governance. We need to have a better understanding of the causal mechanisms by which information support may affect change in management decision-making, to inform robust intervention design and evaluation methods.

Mobile Phone-Based Behavioral Interventions in Pregnancy to Promote Maternal and Fetal Health in High-Income Countries: Systematic Review

Background

Chronic diseases have recently had an increasing effect on maternal-fetal health, especially in high-income countries. However, there remains a lack of discussion regarding health management with technological approaches, including mobile health (mHealth) interventions.

Objective

This study aimed to systematically evaluate mHealth interventions used in pregnancy in high-income countries and their effects on maternal health behaviors and maternal-fetal health outcomes.

Methods

This systematic review identified studies published between January 1, 2000, and November 30, 2018, in MEDLINE via PubMed, Cochrane Library, EMBASE, CINAHL, PsycINFO, Web of Science, and gray literature. Studies were eligible for inclusion if they included only pregnant women in high-income countries and evaluated stand-alone mobile phone interventions intended to promote healthy maternal beliefs, behaviors, and/or maternal-fetal health outcomes. Two researchers independently reviewed and categorized aspects of full-text articles, including source, study design, intervention and control, duration, participant age, attrition rate, main outcomes, and risk of bias. Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were followed, and the study was registered in PROSPERO before initiation.

Results

Of the 2225 records examined, 28 studies were included and categorized into 4 themes: (1) gestational weight gain, obesity and physical activity (n=9); (2) smoking cessation (n=9); (3) influenza vaccination (n=2); and (4) general prenatal health, preventive strategies, and miscellaneous topics (n=8). Reported sample sizes ranged from 16 to 5243 with a median of 91. Most studies were performed in the United States (18/28, 64%) and were randomized controlled trials (21/28, 75%). All participants in the included studies were pregnant at the time of study initiation. Overall, 14% (4/28) of studies showed association between intervention use and improved health outcomes; all 4 studies focused on healthy gestational weight. Among those, 3 studies showed intervention use was associated with less overall gestational weight gain. These 3 studies involved interventions with text messaging or an app in combination with another communication strategy (Facebook or email). Regarding smoking cessation, influenza vaccination, and miscellaneous topics, there was some evidence of positive effects on health behaviors and beliefs, but very limited correlation with improved health outcomes. Data and interventions were heterogeneous, precluding a meta-analysis.

Conclusions

In high-income countries, utilization of mobile phone–based health behavior interventions in pregnancy demonstrates some correlation with positive beliefs, behaviors, and health outcomes. More effective interventions are multimodal in terms of features and tend to focus on healthy gestational weight gain.

A systematic review of vital events tracking by community health agents

Background: Efforts to improve national civil registration and vital statistics (CRVS) systems are focusing on transforming traditionally passive systems into active systems that have the ability to reach the household level. While community health agents remain at the core of many birth and death reporting efforts, previous literature has not explored elements for their successful integration into CRVS efforts.

Objective: To inform future efforts to improve CRVS systems, we conducted a systematic review of literature to understand and describe the design features, resulting data quality, and factors impacting the performance of community health agents involved in tracking vital events.

Methods: We reviewed 393 articles; reviewers extracted key information from 58 articles meeting the eligibility criteria: collection of birth and/or death information outside of a clinic environment by a community agent. Reviewers recorded information in an Excel database on various program aspects, and results were summarized into key themes and topic areas.

Results: The majority of articles described work in rural areas of Africa or South-East Asia. Nearly all articles (86%) cited some form of household visitation by community health agents. Only one article described a process in which vital events tracking activities were linked to official vital events registers. Other factors commonly described included program costs, relationship of community agents to community, and use of mobile devices. About 1/3 of articles reported quantitative information on performance and quality of vital events data tracked; various methods were described for measuring completeness of reporting, which varied greatly across articles.

Conclusions: The multitude of articles on this topic attests to the availability of community health agents to track vital events. Creating a programmatic norm of integrating with CRVS systems the vital events information collected from existing community health programs has the potential to provide governments with information essential for public health decision-making.

A comparative review of mobile health and electronic health utilization in sub-Saharan African countries

This study distinguished between the application of e-health and m-health technologies in sub-Saharan African (SSA) countries based on the dimensions of use, targeted diseases or health conditions, locations of use, and beneficiaries (types of patients or health workers) in a country specific context. It further characterized the main opportunities and challenges associated with these dimensions across the sub-region. A systematic review of the literature was conducted on 66 published peer reviewed articles. The review followed the scientific process of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines of identification, selection, assessment, synthesis and interpretation of findings. The results of the study showed that m-health was prevalent in usage for promoting information for treatment and prevention of diseases as well as serving as an effective technology for reminders towards adherence. For e-health, the uniqueness lay in data acquisition and patients' records management; diagnosis; training and recruitment. While m-health was never used for monitoring or training and recruitment, e-health on the other hand could not serve the purpose of reminders or for reporting cases from the field. Both technologies were however useful for adherence, diagnosis, disease control mechanisms, information provision, and decision-making/referrals. HIV/AIDS, malaria, and maternal (postnatal and antenatal) healthcare were important in both m-health and e-health interventions mostly concentrated in the rural settings of South Africa and Kenya. ICT infrastructure, trained personnel, illiteracy, lack of multilingual text and voice messages were major challenges hindering the effective usage of both m-health and e-health technologies.

Community-based surveillance: A scoping review

Background

Involving community members in identifying and reporting health events for public health surveillance purposes, an approach commonly described as community-based surveillance (CBS), is increasingly gaining interest. We conducted a scoping review to list terms and definitions used to characterize CBS, to identify and summarize available guidance and recommendations, and to map information on past and existing in-country CBS systems.

Methods

We searched eight bibliographic databases and screened the worldwide web for any document mentioning an approach in which community members both collected and reported information on health events from their community for public health surveillance. Two independent reviewers performed double blind screening and data collection, any discrepancy was solved through discussion and consensus.

Findings

From the 134 included documents, several terms and definitions for CBS were retrieved. Guidance and recommendations for CBS were scattered through seven major guides and sixteen additional documents. Seventy-nine unique CBS systems implemented since 1958 in 42 countries were identified, mostly implemented in low and lower-middle income countries (79%). The systems appeared as fragmented (81% covering a limited geographical area and 70% solely implemented in a rural setting), vertical (67% with a single scope of interest), and of limited duration (median of 6 years for ongoing systems and 2 years for ended systems). Collection of information was mostly performed by recruited community members (80%).

Interpretation

While CBS has already been implemented in many countries, standardization is still required on the term and processes to be used. Further research is needed to ensure CBS integrates effectively into the overall public health surveillance system.

Mobile health and the performance of maternal health care workers in low- and middle-income countries: A realist review

Introduction

Maternal health and the performance of health workers is a key concern in low- and middle-income countries. Mobile health technologies are reportedly able to improve workers' performance. However, how this has been achieved for maternal health workers in low-resource settings is not fully substantiated. To address this gap by building theoretical explanations, two questions were posed: How does mobile health influence the performance of maternal health care workers in low- and middle-income countries? What mechanisms and contextual factors are associated with mobile health use for maternal health service delivery in low- and middle-income countries?

Methods

Guided by established guidelines, a realist review was conducted. Five databases were searched for relevant English language articles published between 2009 and 2016. A three-stage framework was developed and populated with explanatory configurations of Intervention-Context-Actors-Mechanism-Outcome. Articles were analyzed retroductively, with identified factors grouped into meaningful clusters.

Results

Of 1254 records identified, 23 articles representing 16 studies were retained. Four main mechanisms were identified: usability and empowerment explaining mobile health adoption, third-party recognition explaining mobile health utilization, and empowerment of health workers explaining improved competence. Evidence was skewed toward the adoption and utilization stage of the framework, with weak explanations for performance outcomes.

Conclusions

Findings suggest that health workers can be empowered to adopt and utilize mobile health in contexts where it is aligned to their needs, workload, training, and skills. In turn, mobile health can empower health workers with skills and confidence when it is perceived as useful and easy to use, in contexts that foster recognition from clients, peers, or supervisors.

Ongoing training of community health workers in low-income andmiddle-income countries: a systematic scoping review of the literature

OBJECTIVES

Understanding the current landscape of ongoing training for community health workers (CHWs) in low-income and middle-income countries (LMICs) is important both for organisations responsible for their training, as well as researchers and policy makers. This scoping review explores this under-researched area by mapping the current delivery implementation and evaluation of ongoing training provision for CHWs in LMICs.

DESIGN

Systematic scoping review.

DATA SOURCES

MEDLINE, Embase, AMED, Global Health, Web of Science, Scopus, ASSIA, LILACS, BEI and ERIC.

STUDY SELECTION

Original studies focusing on the provision of ongoing training for CHWs working in a country defined as low income and middle income according to World Bank Group 2012 classification of economies.

RESULTS

The scoping review found 35 original studies that met the inclusion criteria. Ongoing training activities for CHWs were described as supervision (n=19), inservice or refresher training (n=13) or a mixture of both (n=3). Although the majority of studies emphasised the importance of providing ongoing training, several studies reported no impact of ongoing training on performance indicators. The majority of ongoing training was delivered inperson; however, four studies reported the use of mobile technologies to support training delivery. The outcomes from ongoing training activities were measured and reported in different ways, including changes in behaviour, attitudes and practice measured in a quantitative manner (n=16), knowledge and skills (n=6), qualitative assessments (n=5) or a mixed methods approach combining one of the aforementioned modalities (n=8).

CONCLUSIONS

This scoping review highlights the diverse range of ongoing training for CHWs in LMICs. Given the expansion of CHW programmes globally, more attention should be given to the design, delivery, monitoring and sustainability of ongoing training from a health systems strengthening perspective.

Using community-based reporting of vital events to monitor child mortality: Lessons from rural Ghana

Background

Reducing neonatal and child mortality is a key component of the health-related sustainable development goal (SDG), but most low and middle income countries lack data to monitor child mortality on an annual basis. We tested a mortality monitoring system based on the continuous recording of pregnancies, births and deaths by trained community-based volunteers (CBV).

Methods and findings

This project was implemented in 96 clusters located in three districts of the Northern Region of Ghana. Community-based volunteers (CBVs) were selected from these clusters and were trained in recording all pregnancies, births, and deaths among children under 5 in their catchment areas. Data collection lasted from January 2012 through September 2013. All CBVs transmitted tallies of recorded births and deaths to the Ghana Birth and deaths registry each month, except in one of the study districts (approximately 80% reporting). Some events were reported only several months after they had occurred. We assessed the completeness and accuracy of CBV data by comparing them to retrospective full pregnancy histories (FPH) collected during a census of the same clusters conducted in October-December 2013. We conducted all analyses separately by district, as well as for the combined sample of all districts. During the 21-month implementation period, the CBVs reported a total of 2,819 births and 137 under-five deaths. Among the latter, there were 84 infant deaths (55 neonatal deaths and 29 post-neonatal deaths). Comparison of the CBV data with FPH data suggested that CBVs significantly under-estimated child mortality: the estimated under-5 mortality rate according to CBV data was only 2/3 of the rate estimated from FPH data (95% Confidence Interval for the ratio of the two rates = 51.7 to 81.4). The discrepancies between the CBV and FPH estimates of infant and neonatal mortality were more limited, but varied significantly across districts.

Conclusions

In northern Ghana, a community-based data collection systems relying on volunteers did not yield accurate estimates of child mortality rates. Additional implementation research is needed to improve the timeliness, completeness and accuracy of such systems. Enhancing pregnancy monitoring, in particular, may be an essential step to improve the measurement of neonatal mortality.

Practical issues in the measurement of child survival in health systems trials: experience developing a digital community-based mortality surveillance programme in rural Nepal

Child mortality measurement is essential to the impact evaluation of maternal and child healthcare systems interventions. In the absence of vital statistics systems, however, assessment methodologies for locally relevant interventions are severely challenged. Methods for assessing the under-5 mortality rate for cross-country comparisons, often used in determining progress towards development targets, pose challenges to implementers and researchers trying to assess the population impact of targeted interventions at more local levels. Here, we discuss the programmatic approach we have taken to mortality measurement in the context of delivering healthcare via a public–private partnership in rural Nepal. Both government officials and the delivery organisation, Possible, felt it was important to understand child mortality at a fine-grain spatial and temporal level. We discuss both the short-term and the long-term approach. In the short term, the team chose to use the under-2 mortality rate as a metric for mortality measurement for the following reasons: (1) as overall childhood mortality declines, like it has in rural Nepal, deaths concentrate among children under the age of 2; (2) 2-year cohorts are shorter and thus may show an impact more readily in the short term of intervention trials; and (3) 2-year cohorts are smaller, making prospective census cohorts more feasible in small populations. In the long term, Possible developed a digital continuous surveillance system to capture deaths as they occur, at which point under-5 mortality assessment would be desirable, largely owing to its role as a global standard.

"Real-Time" Monitoring of Under-Five Mortality: A Vision Tempered by Reality

Jennifer Bryce and the RMM Working Group describe the origin and rationale of the Real-Time Monitoring of Under-Five Mortality Collection.