Reliability of community health worker collected data for planning and policy in a peri-urban area of Kisumu, Kenya

Interviewer biases in medical survey data: The example of blood pressure measurements

Health agencies rely upon survey-based physical measures to estimate the prevalence of key global health indicators such as hypertension. Such measures are usually collected by nonhealthcare worker personnel and are potentially subject to measurement error due to variations in interviewer technique and setting, termed "interviewer effects." In the context of physical measurements, particularly in low- and middle-income countries, interviewer-induced biases have not yet been examined. Using blood pressure as a case study, we aimed to determine the relative contribution of interviewer effects on the total variance of blood pressure measurements in three large nationally representative health surveys from the Global South. We utilized 169,681 observations between 2008 and 2019 from three health surveys (Indonesia Family Life Survey, National Income Dynamics Study of South Africa, and Longitudinal Aging Study in India). In a linear mixed model, we modeled systolic blood pressure as a continuous dependent variable and interviewer effects as random effects alongside individual factors as covariates. To quantify the interviewer effect-induced uncertainty in hypertension prevalence, we utilized a bootstrap approach comparing subsamples of observed blood pressure measurements to their adjusted counterparts. Our analysis revealed that the proportion of variation contributed by interviewers to blood pressure measurements was statistically significant but small: ∼ 0.24 - - 2.2 % depending on the cohort. Thus, hypertension prevalence estimates were not substantially impacted at national scales. However, individual extreme interviewers could account for measurement divergences as high as 12%. Thus, highly biased interviewers could have important impacts on hypertension estimates at the subdistrict level.

Wealth and Sedentary Time Are Associated With Dietary Patterns Among Preadolescents in Nairobi City, Kenya

OBJECTIVE

The study aimed to compare dietary patterns in preadolescents in urban areas with different physical activity and socioeconomic profiles in Nairobi, Kenya.

DESIGN

Cross-sectional.

PARTICIPANTS

Preadolescents aged 9-14 years (n = 149) living in low- or middle-income areas in Nairobi.

VARIABLES MEASURED

Sociodemographic characteristics were collected using a validated questionnaire. Weight and height were measured. Diet was assessed using a food frequency questionnaire and physical activity by accelerometer.

ANALYSIS

Dietary patterns (DP) were formed through principal component analysis. Associations of age, sex, parental education, wealth, body mass index, physical activity, and sedentary time with DPs were analyzed with linear regression.

RESULTS

Three DPs explained 36% of the total variance in food consumption: (1) snacks, fast food, and meat; (2) dairy products and plant protein; and (3) vegetables and refined grains. Higher wealth was associated with higher scores of the first DP (P < 0.05).

CONCLUSIONS AND IMPLICATIONS

Consumption of foods often deemed unhealthy (eg, snacks and fast food) was more frequent among preadolescents whose families were wealthier. Interventions that seek ways to promote healthy lifestyles among families residing in urban areas of Kenya are warranted.

Comparing two data collection methods to track vital events in maternal and child health via community health workers in rural Nepal

BACKGROUND

Timely tracking of health outcomes is difficult in low- and middle-income countries without comprehensive vital registration systems. Community health workers (CHWs) are increasingly collecting vital events data while delivering routine care in low-resource settings. It is necessary, however, to assess whether routine programmatic data collected by CHWs are sufficiently reliable for timely monitoring and evaluation of health interventions. To study this, we assessed the consistency of vital events data recorded by CHWs using two methodologies-routine data collected while delivering an integrated maternal and child health intervention, and data from a birth history census approach at the same site in rural Nepal.

METHODS

We linked individual records from routine programmatic data from June 2017 to May 2018 with those from census data, both collected by CHWs at the same site using a mobile platform. We categorized each vital event over a one-year period as 'recorded by both methods,' 'census alone,' or 'programmatic alone.' We further assessed whether vital events data recorded by both methods were classified consistently.

RESULTS

From June 2017 to May 2018, we identified a total of 713 unique births collectively from the census (birth history) and programmatic maternal 'post-delivery' data. Three-fourths of these births (n = 526) were identified by both. There was high consistency in birth location classification among the 526 births identified by both methods. Upon including additional programmatic 'child registry' data, we identified 746 total births, of which 572 births were identified by both census and programmatic methods. Programmatic data (maternal 'post-delivery' and 'child registry' combined) captured more births than census data (723 vs. 595). Both methods consistently classified most infants as 'living,' while infant deaths and stillbirths were largely classified inconsistently or recorded by only one method. Programmatic data identified five infant deaths and five stillbirths not recorded in census data.

CONCLUSIONS

Our findings suggest that data collected by CHWs from routinely tracking pregnancies, births, and deaths are promising for timely program monitoring and evaluation. Despite some limitations, programmatic data may be more sensitive in detecting vital events than cross-sectional census surveys asking women to recall these events.

Cost-effectiveness of community health systems strengthening: quality improvement interventions at community level to realise maternal and child health gains in Kenya

INTRODUCTION

Improvements in maternal and infant health outcomes are policy priorities in Kenya. Achieving these outcomes depends on early identification of pregnancy and quality of primary healthcare. Quality improvement interventions have been shown to contribute to increases in identification, referral and follow-up of pregnant women by community health workers. In this study, we evaluate the cost-effectiveness of using quality improvement at community level to reduce maternal and infant mortality in Kenya.

METHODS

We estimated the cost-effectiveness of quality improvement compared with standard of care treatment for antenatal and delivering mothers using a decision tree model and taking a health system perspective. We used both process (antenatal initiation in first trimester and skilled delivery) and health outcomes (maternal and infant deaths averted, as well as disability-adjusted life years (DALYs)) as our effectiveness measures and actual implementation costs, discounting costs only. We conducted deterministic and probabilistic sensitivity analyses.

RESULTS

We found that the community quality improvement intervention was more cost-effective compared with standard community healthcare, with incremental cost per DALY averted of $249 under the deterministic analysis and 76% likelihood of cost-effectiveness under the probabilistic sensitivity analysis using a standard threshold. The deterministic estimate of incremental cost per additional skilled delivery was US$10, per additional early antenatal care presentation US$155, per maternal death averted US$5654 and per infant death averted US$37 536 (2017 dollars).

CONCLUSIONS

This analysis shows that the community quality improvement intervention was cost-effective compared with the standard community healthcare in Kenya due to improvements in antenatal care uptake and skilled delivery. It is likely that quality improvement interventions are a good investment and may also yield benefits in other health areas.

'Do you trust those data?'-a mixed-methods study assessing the quality of data reported by community health workers in Kenya and Malawi

High-quality data are essential to monitor and evaluate community health worker (CHW) programmes in low- and middle-income countries striving towards universal health coverage. This mixed-methods study was conducted in two purposively selected districts in Kenya (where volunteers collect data) and two in Malawi (where health surveillance assistants are a paid cadre). We calculated data verification ratios to quantify reporting consistency for selected health indicators over 3 months across 339 registers and 72 summary reports. These indicators are related to antenatal care, skilled delivery, immunization, growth monitoring and nutrition in Kenya; new cases, danger signs, drug stock-outs and under-five mortality in Malawi. We used qualitative methods to explore perceptions of data quality with 52 CHWs in Kenya, 83 CHWs in Malawi and 36 key informants. We analysed these data using a framework approach assisted by NVivo11. We found that only 15% of data were reported consistently between CHWs and their supervisors in both contexts. We found remarkable similarities in our qualitative data in Kenya and Malawi. Barriers to data quality mirrored those previously reported elsewhere including unavailability of data collection and reporting tools; inadequate training and supervision; lack of quality control mechanisms; and inadequate register completion. In addition, we found that CHWs experienced tensions at the interface between the formal health system and the communities they served, mediated by the social and cultural expectations of their role. These issues affected data quality in both contexts with reports of difficulties in negotiating gender norms leading to skipping sensitive questions when completing registers; fabrication of data; lack of trust in the data; and limited use of data for decision-making. While routine systems need strengthening, these more nuanced issues also need addressing. This is backed up by our finding of the high value placed on supportive supervision as an enabler of data quality.

Evaluating concordance between government administrative data and externally collected data among high-volume government health facilities in Uttar Pradesh, India

Background: Globally, opportunities to validate government reports through external audits are rare, notably in India. A cross-sectional maternal health study in Uttar Pradesh, India's most populous state, compares government administrative data and externally collected data on maternal health service indicators. Objectives: Our study aims to determine the level of concordance between government-reported health facility data compared to externally collected health facility data on the same maternal healthcare quality indicators. Second, our study aims to explore whether the level of agreement between government administrative data versus the externally collected data differs by level of facility or by type of maternal health service. Methods: Facility assessment surveys were administered to key health staff by government-hired enumerators from January 2017 to March 2017 at nearly 750 government health facilities across UP. The same survey was re-conducted by external data collectors from August 2017 to October 2017 at 40 of the same facilities. We conduct comparative analyses of the two datasets for agreement among the same measures of maternal healthcare quality. Results: The findings indicate concordance between most indicators across government administrative data and externally collected health facility data. However, when stratified by facility-level or service type, results suggest significant over-reporting in the government administrative data on indicators that are incentivized. This finding is consistent across all levels of facilities; however, the most significant disparities appear at higher-level facilities, namely District Hospitals. Conclusion: This study has a number of important programmatic and policy implications. Government administrative health data have the potential to be highly critical in informing large-scale quality improvements in maternal healthcare quality, but its credibility must be readily verifiable and accessible to politicians, researchers, funders, and most importantly, the public, to improve the overall health, patient experience, and well-being of women and newborns.

Is quality affordable for community health systems? Costs of integrating quality improvement into close-to-community health programmes in five low-income and middle-income countries

Introduction

Countries aspiring to universal health coverage view close-to-community (CTC) providers as a low-cost means of increasing coverage. However, due to lack of coordination and unreliable funding, the quality of large-scale CTC healthcare provision is highly variable and routine data about service quality are not trustworthy. Quality improvement (QI) approaches are a means of addressing these issues, yet neither the costs nor the budget impact of integrating QI approaches into CTC programme costs have been assessed.

Methods

This paper examines the costs and budget impact of integrating QI into existing CTC health programmes in five countries (Ethiopia, Indonesia, Kenya, Malawi, Mozambique) between 2015 and 2017. The intervention involved: (1) QI team formation; (2) Phased training interspersed with supportive supervision; which resulted in (3) QI teams independently collecting and analysing data to conduct QI interventions. Project costs were collected using an ingredients approach from a health systems perspective. Based on project costs, costs of local adoption of the intervention were modelled under three implementation scenarios.

Results

Annualised economic unit costs ranged from $62 in Mozambique to $254 in Ethiopia per CTC provider supervised, driven by the context, type of community health model and the intensity of the intervention. The budget impact of Ministry-led QI for community health is estimated at 0.53% or less of the general government expenditure on health in all countries (and below 0.03% in three of the five countries).

Conclusion

CTC provision is a key component of healthcare delivery in many settings, so QI has huge potential impact. The impact is difficult to establish conclusively, but as a first step we have provided evidence to assess affordability of QI for community health. Further research is needed to assess whether QI can achieve the level of benefits that would justify the required investment.

Enhancing the capacity and effectiveness of community health volunteers to improve maternal, newborn and child health: Experience from Kenya

OBJECTIVE

To determine whether a simple monitoring and tracking tool, Mwanzo Mwema Monitoring and Tracking Tool (MMATT), would enable community health volunteers (CHVs) in Kenya to 1) plan their workloads and activities, 2) identify the women, newborns and children most in need of accessing critical maternal, newborn and child health (MNCH) interventions and 3) improve key MNCH indicators.

METHODS

A mixed methods approach was used. Household surveys at baseline (n = 912) and endline (n = 1143) collected data on key MNCH indicators in the four subcounties of Taita Taveta County, Kenya. Eight focus group discussions were held with 40 CHVs to ascertain their perspectives on using the tool.

RESULTS

Qualitative findings revealed that the CHVs found the MMATT to be useful in planning their activities and prioritizing beneficiaries requiring more support to access MNCH services. They also identified potential barriers to care at both the community and health system levels. At endline, previously pregnant women were more likely to have received four or more antenatal care visits, facility delivery, postnatal care within two weeks of delivery and a complete package of care than baseline respondents. Among women with children under 24 months, those at endline were more likely to report early breastfeeding and exclusive breastfeeding for the first six months. These results remained after adjustment for age, subcounty, gravida, mother's education and asset index.

CONCLUSION

Our results demonstrate that simple tools enable CHVs to identify disparities in service delivery and health outcomes, and to identify barriers to MNCH care. Tools that enhance CHVs' ability to plan and prioritize the women and children most in need increase CHVs' potential impact.

How do we know? An assessment of integrated community case management data quality in four districts of Malawi

The World Health Organization contracted annual data quality assessments of Rapid Access Expansion (RAcE) projects to review integrated community case management (iCCM) data quality and the monitoring and evaluation (M&E) system for iCCM, and to suggest ways to improve data quality. The first RAcE data quality assessment was conducted in Malawi in January 2014 and we present findings pertaining to data from the health management information system at the community, facility and other sub-national levels because RAcE grantees rely on that for most of their monitoring data. We randomly selected 10 health facilities (10% of eligible facilities) from the four RAcE project districts, and collected quantitative data with an adapted and comprehensive tool that included an assessment of Malawi's M&E system for iCCM data and a data verification exercise that traced selected indicators through the reporting system. We rated the iCCM M&E system across five function areas based on interviews and observations, and calculated verification ratios for each data reporting level. We also conducted key informant interviews with Health Surveillance Assistants and facility, district and central Ministry of Health staff. Scores show a high-functioning M&E system for iCCM with some deficiencies in data management processes. The system lacks quality controls, including data entry verification, a protocol for addressing errors, and written procedures for data collection, entry, analysis and management. Data availability was generally high except for supervision data. The data verification process identified gaps in completeness and consistency, particularly in Health Surveillance Assistants' record keeping. Staff at all levels would like more training in data management. This data quality assessment illuminates where an otherwise strong M&E system for iCCM fails to ensure some aspects of data quality. Prioritizing data management with documented protocols, additional training and approaches to create efficient supervision practices may improve iCCM data quality.

Combining Healthcare-Based and Participatory Approaches to Surveillance: Trends in Diarrheal and Respiratory Conditions Collected by a Mobile Phone System by Community Health Workers in Rural Nepal

Background

Surveillance systems are increasingly relying upon community-based or crowd-sourced data to complement traditional facilities-based data sources. Data collected by community health workers during the routine course of care could combine the early warning power of community-based data collection with the predictability and diagnostic regularity of facility data. These data could inform public health responses to epidemics and spatially-clustered endemic diseases. Here, we analyze data collected on a daily basis by community health workers during the routine course of clinical care in rural Nepal. We evaluate if such community-based surveillance systems can capture temporal trends in diarrheal diseases and acute respiratory infections.

Methods

During the course of their clinical activities from January to December 2013, community health workers recorded healthcare encounters using mobile phones. In parallel, we accessed condition-specific admissions from 2011–2013 in the hospital from which the community health program was based. We compared diarrhea and acute respiratory infection rates from both the hospital and the community, and assigned three categories of local disease activity (low, medium, and high) to each week in each village cluster with categories determined by tertiles. We compared condition-specific mean hospital rates across categories using ANOVA to assess concordance between hospital and community-collected data.

Results

There were 2,710 cases of diarrhea and 373 cases of acute respiratory infection reported by community health workers during the one-year study period. At the hospital, the average weekly incidence of diarrhea and acute respiratory infections over the three-year period was 1.8 and 3.9 cases respectively per 1,000 people in each village cluster. In the community, the average weekly rate of diarrhea and acute respiratory infections was 2.7 and 0.5 cases respectively per 1,000 people. Both diarrhea and acute respiratory infections exhibited significant differences between the three categories of disease rate burden (diarrhea p = 0.009, acute respiratory infection p = 0.001) when comparing community health worker-collected rates to hospital rates.

Conclusion

Community-level data on diarrhea and acute respiratory infections modestly correlated with hospital data for the same condition in each village each week. Our experience suggests that community health worker-collected data on mobile phones may be a feasible adjunct to other community- and healthcare-related data sources for surveillance of such conditions. Such systems are vitally needed in resource-limited settings like rural Nepal.

Data for Program Management: An Accuracy Assessment of Data Collected in Household Registers by Community Health Workers in Southern Kayonza, Rwanda

Community health workers (CHWs) collect data for routine services, surveys and research in their communities. However, quality of these data is largely unknown. Utilizing poor quality data can result in inefficient resource use, misinformation about system gaps, and poor program management and effectiveness. This study aims to measure CHW data accuracy, defined as agreement between household registers compared to household member interview and client records in one district in Eastern province, Rwanda. We used cluster-lot quality assurance sampling to randomly sample six CHWs per cell and six households per CHW. We classified cells as having 'poor' or 'good' accuracy for household registers for five indicators, calculating point estimates of percent of households with accurate data by health center. We evaluated 204 CHW registers and 1,224 households for accuracy across 34 cells in southern Kayonza. Point estimates across health centers ranged from 79 to 100% for individual indicators and 61 to 72% for the composite indicator. Recording error appeared random for all but the widely under-reported number of women on modern family planning method. Overall, accuracy was largely 'good' across cells, with varying results by indicator. Program managers should identify optimum thresholds for 'good' data quality and interventions to reach them according to data use. Decreasing variability and improving quality will facilitate potential of these routinely-collected data to be more meaningful for community health program management. We encourage further studies assessing CHW data quality and the impact training, supervision and other strategies have on improving it.

Utilization and Monetization of Healthcare Data in Developing Countries

In developing countries with fledgling healthcare systems, the efficient deployment of scarce resources is paramount. Comprehensive community health data and machine learning techniques can optimize the allocation of resources to areas, epidemics, or populations most in need of medical aid or services. However, reliable data collection in low-resource settings is challenging due to a wide range of contextual, business-related, communication, and technological factors. Community health workers (CHWs) are trusted community members who deliver basic health education and services to their friends and neighbors. While an increasing number of programs leverage CHWs for last mile data collection, a fundamental challenge to such programs is the lack of tangible incentives for the CHWs. This article describes potential applications of health data in developing countries and reviews the challenges to reliable data collection. Four practical CHW-centric business models that provide incentive and accountability structures to facilitate data collection are presented. Creating and strengthening the data collection infrastructure is a prerequisite for big data scientists, machine learning experts, and public health administrators to ultimately elevate and transform healthcare systems in resource-poor settings.

Performance of community health workers managing malaria, pneumonia and diarrhoea under the community case management programme in central Uganda: a cross sectional study

BACKGROUND

Lay community health workers (CHWs) have been widely used to provide curative interventions in communities that have traditionally lacked access to health care. Optimal performance of CHWs managing children with malaria, pneumonia and diarrhoea in communities is desired if a reduction in childhood morbidity and mortality is to be achieved. This study assessed factors influencing performance of CHWs managing malaria, pneumonia and diarrhoea under the Integrated Community Case Management (iCCM) programme in Wakiso district, central Uganda.

METHODS

A cross sectional study was conducted among 336 CHWs. Data was collected using interviews and record reviews. Performance was measured using composite scores based on the core activities of CHWs under the iCCM programme. These core activities included: treating children under five years, referring severely sick children including newborns, home visits, counseling caregivers on home care, record keeping and community sensitization. Descriptive and inferential statistics using odds ratios were done to determine factors influencing performance of CHWs.

RESULTS

Of the 336 respondents, 242 (72%) were females and the overall level of good performance was 21.7% (95% CI, 17.3-26.1%). Factors significantly associated with performance were: sex (females) (AOR 2.65; 95% CI, 1.29 -5.43), community support (AOR 2.29; 95% CI, 1.27-4.14), receiving feedback from health facilities (AOR 4.90; 95% CI, 2.52-9.51) and having drugs in the previous three months (AOR 2.99; 95% CI, 1.64-5.42).

CONCLUSION

Only one in every five CHWs performed optimally under the iCCM programme. Strategies to improve drug supply, community support and feedback provision from the formal health system are necessary to improve the performance of CHWs.

Utilizing community health worker data for program management and evaluation: systems for data quality assessments and baseline results from Rwanda

Community health workers (CHWs) have and continue to play a pivotal role in health services delivery in many resource-constrained environments. The data routinely generated through these programs are increasingly relied upon for providing information for program management, evaluation and quality assurance. However, there are few published results on the quality of CHW-generated data, and what information exists suggests quality is low. An ongoing challenge is the lack of routine systems for CHW data quality assessments (DQAs). In this paper, we describe a system developed for CHW DQAs and results of the first formal assessment in southern Kayonza, Rwanda, May-June 2011. We discuss considerations for other programs interested in adopting such systems. While the results identified gaps in the current data quality, the assessment also identified opportunities for strengthening the data to ensure suitable levels of quality for use in management and evaluation.