Novel approaches to measuring knowledge among frontline health workers in India: Are phone surveys a reliable option?

Assessing the reliability of phone surveys to measure reproductive, maternal and child health knowledge among pregnant women in rural India: a feasibility study

OBJECTIVES

Efforts to understand the factors influencing the uptake of reproductive, maternal, newborn, child health and nutrition (RMNCH&N) services in high disease burden low-resource settings have often focused on face-to-face surveys or direct observations of service delivery. Increasing access to mobile phones has led to growing interest in phone surveys as a rapid, low-cost alternatives to face-to-face surveys. We assess determinants of RMNCH&N knowledge among pregnant women with access to phones and examine the reliability of alternative modalities of survey delivery.

PARTICIPANTS

Women 5-7 months pregnant with access to a phone.

SETTING

Four districts of Madhya Pradesh, India.

DESIGN

Cross-sectional surveys administered face-to-face and within 2 weeks, the same surveys were repeated among two random subsamples of the original sample: face-to-face (n=205) and caller-attended telephone interviews (n=375). Bivariate analyses, multivariable linear regression, and prevalence and bias-adjusted kappa scores are presented.

RESULTS

Knowledge scores were low across domains: 52% for maternal nutrition and pregnancy danger signs, 58% for family planning, 47% for essential newborn care, 56% infant and young child feeding, and 58% for infant and young child care. Higher knowledge (≥1 composite score) was associated with older age; higher levels of education and literacy; living in a nuclear family; primary health decision-making; greater attendance in antenatal care and satisfaction with accredited social health activist services. Survey questions had low inter-rater and intermodal reliability (kappa<0.70) with a few exceptions. Questions with the lowest reliability included true/false questions and those with unprompted, multiple response options. Reliability may have been hampered by the sensitivity of the content, lack of privacy, enumerators' and respondents' profile differences, rapport, social desirability bias, and/or enumerator's ability to adequately convey concepts or probe.

CONCLUSIONS

Phone surveys are a reliable modality for generating population-level estimates data about pregnant women's knowledge, however, should not be used for individual-level tracking.

TRIAL REGISTRATION NUMBER

NCT03576157.

Who do phone surveys miss, and how to reduce exclusion: recommendations from phone surveys in nine Indian states

Computer-assisted telephone interviews (CATI) through mobile phones are a low-cost, rapid and safe way to collect data. However, decisions for how such mobile phone surveys are designed and implemented, and their data analysed, can have implications for the sample reached, and in turn affect the generalisability of sample estimates. In this practice paper, we propose a framework for extending the use of CATI–mobile phone surveys in India, which can be applied broadly to future surveys conducted using this method. Across the stages of design, implementation and analysis, we outline challenges in ensuring that the data collected through such surveys are representative and provide recommendations for reducing non-coverage and non-response errors, thereby enabling practitioners in India to use CATI–mobile phone surveys to estimate population statistics with lower bias. We support our analysis by drawing on primary data that we collected in five mobile phone surveys across nine Indian states in 2020. Our recommendations can help practitioners in India improve the representativeness of data collected through mobile phone surveys and generate more accurate estimates.

Another voice in the crowd: the challenge of changing family planning and child feeding practices through mHealth messaging in rural central India

Introduction

Kilkari is one of the world’s largest mobile phone-based health messaging programmes. Developed by BBC Media Action, it provides weekly stage-based information to pregnant and postpartum women and their families, including on infant and young child feeding (IYCF) and family planning, to compliment the efforts of frontline health workers. The quantitative component of a randomised controlled trial (RCT) in the Indian state of Madhya Pradesh found that exposure to Kilkari increased modern contraceptive uptake but did not change IYCF practices. This qualitative research complements the RCT to explore why these findings may have emerged.

Methods

We used system generated data to identify households within the RCT with very high to medium Kilkari listenership. Mothers (n=29), as well as husbands and extended family members (n=25 interviews/family group discussions) were interviewed about IYCF and family planning, including their reactions to Kilkari’s calls on these topics. Analysis was informed by the theory of reciprocal determinism, which positions behaviour change within the interacting domains of individual attributes, social and environmental determinants, and existing practices.

Results

While women who owned and controlled their own phones were the Kilkari listeners, among women who did not own their own phones, it was often their husbands who listened. Spouses did not discuss Kilkari messages. Respondents retained and appreciated Kilkari messages that aligned with their pre-existing worldviews, social norms, and existing practices. However, they overlooked or de-emphasised content that did not. In this way, they reported agreeing with and trusting Kilkari while persisting with practices that went against Kilkari’s recommendations, particularly non-exclusive breastfeeding and inappropriate complementary feeding.

Conclusion

To deepen impact, digital direct to beneficiary services need to be complimented by wider communication efforts (e.g., sustained face-to-face, media, community engagement) to change social norms, taking into account the role of socio-environmental, behavioural, and individual determinants.

Examining the reach and exposure of a mobile phone-based training programme for frontline health workers (ASHAs) in 13 states across India

Mobile phones are increasingly used to facilitate in-service training for frontline health workers (FLHWs). Mobile learning (mLearning) programmes have the potential to provide FLHWs with high quality, inexpensive, standardised learning at scale, and at the time and location of their choosing. However, further research is needed into FLHW engagement with mLearning content at scale, a factor which could influence knowledge and service delivery. Mobile Academy is an interactive voice response training course for FLHWs in India, which aims to improve interpersonal communication skills and refresh knowledge of preventative reproductive, maternal, neonatal and child health. FLHWs dial in to an audio course consisting of 11 chapters, each with a 4-question true/false quiz, resulting in a cumulative pass/fail score. In this paper, we analyse call data records from the national version of Mobile Academy to explore coverage, user engagement and completion. Over 158 596 Accredited Social Health Activists (ASHAs) initiated the national version, while 111 994 initiated the course on state-based platforms. Together, this represents 41% of the estimated total number of ASHAs registered in the government database across 13 states. Of those who initiated the national version, 81% completed it; and of those, over 99% passed. The initiation and completion rates varied by state, with Rajasthan having the highest initiation rate. Many ASHAs made multiple calls in the afternoons and evenings but called in for longer durations earlier in the day. Findings from this analysis provide important insights into the differential reach and uptake of the programme across states.