Validation of maternal reports for low birthweight and preterm birth indicators in rural Nepal

Estimating birthweight reduction attributable to maternal ozone exposure in low- and middle-income countries

The effect of O3 on birthweight in low- and middle-income countries (LMICs) remains unknown. A multicenter epidemiological study was conducted to evaluate the association between maternal peak-season O3 exposure and birthweight, using 697,148 singleton newborns obtained in 54 LMICs between 2003 and 2019. We estimated the birthweight reduction attributable to peak-season O3 exposure in 123 LMICs based on a nonlinear exposure-response function (ERF). With every 10-part per billion increment in O3 concentration, we found a reduction in birthweight of 19.9 g [95% confidence interval (CI): 14.8 to 24.9 g]. The nonlinear ERF had a monotonic decreasing curve, and no safe O3 exposure threshold was identified. The mean reduction in birthweight reduction attributable to O3 across the 123 LMICs was 43.8 g (95% CI: 30.5 to 54.3 g) in 2019. The reduction in O3-related birthweight was greatest in countries in South Asia, the Middle East, and North Africa. Effective O3 pollution control policies have the potential to substantially improve infant health.

Validation of Parent-reported Gestational Age Categories for Children Less Than 6 Years of Age

BACKGROUND

Preterm birth is an important outcome or exposure in epidemiologic research. When administrative data on measured gestational age is not available, parent-reported gestational age can be obtained from questionnaires, which is subject to potential bias. To our knowledge, few studies have assessed the validity of parent-reported gestational age categories, including commonly defined categories of preterm birth.

METHODS

We used linked data from primarily healthy children <6 years of age in TARGet Kids! in Toronto, Canada, and ICES administrative healthcare data from April 2011 to March 2020. We assessed the criterion validity of questionnaire-based parent-reported gestational age by calculating sensitivity and specificity for term (≥37 weeks), late preterm (34-36 weeks), and moderately preterm (32-33 weeks) gestational age categories, using administrative healthcare records of gestational age as the criterion standard. We conducted subgroup analyses for various parent and socioeconomic factors that may influence recall.

RESULTS

Of the 4684 participants, 97.3% correctly classified the gestational age category according to administrative healthcare data. Parent-reported gestational age sensitivity ranged from 83.7% to 98.5% and specificity ranged from 88.3% to 99.8%, depending on category. For each subgroup characteristic, sensitivity and specificity were all ≥70%. Lower educational attainment, lower family income, father reporting, ≥1 year since birth, ≥2 children, lower parent age, and reported gestational diabetes and/or hypertension were associated with slightly lower sensitivity and/or specificity.

CONCLUSIONS

In this linked cohort, parent-reported gestational age categories had high accuracy. Criterion validity varied minimally among some parent and socioeconomic factors. Our findings can inform future quantitative bias analyses.

Exploring the accuracy of self-reported maternal and newborn care in select studies from low and middle-income country settings: do respondent and facility characteristics affect measurement?

BACKGROUND

Accurate data on the receipt of essential maternal and newborn health interventions is necessary to interpret and address gaps in effective coverage. Validation results of commonly used content and quality of care indicators routinely implemented in international survey programs vary across settings. We assessed how respondent and facility characteristics influenced the accuracy of women's recall of interventions received in the antenatal and postnatal periods.

METHODS

We synthesized reporting accuracy using data from a known sample of validation studies conducted in Sub-Saharan Africa and Southeast Asia, which assessed the validity of women's self-report of received antenatal care (ANC) (N = 3 studies, 3,169 participants) and postnatal care (PNC) (N = 5 studies, 2,462 participants) compared to direct observation. For each study, indicator sensitivity and specificity are presented with 95% confidence intervals. Univariate fixed effects and bivariate random effects models were used to examine whether respondent characteristics (e.g., age group, parity, education level), facility quality, or intervention coverage level influenced the accuracy of women's recall of whether interventions were received.

RESULTS

Intervention coverage was associated with reporting accuracy across studies for the majority (9 of 12) of PNC indicators. Increasing intervention coverage was associated with poorer specificity for 8 indicators and improved sensitivity for 6 indicators. Reporting accuracy for ANC or PNC indicators did not consistently differ by any other respondent or facility characteristic.

CONCLUSIONS

High intervention coverage may contribute to higher false positive reporting (poorer specificity) among women who receive facility-based maternal and newborn care while low intervention coverage may contribute to false negative reporting (lower sensitivity). While replication in other country and facility settings is warranted, results suggest that monitoring efforts should consider the context of care when interpreting national estimates of intervention coverage.

Moderate Accuracy of Survey Responses about Infant and Young Child Feeding Counseling Reported by Mothers with Children Less than 1 Year of Age in India

BACKGROUND

Counseling on infant and young child feeding (IYCF) to support optimal breastfeeding and complementary feeding practices is an essential intervention, and accurate coverage data is needed to identify gaps and monitor progress. However, coverage information captured during household surveys has not yet been validated.

OBJECTIVES

We examined the validity of maternal reports of IYCF counseling received during community-based contacts and factors associated with reporting accuracy.

METHODS

Direct observations of home visits conducted by community workers in 40 villages in Bihar, India served as the "gold standard" to maternal reports of IYCF counseling received during 2-wk follow-up surveys (n = 444 mothers with children less than 1 y of age, interviews matched to direct observations). Individual-level validity was assessed by calculating sensitivity, specificity, and AUC. Population-level bias was measured using the inflation factor (IF). Multivariable regression models were used to examine factors associated with response accuracy.

RESULTS

Prevalence of IYCF counseling during home visits was very high (90.1%). Maternal report of any IYCF counseling received in the past 2 wk was moderate (AUC: 0.60; 95% CI: 0.52, 0.67), and population bias was low (IF = 0.90). However, the recall of specific counseling messages varied. Maternal report of any breastfeeding, exclusive breastfeeding, and dietary diversity messages had moderate validity (AUC > 0.60), but other child feeding messages had low individual validity. Child age, maternal age, maternal education, mental stress, and social desirability were associated with reporting accuracy of multiple indicators.

CONCLUSIONS

Validity of IYCF counseling coverage was moderate for several key indicators. IYCF counseling is an information-based intervention that may be received from various sources, and it may be challenging to achieve higher reporting accuracy over a longer recall period. We consider the modest validity results as positive and suggest that these coverage indicators may be useful for measuring coverage and tracking progress over time.

Cognitive testing of questions about antenatal care and nutrition interventions in southern Nepal

Large scale surveys such as the Demographic and Health Surveys (DHS) are used to measure the coverage and quality of antenatal care (ANC)-related services. Studies have increasingly validated questions from these surveys, though few have explored respondent comprehension or associated thought processes. This study aimed to use cognitive testing and validation approaches to understand how survey respondents understand questions related to ANC-related nutrition services. The study was nested within a larger validation study in southern Nepal. Pregnant women's receipt of ANC related services was directly observed at five health posts followed by a recall interview at 6 months postpartum. A week later, a survey module was re-administered to 30 women containing 15 questions about receipt of ANC care and specifically nutrition-related services. Detailed probing was used to identify cognitive challenges related to comprehension, retrieval, judgement, and response. Respondents accurately recalled the four specific ANC visits recommended by the government of Nepal but those with more visits struggled to estimate the total number of ANC visits they had made. A number of terms including "antenatal care, "nutrition" and "breastfeeding" were challenging for many respondents to understand. Visits to private providers including for ultrasounds were inconsistently included in ANC visit counts suggesting that question wording could better specify the type of care. Many respondents over-estimated the number of iron folic acid (IFA) supplements taken during pregnancy, and recall was challenging. Calculations were based on estimating the number of months between first ANC visit to delivery, and only sometimes factored in missed tablets. Opportunities exist to improve questions to facilitate better comprehension by respondents through a combination of using local terms and explanations, reordering some questions, and adapting questions to better match respondents' approaches to estimating numeric responses.

Validation of maternal report of nutrition-related interventions and counselling during antenatal care in southern Nepal

The delivery of nutrition-related interventions and counselling during antenatal care is critical for a healthy pregnancy for both mother and child. However, the accuracy of maternal reports of many of these services during household surveys has not yet been examined. Our objectives were to assess the validity of the maternal reports of 10 antenatal nutrition interventions, including counselling, and examine associates between maternal characteristics and accuracy. Maternal report of services received collected during a post-partum survey was compared to the gold standard, the direct observation of all women's antenatal care visits. Individual-level validity was assessed by calculating indicator sensitivity, specificity and area under the operating curve (AUC). The inflation factor (IF) measured population-level bias. For five indicators, the high true coverage limited our ability to assess the validity of the maternal reports. There were no indicators that had both high individual-level validity (AUC > 0.70) and low population bias (0.75 < IF < 1.25). Indicators with greater true coverage estimates had higher sensitivity and lower specificity estimates compared to those indicators with lower true coverage. There were no maternal characteristics associated with the accuracy of the report. Maternal report of antenatal nutrition-related interventions and counselling during household surveys was found to have variable validity across indicators. Additional research in settings with varying coverage levels should be considered to best inform antenatal care coverage measurement in household surveys.

To call or not to call: exploring the validity of telephone interviews to derive maternal self-reports of experiences with facility childbirth care in northern Nigeria

Background

To institutionalise respectful maternity care, frequent data on the experience of childbirth care is needed by health facility staff and managers. Telephone interviews have been proposed as a low-cost alternative to derive timely and actionable maternal self-reports of experience of care. However, evidence on the validity of telephone interviews for this purpose is limited.

Methods

Eight indicators of positive maternity care experience and 18 indicators of negative maternity care experience were investigated. We compared the responses from exit interviews with women about their childbirth care experience (reference standard) to follow-up telephone interviews with the same women 14 months after childbirth. We calculated individual-level validity metrics including, agreement, sensitivity, specificity, area under the receiver operating characteristic curve (AUC). We compared the characteristics of women included in the telephone follow-up interviews to those from the exit interviews.

Results

Demographic characteristics were similar between the original exit interview group (n=388) and those subsequently reached for telephone interview (n=294). Seven of the eight positive maternity care experience indicators had reported prevalence higher than 50% at both exit and telephone interviews. For these indicators, agreement between the exit and the telephone interviews ranged between 50% and 92%; seven positive indicators met the criteria for validation analysis, but all had an AUC below 0.6. Reported prevalence for 15 of the 18 negative maternity care experience indicators was lower than 5% at exit and telephone interviews. For these 15 indicators, agreement between exit and telephone interview was high at over 80%. Just three negative indicators met the criteria for validation analysis, and all had an AUC below 0.6.

Conclusions

The telephone interviews conducted 14 months after childbirth did not yield results that were consistent with exit interviews conducted at the time of facility discharge. Women’s reports of experience of childbirth care may be influenced by the location of reporting or changes in the recall of experiences of care over time.

Validation of Maternal Report of Receipt of Iron-Folic Acid Supplementation during Antenatal Care in Rural Southern Nepal

BACKGROUND

Coverage of iron-folic acid (IFA) supplementation is a key indicator for tracking programmatic progress within and across countries. However, the validity of maternal report of this information during household surveys has yet to be determined.

OBJECTIVES

This study aimed to examine the validity of maternal recall of receipt of IFA supplementation during antenatal care (ANC) and factors associated with accuracy of maternal recall.

METHODS

A longitudinal cohort design was employed. The direct observation of the IFA received during all ANC visits at the 5 study health posts served as the "gold standard" to the maternal report of IFA received during the postpartum interview. Individual-level validity was assessed by calculating indicator sensitivity, specificity, and AUC. The inflation factor (IF) measured population-level bias. A multivariable log-binomial model was used to assess factors associated with accurate recall.

RESULTS

The majority (95.8%) of women were observed receiving IFA during pregnancy. Women overreported the number of IFA tablets received compared with what was observed during ANC visits (mean difference: 45 tablets). Maternal report of any IFA receipt was moderate (AUC = 0.60; 95% CI: 0.50, 0.71), and population bias was low (IF = 1.01). However, the individual-level validity was poor across the 7 IFA tablet count categories; the AUC for categories ranged from misleading to moderate. Driven by the trend of maternal overreport, the IF indicated that maternal report drastically underestimated the coverage of lower tablet categories and overestimated the coverage of higher tablet counts. Accuracy of maternal report was not associated with months since last ANC observation nor any maternal characteristics.

CONCLUSIONS

Maternal report of the amount of IFA supplementation received during pregnancy produced extremely biased population coverage and performed poorly to moderately for individual-level validity. It is imperative to improve this indicator because it is used in global frameworks and national program planning.

Effect of birthweight measurement quality improvement on low birthweight prevalence in rural Ethiopia

BACKGROUND

Low birthweight (LBW) (< 2500 g) is a significant determinant of infant morbidity and mortality worldwide. In low-income settings, the quality of birthweight data suffers from measurement and recording errors, inconsistent data reporting systems, and missing data from non-facility births. This paper describes birthweight data quality and the prevalence of LBW before and after implementation of a birthweight quality improvement (QI) initiative in Amhara region, Ethiopia.

METHODS

A comparative pre-post study was performed in selected rural health facilities located in West Gojjam and South Gondar zones. At baseline, a retrospective review of delivery records from February to May 2018 was performed in 14 health centers to collect birthweight data. A birthweight QI initiative was introduced in August 2019, which included provision of high-quality digital infant weight scales (precision 5 g), routine calibration, training in birth weighing and data recording, and routine field supervision. After the QI implementation, birthweight data were prospectively collected from late August to early September 2019, and December 2019 to June 2020. Data quality, as measured by heaping (weights at exact multiples of 500 g) and rounding to the nearest 100 g, and the prevalence of LBW were calculated before and after QI implementation.

RESULTS

We retrospectively reviewed 1383 delivery records before the QI implementation and prospectively measured 1371 newborn weights after QI implementation. Heaping was most frequently observed at 3000 g and declined from 26% pre-initiative to 6.7% post-initiative. Heaping at 2500 g decreased from 5.4% pre-QI to 2.2% post-QI. The percentage of rounding to the nearest 100 g was reduced from 100% pre-initiative to 36.5% post-initiative. Before the QI initiative, the prevalence of recognized LBW was 2.2% (95% confidence interval [CI]: 1.5-3.1) and after the QI initiative increased to 11.7% (95% CI: 10.1-13.5).

CONCLUSIONS

A QI intervention can improve the quality of birthweight measurements, and data measurement quality may substantially affect estimates of LBW prevalence.

Methodological considerations for linking household and healthcare provider data for estimating effective coverage: a systematic review

OBJECTIVE

To assess existing knowledge related to methodological considerations for linking population-based surveys and health facility data to generate effective coverage estimates. Effective coverage estimates the proportion of individuals in need of an intervention who receive it with sufficient quality to achieve health benefit.

DESIGN

Systematic review of available literature.

DATA SOURCES

Medline, Carolina Population Health Center and Demographic and Health Survey publications and handsearch of related or referenced works of all articles included in full text review. The search included publications from 1 January 2000 to 29 March 2021.

ELIGIBILITY CRITERIA

Publications explicitly evaluating (1) the suitability of data, (2) the implications of the design of existing data sources and (3) the impact of choice of method for combining datasets to obtain linked coverage estimates.

RESULTS

Of 3805 papers reviewed, 70 publications addressed relevant issues. Limited data suggest household surveys can be used to identify sources of care, but their validity in estimating intervention need was variable. Methods for collecting provider data and constructing quality indices were diverse and presented limitations. There was little empirical data supporting an association between structural, process and outcome quality. Few studies addressed the influence of the design of common data sources on linking analyses, including imprecise household geographical information system data, provider sampling design and estimate stability. The most consistent evidence suggested under certain conditions, combining data based on geographical proximity or administrative catchment (ecological linking) produced similar estimates to linking based on the specific provider utilised (exact match linking).

CONCLUSIONS

Linking household and healthcare provider data can leverage existing data sources to generate more informative estimates of intervention coverage and care. However, existing evidence on methods for linking data for effective coverage estimation are variable and numerous methodological questions remain. There is need for additional research to develop evidence-based, standardised best practices for these analyses.

Adherence to iron and folic acid supplementation during pregnancy among postnatal mothers seeking maternal and child healthcare at Kakamega level 5 hospital in Kenya: a cross-sectional study

Background: Maternal anaemia is a public health problem worldwide, and its aetiology is linked to iron deficiency. The high nutrient demand during pregnancy exacerbates the condition. To meet the increased nutritional demand, supplementation of iron and folic acid (IFA) is key. The supplements are provided freely to pregnant women during antenatal visits at public health facilities, however, their uptake and adherence in Kenya remain unacceptably low. Methods: A hospital-based cross-sectional study involving 241 postnatal mothers seeking maternal and child healthcare (MCH) care at Kakamega level 5 hospital was conducted. Both quantitative and qualitative data were collected. Quantitative data were collected from 241 eligible postnatal mothers, while qualitative data were obtained through key informant interviews with community health volunteers and healthcare providers. Results: There was a moderate adherence to IFA supplementation (60.6%) during pregnancy among postnatal mothers seeking MCH care at Kakamega level 5 hospital. Some of the reasons for non-adherence stated by the respondents included; IFA related side effects (41.3%), forgetfulness (37.3%) and bad smell of the IFA supplements (10.3%). Higher IFA adherence was noted among the primigravida participants (OR=2.704; 95% CI: 1.262, 5.793; p=0.010) compared to multigravida participants, and those with a higher knowledge level of anaemia (OR=3.215; 95% CI: 1.346, 7.68; p=0.009) compared to their counterparts with low anaemia knowledge. Other factors that showed correlation with IFA adherence were: IFA education, pregnancy counselling before conception and the number of antenatal care visits attained. Conclusion: There is a moderate adherence to IFA supplementation during pregnancy among mothers seeking MCH at Kakamega level 5 hospital. The greatest impediments of IFA compliance during pregnancy are IFA side effects, forgetfulness and the bad smell of the IFA tablets. Therefore, providing IFA education to pregnant mothers incorporated with probable ways of managing the IFA side effects would contribute to IFA supplementation adherence.

Survey of women's report for 33 maternal and newborn indicators: EN-BIRTH multi-country validation study

BACKGROUND

Population-based household surveys, notably the Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS), remain the main source of maternal and newborn health data for many low- and middle-income countries. As part of the Every Newborn Birth Indicators Research Tracking in Hospitals (EN-BIRTH) study, this paper focuses on testing validity of measurement of maternal and newborn indicators around the time of birth (intrapartum and postnatal) in survey-report.

METHODS

EN-BIRTH was an observational study testing the validity of measurement for selected maternal and newborn indicators in five secondary/tertiary hospitals in Bangladesh, Nepal and Tanzania, conducted from July 2017 to July 2018. We compared women's report at exit survey with the gold standard of direct observation or verification from clinical records for women with vaginal births. Population-level validity was assessed by validity ratios (survey-reported coverage: observer-assessed coverage). Individual-level accuracy was assessed by sensitivity, specificity and percent agreement. We tested indicators already in DHS/MICS as well as indicators with potential to be included in population-based surveys, notably the first validation for small and sick newborn care indicators.

RESULTS

33 maternal and newborn indicators were evaluated. Amongst nine indicators already present in DHS/MICS, validity ratios for baby dried or wiped, birthweight measured, low birthweight, and sex of baby (female) were between 0.90-1.10. Instrumental birth, skin-to-skin contact, and early initiation of breastfeeding were highly overestimated by survey-report (2.04-4.83) while umbilical cord care indicators were massively underestimated (0.14-0.22). Amongst 24 indicators not currently in DHS/MICS, two newborn contact indicators (kangaroo mother care 1.00, admission to neonatal unit 1.01) had high survey-reported coverage amongst admitted newborns and high sensitivity. The remaining indicators did not perform well and some had very high "don't know" responses.

CONCLUSIONS

Our study revealed low validity for collecting many maternal and newborn indicators through an exit survey instrument, even with short recall periods among women with vaginal births. Household surveys are already at risk of overload, and some specific clinical care indicators do not perform well and may be under-powered. Given that approximately 80% of births worldwide occur in facilities, routine registers should also be explored to track coverage of key maternal and newborn health interventions, particularly for clinical care.

Birthweight measurement processes and perceived value: qualitative research in one EN-BIRTH study hospital in Tanzania

BACKGROUND

Globally an estimated 20.5 million liveborn babies are low birthweight (LBW) each year, weighing less than 2500 g. LBW babies have increased risk of mortality even beyond the neonatal period, with an ongoing risk of stunting and non-communicable diseases. LBW is a priority global health indicator. Now almost 80% of births are in facilities, yet birthweight data are lacking in most high-mortality burden countries and are of poor quality, notably with heaping especially on values ending in 00. We aimed to undertake qualitative research in a regional hospital in Dar es Salaam, Tanzania, observing birthweight weighing scales, exploring barriers and enablers to weighing at birth as well as perceived value of birthweight data to health workers, women and stakeholders.

METHODS

Observations were undertaken on type of birthweight scale availability in hospital wards. In-depth semi-structured interviews (n = 21) were conducted with three groups: women in postnatal and kangaroo mother care wards, health workers involved in birthweight measurement and recording, and stakeholders involved in data aggregation in Temeke Hospital, Tanzania, a site in the EN-BIRTH study. An inductive thematic analysis was undertaken of translated interview transcripts.

RESULTS

Of five wards that were expected to have scales, three had functional scales, and only one of the functional scales was digital. The labour ward weighed the most newborns using an analogue scale that was not consistently zeroed. Hospital birthweight data were aggregated monthly for reporting into the health management information system. Birthweight measurement was highly valued by all respondents, notably families and healthcare workers, and local use of data was considered an enabler. Perceived barriers to high quality birthweight data included: gaps in availability of precise weighing devices, adequate health workers and imprecise measurement practices.

CONCLUSION

Birthweight measurement is valued by families and health workers. There are opportunities to close the gap between the percentage of babies born in facilities and the percentage accurately weighed at birth by providing accurate scales, improving skills training and increasing local use of data. More accurate birthweight data are vitally important for all babies and specifically to track progress in preventing and improving immediate and long-term care for low birthweight children.

A, Ruysen H, El Arifeen S, Mee P, Gladstone ME, Blencowe H, Lawn JE. Birthweight: EN-BIRTH multi-country validation study

BACKGROUND

Accurate birthweight is critical to inform clinical care at the individual level and tracking progress towards national/global targets at the population level. Low birthweight (LBW) < 2500 g affects over 20.5 million newborns annually. However, data are lacking and may be affected by heaping. This paper evaluates birthweight measurement within the Every Newborn Birth Indicators Research Tracking in Hospitals (EN-BIRTH) study.

METHODS

The EN-BIRTH study took place in five hospitals in Bangladesh, Nepal and Tanzania (2017-2018). Clinical observers collected time-stamped data (gold standard) for weighing at birth. We compared accuracy for two data sources: routine hospital registers and women's report at exit interview survey. We calculated absolute differences and individual-level validation metrics. We analysed birthweight coverage and quality gaps including timing and heaping. Qualitative data explored barriers and enablers for routine register data recording.

RESULTS

Among 23,471 observed births, 98.8% were weighed. Exit interview survey-reported weighing coverage was 94.3% (90.2-97.3%), sensitivity 95.0% (91.3-97.8%). Register-reported coverage was 96.6% (93.2-98.9%), sensitivity 97.1% (94.3-99%). Routine registers were complete (> 98% for four hospitals) and legible > 99.9%. Weighing of stillbirths varied by hospital, ranging from 12.5-89.0%. Observed LBW rate was 15.6%; survey-reported rate 14.3% (8.9-20.9%), sensitivity 82.9% (75.1-89.4%), specificity 96.1% (93.5-98.5%); register-recorded rate 14.9%, sensitivity 90.8% (85.9-94.8%), specificity 98.5% (98-99.0%). In surveys, "don't know" responses for birthweight measured were 4.7%, and 2.9% for knowing the actual weight. 95.9% of observed babies were weighed within 1 h of birth, only 14.7% with a digital scale. Weight heaping indices were around two-fold lower using digital scales compared to analogue. Observed heaping was almost 5% higher for births during the night than day. Survey-report further increased observed birthweight heaping, especially for LBW babies. Enablers to register birthweight measurement in qualitative interviews included digital scale availability and adequate staffing.

CONCLUSIONS

Hospital registers captured birthweight and LBW prevalence more accurately than women's survey report. Even in large hospitals, digital scales were not always available and stillborn babies not always weighed. Birthweight data are being captured in hospitals and investment is required to further improve data quality, researching of data flow in routine systems and use of data at every level.

Gestational age data completeness, quality and validity in population-based surveys: EN-INDEPTH study

BACKGROUND

Preterm birth (gestational age (GA) <37 weeks) is the leading cause of child mortality worldwide. However, GA is rarely assessed in population-based surveys, the major data source in low/middle-income countries. We examined the performance of new questions to measure GA in household surveys, a subset of which had linked early pregnancy ultrasound GA data.

METHODS

The EN-INDEPTH population-based survey of 69,176 women was undertaken (2017-2018) in five Health and Demographic Surveillance System sites in Bangladesh, Ethiopia, Ghana, Guinea-Bissau and Uganda. We included questions regarding GA in months (GAm) for all women and GA in weeks (GAw) for a subset; we also asked if the baby was 'born before expected' to estimate preterm birth rates. Survey data were linked to surveillance data in two sites, and to ultrasound pregnancy dating at <24 weeks in one site. We assessed completeness and quality of reported GA. We examined the validity of estimated preterm birth rates by sensitivity and specificity, over/under-reporting of GAw in survey compared to ultrasound by multinomial logistic regression, and explored perceptions about GA and barriers and enablers to its reporting using focus group discussions (n = 29).

RESULTS

GAm questions were almost universally answered, but heaping on 9 months resulted in underestimation of preterm birth rates. Preference for reporting GAw in even numbers was evident, resulting in heaping at 36 weeks; hence, over-estimating preterm birth rates, except in Matlab where the peak was at 38 weeks. Questions regarding 'born before expected' were answered but gave implausibly low preterm birth rates in most sites. Applying ultrasound as the gold standard in Matlab site, sensitivity of survey-GAw for detecting preterm birth (GAw <37) was 60% and specificity was 93%. Focus group findings suggest that women perceive GA to be important, but usually counted in months. Antenatal care attendance, women's education and health cards may improve reporting.

CONCLUSIONS

This is the first published study assessing GA reporting in surveys, compared with the gold standard of ultrasound. Reporting GAw within 5 years' recall is feasible with high completeness, but accuracy is affected by heaping. Compared to ultrasound-GAw, results are reasonably specific, but sensitivity needs to be improved. We propose revised questions based on the study findings for further testing and validation in settings where pregnancy ultrasound data and/or last menstrual period dates/GA recorded in pregnancy are available. Specific training of interviewers is recommended.

Birthweight data completeness and quality in population-based surveys: EN-INDEPTH study

BACKGROUND

Low birthweight (< 2500 g) is an important marker of maternal health and is associated with neonatal mortality, long-term development and chronic diseases. Household surveys remain an important source of population-based birthweight information, notably Demographic and Health Surveys (DHS) and UNICEF's Multiple Indicator Cluster Surveys (MICS); however, data quality concerns remain. Few studies have addressed how to close these gaps in surveys.

METHODS

The EN-INDEPTH population-based survey of 69,176 women was undertaken in five Health and Demographic Surveillance System sites (Matlab-Bangladesh, Dabat-Ethiopia, Kintampo-Ghana, Bandim-Guinea-Bissau, IgangaMayuge-Uganda). Responses to existing DHS/MICS birthweight questions on 14,411 livebirths were analysed and estimated adjusted odds ratios (aORs) associated with reporting weighing, birthweight and heaping reported. Twenty-eight focus group discussions with women and interviewers explored barriers and enablers to reporting birthweight.

RESULTS

Almost all women provided responses to birthweight survey questions, taking on average 0.2 min to answer. Of all babies, 62.4% were weighed at birth, 53.8% reported birthweight and 21.1% provided health cards with recorded birthweight. High levels of heterogeneity were observed between sites. Home births and neonatal deaths were less likely to be weighed at birth (home births aOR 0.03(95%CI 0.02-0.03), neonatal deaths (aOR 0.19(95%CI 0.16-0.24)), and when weighed, actual birthweight was less likely to be known (aOR 0.44(95%CI 0.33-0.58), aOR 0.30(95%CI 0.22-0.41)) compared to facility births and post-neonatal survivors. Increased levels of maternal education were associated with increases in reporting weighing and knowing birthweight. Half of recorded birthweights were heaped on multiples of 500 g. Heaping was more common in IgangaMayuge (aOR 14.91(95%CI 11.37-19.55) and Dabat (aOR 14.25(95%CI 10.13-20.3) compared to Bandim. Recalled birthweights were more heaped than those recorded by card (aOR 2.59(95%CI 2.11-3.19)). A gap analysis showed large missed opportunity between facility birth and known birthweight, especially for neonatal deaths. Qualitative data suggested that knowing their baby's weight was perceived as valuable by women in all sites, but lack of measurement and poor communication, alongside social perceptions and spiritual beliefs surrounding birthweight, impacted women's ability to report birthweight.

CONCLUSIONS

Substantial data gaps remain for birthweight data in household surveys, even amongst facility births. Improving the accuracy and recording of birthweights, and better communication with women, for example using health cards, could improve survey birthweight data availability and quality.

Community-based cluster randomized controlled trial: empowering households to identify and provide appropriate care for low-birthweight newborns in Nepal

Background

Most newborn deaths occur among those of low birthweight (LBWt), due to prematurity &/or impaired fetal growth. Simple practices can substantially mitigate this risk. In low-income country settings where many births occur at home, strategies are needed that empower mothers to determine if their babies are at higher risk and, if so, to take measures to reduce risk. Earlier studies suggest that foot-length may be a good proxy for birthweight. An earlier Nepal study found a 6.9 cm cut-off performed relatively well, differentiating normal from low birthweight.

Methods

Community-based, cluster-randomized controlled trial. Objective: to determine whether family-administered screening, associated with targeted messages improves care practices known to mitigate LBWt-associated risks. Participants: women participating in a parent trial in rural Nepal, recruited late in pregnancy. Women were given a 6.9 cm card to assess whether the baby’s foot is small; if so, to call a number on the card for advice. Follow-up visits were made over the 2 weeks following the birth, assessing for 2 behavioral outcomes: reported skin-to-skin thermal care, and care-seeking outside the home; assessed restricting to low birthweight (using 2 cutoffs: 2500 g and 2000 g). Randomization: 17 clusters intervention, 17 control.

The study also documented performance along the presumed causal chain from intervention through behavioral impact.

Results

2022 intervention, 2432 control. Intervention arm: 519 had birthweight < 2500 g (vs. 663 among controls), of which 503 were available for analysis (vs. 649 among controls). No significant difference found on care-seeking; for those < 2500 g RR 1.13 (95%CI: 0.97–1.131). A higher proportion of those in the intervention arm reported skin-to-skin thermal care than among controls; for those < 2500 g RR 2.50 (95%CI: 2.01–3.1). However, process measures suggest this apparent effect cannot be attributed to the intervention; the card performed poorly as a proxy for LBWt, misclassifying 84.5% of those < 2000 as normal weight.

Conclusions

Although the trial found an apparent effect on one of the behavioral outcomes, this cannot be attributed to the intervention; most likely it was a result of pure chance. Other approaches are needed for identifying small, at-risk babies in such settings, and targeting them for appropriate care messaging.

Trial registration

ClinicalTrials.gov NCT02802332, registered 6/16/2016.

What is meant by validity in maternal and newborn health measurement? A conceptual framework for understanding indicator validation

BACKGROUND

Rigorous monitoring supports progress in achieving maternal and newborn mortality and morbidity reductions. Recent work to strengthen measurement for maternal and newborn health highlights the existence of a large number of indicators being used for this purpose. The definitions and data sources used to produce indicator estimates vary and challenges exist with completeness, accuracy, transparency, and timeliness of data. The objective of this study is to create a conceptual overview of how indicator validity is defined and understood by those who develop and use maternal and newborn health indicators.

METHODS

A conceptual framework of validity was developed using mixed methods. We were guided by principles for conceptual frameworks and by a review of the literature and key maternal and newborn health indicator guidance documents. We also conducted qualitative semi-structured interviews with 32 key informants chosen through purposive sampling.

RESULTS

We categorised indicator validity into three main types: criterion, convergent, and construct. Criterion or diagnostic validity, comparing a measure with a gold standard, has predominantly been used to assess indicators of care coverage and content. Studies assessing convergent validity quantify the extent to which two or more indicator measurement approaches, none of which is a gold-standard, relate. Key informants considered construct validity, or the accuracy of the operationalisation of a concept or phenomenon, a critical part of the overall assessment of indicator validity.

CONCLUSION

Given concerns about the large number of maternal and newborn health indicators currently in use, a more consistent understanding of validity can help guide prioritization of key indicators and inform development of new indicators. All three types of validity are relevant for evaluating the performance of maternal and newborn health indicators. We highlight the need to establish a common language and understanding of indicator validity among the various global and local stakeholders working within maternal and newborn health.

Validating women’s reports of antenatal and postnatal care received in Bangladesh, Cambodia and Kenya

Background

Global indicators for monitoring progress in maternal and newborn health have tended to rely on contact coverage indicators rather than the content of services received. As part of the effort to improve measurement of progress in maternal and newborn health, this study examines how accurately women can report on information and health interventions received during an antenatal or postnatal health consultation at health facilities in Bangladesh, Cambodia and Kenya.

Methods

We conducted secondary analysis of matched observation and client interview data to compare women’s reports of care received at exit interview with observation by a trained third-party observer. We assessed indicator accuracy by calculating sensitivity, specificity, area under the receiver operating characteristic curve (AUC) and inflation factor (IF). Indicators considered to have both high individual accuracy (an AUC value of 0.70 or greater) and low population-level bias (0.75<IF<1.25) were considered to have acceptable validity. In addition, we considered the number of countries where both validation criteria were met.

Results

For indicators of antenatal care, we found 16 of 18 indicators in Bangladesh, 3 of 6 in Cambodia and 3 of 8 in Kenya met both validation criteria. For postnatal care, we found evidence of acceptable validity for 6 of 8 indicators in Bangladesh, 5 of 14 in Cambodia and 3 of 16 in Kenya. In general, we documented higher validity for indicators related to concrete, observable actions, as opposed to information or advice given. Women were more likely to recall care received for themselves, rather than for their newborn.

Conclusions

Women reported accurately on multiple aspects of antenatal and postnatal care. While we describe broad patterns in the types of indicators likely to be recalled with accuracy, differences by setting warrant further investigation. Findings inform efforts to better monitor the coverage and quality of maternal and newborn health interventions.

Monitoring childbirth care in primary health facilities: a validity study in Gombe State, northeastern Nigeria

BACKGROUND

Improving the quality of facility-based births is a critical strategy for reducing the high burden of maternal and neonatal mortality and morbidity across all settings. Accurate data on childbirth care is essential for monitoring progress. In northeastern Nigeria, we assessed the validity of childbirth care indicators in a rural primary health care context, as documented by health workers and reported by women at different recall periods.

METHODS

We compared birth observations (gold standard) to: (i) facility exit interviews with observed women; (ii) household follow-up interviews 9-22 months after childbirth; and (iii) health worker documentation in the maternity register. We calculated sensitivity, specificity, and area under the receiver operating curve (AUC) to determine individual-level reporting accuracy. We calculated the inflation factor (IF) to determine population-level validity.

RESULTS

Twenty-five childbirth care indicators were assessed to validate health worker documentation and women's self-reports. During exit interviews, women's recall had high validity (AUC≥0.70 and 0.75

CONCLUSIONS

In addition to standard household surveys, monitoring of facility-based childbirth care should consider drawing from and linking multiple data sources, including routine health facility data and exit interviews with recently delivered women.

Collapse

Key Words Collapse

MESH Headings

• Adolescent
• Adult
• Delivery, Obstetric/standards
• Documentation/methods
• Female
• Humans
• Infant, Newborn
• Medical Records
• Middle Aged
• Nigeria
• Pregnancy
• Primary Health Care
• Reproducibility of Results
• Rural Health Services
• Self Report
• Young Adult

Collapse

Grants Collapse

Affiliation(s)

Antoinette Alas Bhattacharya Department of Disease Control, London School of Hygiene & Tropical Medicine, London, UK
Elizabeth Allen Department of Medical Statistics, London School of Hygiene & Tropical Medicine, London, UK
Nasir Umar Department of Disease Control, London School of Hygiene & Tropical Medicine, London, UK
Adamu Umar Usman Data Research and Mapping Consult, Abuja, Nigeria
Habila Felix State Primary Health Care Development Agency, Gombe, Nigeria
Ahmed Audu State Primary Health Care Development Agency, Gombe, Nigeria
Joanna Rm Schellenberg Department of Disease Control, London School of Hygiene & Tropical Medicine, London, UK
Tanya Marchant Department of Disease Control, London School of Hygiene & Tropical Medicine, London, UK

Collapse

National, regional, and worldwide estimates of low birthweight in 2015, with trends from 2000: a systematic analysis

Background

Low birthweight (LBW) of less than 2500 g is an important marker of maternal and fetal health, predicting mortality, stunting, and adult-onset chronic conditions. Global nutrition targets set at the World Health Assembly in 2012 include an ambitious 30% reduction in LBW prevalence between 2012 and 2025. Estimates to track progress towards this target are lacking; with this analysis, we aim to assist in setting a baseline against which to assess progress towards the achievement of the World Health Assembly targets.

Methods

We sought to identify all available LBW input data for livebirths for the years 2000–16. We considered population-based national or nationally representative datasets for inclusion if they contained information on birthweight or LBW prevalence for livebirths. A new method for survey adjustment was developed and used. For 57 countries with higher quality time-series data, we smoothed country-reported trends in birthweight data by use of B-spline regression. For all other countries, we estimated LBW prevalence and trends by use of a restricted maximum likelihood approach with country-level random effects. Uncertainty ranges were obtained through bootstrapping. Results were summed at the regional and worldwide level.

Findings

We collated 1447 country-years of birthweight data (281 million births) for 148 countries of 195 UN member states (47 countries had no data meeting inclusion criteria). The estimated worldwide LBW prevalence in 2015 was 14·6% (uncertainty range [UR] 12·4–17·1) compared with 17·5% (14·1–21·3) in 2000 (average annual reduction rate [AARR] 1·23%). In 2015, an estimated 20·5 million (UR 17·4–24·0 million) livebirths were LBW, 91% from low-and-middle income countries, mainly southern Asia (48%) and sub-Saharan Africa (24%).

Interpretation

Although these estimates suggest some progress in reducing LBW between 2000 and 2015, achieving the 2·74% AARR required between 2012 and 2025 to meet the global nutrition target will require more than doubling progress, involving both improved measurement and programme investments to address the causes of LBW throughout the lifecycle.

Funding

Bill & Melinda Gates Foundation, The Children's Investment Fund Foundation, United Nations Children's Fund (UNICEF), and WHO.

Validity of self-reported receipt of iron supplements during pregnancy: implications for coverage measurement

Background

Iron-deficiency anemia during pregnancy is an underlying cause of maternal deaths, and reducing risk through routine iron supplementation is a key component of antenatal care (ANC) programs in most low- and middle income countries. Supplementation coverage during pregnancy is estimated from maternal self-reports in population-based household surveys, yet recall bias and social desirability bias lead to errors of unknown magnitude.

Methods

We linked data from household and health facility surveys from 16 countries to estimate input-adjusted coverage of iron supplementation during pregnancy. We assessed the validity of reported receipt of iron supplements in client exit interviews using direct observation as the gold standard across 9 countries with a recent Service Provision Assessment (SPA). Using a sample of 227 women who participated in the Nepal Oil Massage Study (NOMS), we also assessed the validity of self-reported receipt of iron folic acid (IFA) supplements. We used Poisson regression models to explore the association between client and health facility characteristics and agreement of self-reported receipt of iron supplements compared to direct observation.

Results

Across the 16 countries, iron supplements were in supply at most of the 9215 sampled health facilities offering ANC services (91%). We estimated that between 48 and 93% of women attended at least one ANC visit at a health facility with iron supplements available. The specificity of recall of receipt of iron supplementation immediately following a visit was 79.3% and the sensitivity was 88.7% for the entire sample. Individual-level accuracy was high (Area under the curve > 0.7) and population bias low (0.75 < inflation factor < 1.25) across all countries. By contrast, in the NOMS sub-study, the accuracy of self-reported receipt of IFA supplements after 1–2 years was poor (sensitivity 86.1%, specificity 34.3%). Adjusted regression analyses indicated that older age and higher level of education were associated with poorer agreement between self-reports and direct observation.

Conclusions

These findings suggest the need for caution when using self-reported measures with an extended recall period. Further validation studies using conditions similar to widely used population-based household surveys are warranted.

Why some mothers overestimate birth size and length of pregnancy in rural Nepal

Background

Quantitative validation studies alone may not be able to distinguish between instances when participants did not accurately report an event vs when participants did not understand a question. We used an explanatory qualitative study design to acquire an in-depth understanding of why some mothers in rural Nepal overestimate birth size of their newborn and their length of pregnancy.

Methods

We conducted two focus group discussions (FGDs) with study staff who administered a quantitative questionnaire and 12 in-depth interviews (IDIs) with mothers who had participated in a quantitative validation study. Transcripts were coded and analyzed for themes in patterns of meaning within and across FGDs and IDIs. Using this thematic map, we synthesized our data into common and divergent responses from participants to facilitate our interpretation of the quantitative findings.

Results

We identified five themes specific to this analysis: difficulties with the length of pregnancy question, challenges in administering the birth size question, the perceived effect of time since birth on mothers' ability to remember information, the language and style differences specific to this setting, and the study context shaping the relationship between study staff and mothers who participated and how this may have influenced mothers' responses. Visual aids may help to scale the question about birth size within a cultural frame of reference for maternal reports to be more interpretable. Among both study staff and mothers, a longer period of time since the birth of a child was thought to be associated with diminished accuracy of maternal reports, a perception not supported by our previously published quantitative findings.

Conclusions

Poor validity of low birth weight (LBW) and preterm birth indicators based on maternal reports may be partly attributed to challenges in maternal understanding of questions assessing birth size and length of pregnancy. Additional research is needed to confirm these findings regarding maternal comprehension and to further evaluate the utility of visual aids developed for this study.

Validation studies for population-based intervention coverage indicators: design, analysis, and interpretation

Background

Population-based intervention coverage indicators are widely used to track country and program progress in improving health and to evaluate health programs. Indicator validation studies that compare survey responses to a “gold standard” measure are useful to understand whether the indicator provides accurate information. The Improving Coverage Measurement (ICM) Core Group has developed and implemented a standard approach to validating coverage indicators measured in household surveys, described in this paper.

Methods

The general design of these studies includes measurement of true health status and intervention receipt (gold standard), followed by interviews with the individuals observed, and a comparison of the observations (gold standard) to the responses to survey questions. The gold standard should use a data source external to the respondent to document need for and receipt of an intervention. Most frequently, this is accomplished through direct observation of clinical care, and/or use of a study-trained clinician to obtain a gold standard diagnosis. Follow-up interviews with respondents should employ standard survey questions, where they exist, as well as alternative or additional questions that can be compared against the standard household survey questions.

Results

Indicator validation studies should report on participation at every stage, and provide data on reasons for non-participation. Metrics of individual validity (sensitivity, specificity, area under the receiver operating characteristic curve) and population-level validity (inflation factor) should be reported, as well as the percent of survey responses that are “don’t know” or missing. Associations between interviewer and participant characteristics and measures of validity should be assessed and reported.

Conclusions

These methods allow respondent-reported coverage measures to be validated against more objective measures of need for and receipt of an intervention, and should be considered together with cognitive interviewing, discriminative validity, or reliability testing to inform decisions about which indicators to include in household surveys. Public health researchers should assess the evidence for validity of existing and proposed household survey coverage indicators and consider validation studies to fill evidence gaps.