Linking household survey and health facility data for effective coverage measures: a comparison of ecological and individual linking methods using the Multiple Indicator Cluster Survey in Côte d'Ivoire

Delays in accessing high-quality care for newborns in East Africa: An analysis of survey data in Malawi, Mozambique, and Tanzania

Background

Despite the existence of evidence-based interventions, substantial progress in reducing neonatal mortality is lagging, indicating that small and sick newborns (SSNs) are likely not receiving the care they require to survive and thrive. The 'three delays model' provides a framework for understanding the challenges in accessing care for SSNs. However, the extent to which each delay impacts access to care for SSNs is not well understood. To fill this evidence gap, we explored the impact of each of the three delays on access to care for SSNs in Malawi, Mozambique, and Tanzania.

Methods

Secondary analyses of data from three different surveys served as the foundation of this study. To understand the impact of delays in the decision to seek care (delay 1) and the ability to reach an appropriate point of care (delay 2), we investigated time trends in place of birth disaggregated by facility type. We also explored care-seeking behaviours for newborns who died. To understand the impact of delays in accessing high-quality care after reaching a facility (delay 3), we measured facility readiness to manage care for SSNs. We used this measure to adjust institutional delivery coverage for SSN care readiness.

Results

Coverage of institutional deliveries was substantially lower after adjusting for facility readiness to manage SSN care, with decreases of 30 percentage points (pp) in Malawi, 14 pp in Mozambique, and 24 pp in Tanzania. While trends suggest more SSNs are born in facilities, substantial gaps remain in facilities' capacities to provide lifesaving interventions. In addition, exploration of care-seeking pathways revealed that a substantial proportion of newborn deaths occurred outside of health facilities, indicating barriers in the decision to seek care or the ability to reach an appropriate source of care may also prevent SSNs from receiving these interventions.

Conclusions

Investments are needed to overcome delays in accessing high-quality care for the most vulnerable newborns, those who are born small or sick. As more mothers and newborns access health services in low- and middle-income countries, ensuring that life-saving interventions for SSNs are available at the locations where newborns are born and seek care after birth is critical.

Exploring approaches to weighting estimates of facility readiness to provide health services used for estimating input-adjusted effective coverage: a case study using data from Tanzania

The ideal approach for calculating effective coverage of health services using ecological linking requires accounting for variability in facility readiness to provide health services and patient volume by incorporating adjustments for facility type into estimates of facility readiness and weighting facility readiness estimates by service-specific caseload. The aim of this study is to compare the ideal caseload-weighted facility readiness approach to two alternative approaches: (1) facility-weighted readiness and (2) observation-weighted readiness to assess the suitability of each as a proxy for caseload-weighted facility readiness. We utilised the 2014-2015 Tanzania Service Provision Assessment along with routine health information system data to calculate facility readiness estimates using the three approaches. We then conducted equivalence testing, using the caseload-weighted estimates as the ideal approach and comparing with the facility-weighted estimates and observation-weighted estimates to test for equivalence. Comparing the facility-weighted readiness estimates to the caseload-weighted readiness estimates, we found that 58% of the estimates met the requirements for equivalence. In addition, the facility-weighted readiness estimates consistently underestimated, by a small percentage, facility readiness as compared to the caseload-weighted readiness estimates. Comparing the observation-weighted readiness estimates to the caseload-weighted readiness estimates, we found that 64% of the estimates met the requirements for equivalence. We found that, in this setting, both facility-weighted readiness and observation-weighted readiness may be reasonable proxies for caseload-weighted readiness. However, in a setting with more variability in facility readiness or larger differences in facility readiness between low caseload and high caseload facilities, the observation-weighted approach would be a better option than the facility-weighted approach. While the methods compared showed equivalence, our results suggest that selecting the best method for weighting readiness estimates will require assessing data availability alongside knowledge of the country context.

Effective coverage of newborn postnatal care in Ethiopia: Measuring inequality and spatial distribution of quality-adjusted coverage

Neonatal health is a significant global public health concern, and the first two days of life are crucial for newborn survival. Most studies on newborn postnatal care have focused on crude coverage measures, which limit the evaluation of care quality. However, evidence suggests a shift towards emphasising effective coverage, which incorporates the quality of care when measuring intervention coverage. This research aimed to assess the effective coverage of newborn postnatal care in Ethiopia while also examining its inequalities and spatial distribution. The study used secondary data from the 2016 Ethiopian Demographic and Health Survey, which was a cross-sectional community-based study. A total weighted sample of 4169 women was used for analyses. We calculated crude coverage, which is the proportion who received a postnatal check within 48 hours of birth and quality-adjusted coverage (effective coverage), which is the proportion who received a postnatal check within 48 hours of birth and reported receipt of 6 or more contents of care provided by health care providers. Concentration index and concentration curves were used to estimate the socioeconomic-related inequalities in quality-adjusted newborn postnatal care. The spatial statistic was analysed by using Arc-GIS. The crude coverage of newborn postnatal care was found to be 13.2%, while the effective coverage was 9%. High-quality postnatal care was disproportionately concentrated among the rich. A spatial variation was found in quality-adjusted coverage of newborn postnatal care across regions. The findings suggest that there is a significant gap in the coverage and quality of postnatal care for newborns across regions in Ethiopia. The low rates of coverage and effective coverage, combined with the concentration of high-quality care among the rich and the spatial variation across regions, highlight the need for targeted interventions and policies to address the inequalities in access to high-quality postnatal care for newborns.

Use of a digital job-aid in improving antenatal clinical protocols and quality of care in rural primary-level health facilities in Burkina Faso: a quasi-experimental evaluation

OBJECTIVE

We assessed the impact of a digital clinical decision support (CDS) tool in improving health providers adherence to recommended antenatal protocols and service quality in rural primary-level health facilities in Burkina Faso.

DESIGN

A quasi-experimental evaluation based on a cross-sectional post-intervention assessment comparing the intervention district to a comparison group.

SETTING AND PARTICIPANTS

The study included 331 direct observations and exit interviews of pregnant women seeking antenatal care (ANC) across 48 rural primary-level health facilities in Burkina Faso in 2021.

INTERVENTION

Digital CDS tool to improve health providers adherence to recommended antenatal protocols.

OUTCOME MEASURES

We analysed the quality of care on both the supply and demand sides. Quality-of-care service scores were based on actual care provided and expected care according to standards. Pregnant women's knowledge of counselling and satisfaction score after receiving care were also calculated. Other outcomes included time of clinical encounter.

RESULTS

The overall quality of health service provision was comparable across intervention and comparison health facilities (52% vs 51%) despite there being a significantly higher proportion of lower skilled providers in the intervention arm (42.5% vs 17.8%). On average, ANC visits were longer in the intervention area (median 24 min, IQR 18) versus comparison area (median 12 min, IQR: 8). The intervention arm had a significantly higher score difference in women's knowledge of received counselling (16.4 points, 95% CI 10.37 to 22.49), and women's satisfaction (16.18 points, 95% CI: 9.95 to 22.40).

CONCLUSION

Digital CDS tools provide a valuable opportunity to achieve substantial improvements of the quality of ANC and broadly maternal and newborn health in settings with high burden mortality and less trained health cadres when adequately implemented.

The effect of sampling health facilities on estimates of effective coverage: a simulation study

BACKGROUND

Most existing facility assessments collect data on a sample of health facilities. Sampling of health facilities may introduce bias into estimates of effective coverage generated by ecologically linking individuals to health providers based on geographic proximity or administrative catchment.

METHODS

We assessed the bias introduced to effective coverage estimates produced through two ecological linking approaches (administrative unit and Euclidean distance) applied to a sample of health facilities. Our analysis linked MICS household survey data on care-seeking for child illness and childbirth care with data on service quality collected from a census of health facilities in the Savanes region of Cote d'Ivoire. To assess the bias introduced by sampling, we drew 20 random samples of three different sample sizes from our census of health facilities. We calculated effective coverage of sick child and childbirth care using both ecological linking methods applied to each sampled facility data set. We compared the sampled effective coverage estimates to ecologically linked census-based estimates and estimates based on true source of care. We performed sensitivity analyses with simulated preferential care-seeking from higher-quality providers and randomly generated provider quality scores.

RESULTS

Sampling of health facilities did not significantly bias effective coverage compared to either the ecologically linked estimates derived from a census of facilities or true effective coverage estimates using the original data or simulated random quality sensitivity analysis. However, a few estimates based on sampling in a setting where individuals preferentially sought care from higher-quality providers fell outside of the estimate bounds of true effective coverage. Those cases predominantly occurred using smaller sample sizes and the Euclidean distance linking method. None of the sample-based estimates fell outside the bounds of the ecologically linked census-derived estimates.

CONCLUSIONS

Our analyses suggest that current health facility sampling approaches do not significantly bias estimates of effective coverage produced through ecological linking. Choice of ecological linking methods is a greater source of bias from true effective coverage estimates, although facility sampling can exacerbate this bias in certain scenarios. Careful selection of ecological linking methods is essential to minimize the potential effect of both ecological linking and sampling error.

Evaluations of effective coverage of maternal and child health services: A systematic review

Conventionally used coverage measures do not reflect the quality of care. Effective coverage (EC) assesses the extent to which health care services deliver potential health gains to the population by integrating concepts of utilization, need and quality. We aimed to conduct a systematic review of studies evaluating EC of maternal and child health services, quality measurement strategies and disparities across wealth quantiles. A systematic search was performed in six electronic databases [MEDLINE, EMBASE, Cumulative Index of Nursing and Allied Health (CINAHL), Scopus, Web of Science and Maternity and Infant Care] and grey literature. We also undertook a hand search of references. We developed search terms having no restrictions based on publication period, country or language. We included studies which reported EC estimates based on the World Health Organization framework of measuring EC. Twenty-seven studies, all from low- and middle-income settings (49 countries), met the criteria and were included in the narrative synthesis of the results. Maternal and child health intervention(s) and programme(s) were assessed either at an individual level or as an aggregated measure of health system performance or both. The EC ranged from 0% for post-partum care to 95% for breastfeeding. When crude coverage measures were adjusted to account for the quality of care, the EC values turned lower. The gap between crude coverage and EC was as high as 86%, and it signified a low quality of care. The assessment of the quality of care addressed structural, process and outcome domains individually or combined. The wealthiest 20% had higher EC of services than the poorest 20%, an inequitable distribution of coverage. More efforts are needed to improve the quality of maternal and child health services and to eliminate the disparities. Moreover, considering multiple dimensions of quality and the use of standard measurements are recommended to monitor coverage effectively.

Evidence for an expanded hypertension care cascade in low- and middle-income countries: a scoping review

Background

With nearly 90% of annual hypertension-related deaths occurring in low- and middle-income countries (LMICs), there is an urgent need to measure the coverage of health services that effectively manage hypertension. However, there is little agreement on how to define effective coverage and the existing hypertension care cascade (hypertension prevalence, percent aware, percent treated, and percent controlled) does not account for the quality of care received by patients. This study reviews definitions of effective coverage and service quality for hypertension management services and proposes an expanded hypertension care cascade to improve measurement of health systems performance.

Methods

A systematic scoping review of literature published in six electronic databases between January 2000 and October 2020 identified studies that defined effective coverage of hypertension management services or integrated dimensions of service quality into population-based estimates of hypertension management in LMICs. Findings informed an expanded hypertension care cascade from which quality-adjusted service coverage can be calculated to approximate effective coverage.

Results

The review identified 18 relevant studies, including 6 that defined effective coverage for hypertension management services and 12 that reported a measure of service quality in a population-based study. Based on commonly reported barriers to hypertension management, new steps on the proposed expanded care cascade include (i) population screened, (ii) population linked to quality care, and (iii) population adhering to prescribed treatment.

Conclusion

There is little consensus on the definition of effective coverage of hypertension management services, and most studies do not describe the quality of hypertension management services provided to populations. Incorporating aspects of service quality to the hypertension care cascade allows for the calculation of quality-adjusted coverage of relevant services, enabling an appropriate measurement of health systems performance through effective coverage.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08190-0.

A rapid systematic review and evidence synthesis of effective coverage measures and cascades for childbirth, newborn and child health in low- and middle-income countries

BACKGROUND

Effective coverage measures aim to estimate the proportion of a population in need of a service that received a positive health outcome. In 2020, the Effective Coverage Think Tank Group recommended using a 'coverage cascade' for maternal, newborn, child and adolescent health and nutrition (MNCAHN), which organises components of effective coverage in a stepwise fashion, with each step accounting for different aspects of quality of care (QoC), applied at the population level. The cascade outlines six steps that increase the likelihood that the population in need experience the intended health benefit: 1) the population in need (target population) who contact a health service; 2) that has the inputs available to deliver the service; 3) who receive the health service; 4) according to quality standards; 5) and adhere to prescribed medication(s) or health workers instructions; and 6) experience the expected health outcome. We examined how effective coverage of life-saving interventions from childbirth to children aged nine has been defined and assessed which steps of the cascade are captured by existing measures.

METHODS

We undertook a rapid systematic review. Seven scientific literature databases were searched covering the period from May 1, 2017 to July, 8 2021. Reference lists from reviews published in 2018 and 2019 were examined to identify studies published prior to May 2017. Eligible studies reported population-level contact coverage measures adjusted for at least one dimension of QoC.

RESULTS

Based on these two search approaches this review includes literature published from 2010 to 2021. From 16 662 records reviewed, 33 studies were included, reporting 64 effective coverage measures. The most frequently examined measures were for childbirth and immediate newborn care (n = 24). No studies examined measures among children aged five to nine years. Definitions of effective coverage varied across studies. Key sources of variability included (i) whether a single effective coverage measure was reported for a package of interventions or separate measures were calculated for each intervention; (ii) the number and type of coverage cascade steps applied to adjust for QoC; and (iii) the individual items included in the effective coverage definition and the methods used to generate a composite quality measure.

CONCLUSION

In the MNCAHN literature there is substantial heterogeneity in both definitions and construction of effective coverage, limiting the comparability of measures over time and place. Current measurement approaches are not closely aligned with the proposed cascade. For widespread adoption, there is a need for greater standardisation of indicator definitions and transparency in reporting, so governments can use these measures to improve investments in MNACHN and implement life-saving health policies and programs.

Operationalising effective coverage measurement of facility based childbirth in Gombe State; a comparison of data sources

Estimating effective coverage of childbirth care requires linking population based data sources to health facility data. For effective coverage to gain widespread adoption there is a need to focus on the feasibility of constructing these measures using data typically available to decision makers in low resource settings. We estimated effective coverage of childbirth care in Gombe State, northeast Nigeria, using two different combinations of facility data sources and examined their strengths and limitations for decision makers. Effective coverage captures information on four steps: access, facility inputs, receipt of interventions and process quality. We linked data from the 2018 Nigerian Demographic and Health Survey (NDHS) to two sources of health facility data: (1) comprehensive health facility survey data generated by a research project; and (2) District Health Information Software 2 (DHIS2). For each combination of data sources, we examined which steps were feasible to calculate, the size of the drop in coverage between steps and the resulting estimate of effective coverage. Analysis included 822 women with a recent live birth, 30% of whom attended a facility for childbirth. Effective coverage was low: 2% based on the project data and less than 1% using the DHIS2. Linking project data with NDHS, it was feasible to measure all four steps; using DHIS2 it was possible to estimate three steps: no data was available to measure process quality. The provision of high quality care is suboptimal in this high mortality setting where access and facility readiness to provide care, crucial foundations to the provision of high quality of care, have not yet been met. This study demonstrates that partial effective coverage measures can be constructed from routine data combined with nationally representative surveys. Advocacy to include process of care indicators in facility summary reports could optimise this data source for decision making.

Methodological considerations for linking household and healthcare provider data for estimating effective coverage: a systematic review

OBJECTIVE

To assess existing knowledge related to methodological considerations for linking population-based surveys and health facility data to generate effective coverage estimates. Effective coverage estimates the proportion of individuals in need of an intervention who receive it with sufficient quality to achieve health benefit.

DESIGN

Systematic review of available literature.

DATA SOURCES

Medline, Carolina Population Health Center and Demographic and Health Survey publications and handsearch of related or referenced works of all articles included in full text review. The search included publications from 1 January 2000 to 29 March 2021.

ELIGIBILITY CRITERIA

Publications explicitly evaluating (1) the suitability of data, (2) the implications of the design of existing data sources and (3) the impact of choice of method for combining datasets to obtain linked coverage estimates.

RESULTS

Of 3805 papers reviewed, 70 publications addressed relevant issues. Limited data suggest household surveys can be used to identify sources of care, but their validity in estimating intervention need was variable. Methods for collecting provider data and constructing quality indices were diverse and presented limitations. There was little empirical data supporting an association between structural, process and outcome quality. Few studies addressed the influence of the design of common data sources on linking analyses, including imprecise household geographical information system data, provider sampling design and estimate stability. The most consistent evidence suggested under certain conditions, combining data based on geographical proximity or administrative catchment (ecological linking) produced similar estimates to linking based on the specific provider utilised (exact match linking).

CONCLUSIONS

Linking household and healthcare provider data can leverage existing data sources to generate more informative estimates of intervention coverage and care. However, existing evidence on methods for linking data for effective coverage estimation are variable and numerous methodological questions remain. There is need for additional research to develop evidence-based, standardised best practices for these analyses.

Effective coverage of nutrition interventions across the continuum of care in Bangladesh: insights from nationwide cross-sectional household and health facility surveys

INTRODUCTION

Improving the impact of nutrition interventions requires adequate measurement of both reach and quality of interventions, but limited evidence exists on advancing coverage measurement. We adjusted contact-based coverage estimates, taking into consideration the inputs required to deliver quality nutrition services, to calculate input-adjusted coverage of nutrition interventions across the continuum of care from pregnancy through early childhood in Bangladesh.

METHODS

We used data from the 2014 Bangladesh Demographic and Health Surveys to assess use of maternal and child health services and the 2014 Service Provision Assessment to determine facility readiness to deliver nutrition interventions. Service readiness captured availability of nutrition-specific inputs (including human resources and training, equipment, diagnostics and medicines). Contact coverage was combined with service readiness to create a measure of input-adjusted coverage at the national and regional levels, across place of residence, and by maternal education and household socioeconomic quintiles.

RESULTS

Contact coverage varied from 28% for attending at least four ANC visits to 38% for institutional delivery, 35% for child growth monitoring and 81% for sick child care. Facilities demonstrated incomplete readiness for nutrition interventions, ranging from 48% to 51% across services. Nutrition input-adjusted coverage was suboptimal (18% for ANC, 23% for institutional delivery, 20% for child growth monitoring and 52% for sick child care) and varied between regions within the country. Inequalities in input-adjusted coverage were large during ANC and institutional delivery (14-17 percentage points (pp) between urban and rural areas, 15 pp between low and high education, and 28-34 pp between highest and lowest wealth quintiles) and less variable for sick child care (<2 pp).

CONCLUSION

Nutrition input-adjusted coverage was suboptimal and varied subnationally and across the continuum of care in Bangladesh. Special efforts are needed to improve the reach as well as the quality of health and nutrition services to achieve the Sustainable Development Goals.

Linking household surveys and facility assessments: a comparison of geospatial methods using nationally representative data from Malawi

BACKGROUND

Linking facility and household surveys through geographic methods is a popular technique to draw conclusions about the relationship between health services and population health outcomes at local levels. These methods are useful tools for measuring effective coverage and tracking progress towards Universal Health Coverage, but are understudied. This paper compares the appropriateness of several geospatial methods used for linking individuals (within displaced survey cluster locations) to their source of family planning (at undisplaced health facilities) at a national level.

METHODS

In Malawi, geographic methods linked a population health survey, rural clusters from the Woman's Questionnaire of the 2015 Malawi Demographic and Health Survey (MDHS 2015), to Malawi's national health facility census to understand the service environment where women receive family planning services. Individuals from MDHS 2015 clusters were linked to health facilities through four geographic methods: (i) closest facility, (ii) buffer (5 km), (iii) administrative boundary, and (iv) a newly described theoretical catchment area method. Results were compared across metrics to assess the number of unlinked clusters (data lost), the number of linkages per cluster (precision of linkage), and the number of women linked to their last source of modern contraceptive (appropriateness of linkage).

RESULTS

The closest facility and administrative boundary methods linked every cluster to at least one facility, while the 5-km buffer method left 288 clusters (35.3%) unlinked. The theoretical catchment area method linked all but one cluster to at least one facility (99.9% linked). Closest facility, 5-km buffer, administrative boundary, and catchment methods linked clusters to 1.0, 1.4, 21.1, and 3.3 facilities on average, respectively. Overall, the closest facility, 5-km buffer, administrative boundary, and catchment methods appropriately linked 64.8%, 51.9%, 97.5%, and 88.9% of women to their last source of modern contraceptive, respectively.

CONCLUSIONS

Of the methods studied, the theoretical catchment area linking method loses a marginal amount of population data, links clusters to a relatively low number of facilities, and maintains a high level of appropriate linkages. This linking method is demonstrated at scale and can be used to link individuals to qualities of their service environments and better understand the pathways through which interventions impact health.

Operationalising health systems thinking: a pathway to high effective coverage

Background

The global health community has recognised the importance of defining and measuring the effective coverage of health interventions and their implementation strength to monitor progress towards global mortality and morbidity targets. Existing health system models and frameworks guide thinking around these measurement areas; however, they fall short of adequately capturing the dynamic and multi-level relationships between different components of the health system. These relationships must be articulated for measurement and managed to effectively deliver health interventions of sufficient quality to achieve health impacts. Save the Children’s Saving Newborn Lives programme and EnCompass LLC, its evaluation partner, developed and applied the Pathway to High Effective Coverage as a health systems thinking framework (hereafter referred to as the Pathway) in its strategic planning, monitoring and evaluation.

Methods

We used an iterative approach to develop, test and refine thinking around the Pathway. The initial framework was developed based on existing literature, then shared and vetted during consultations with global health thought leaders in maternal and newborn health.

Results

The Pathway is a robust health systems thinking framework that unpacks system, policy and point of intervention delivery factors, thus encouraging specific actions to address gaps in implementation and facilitate the achievement of high effective coverage. The Pathway includes six main components – (1) national readiness; (2) system structures; (3) management capacity; (4) implementation strength; (5) effective coverage; and (6) impact. Each component is comprised of specific elements reflecting the range of facility-, community- and home-based interventions. We describe applications of the Pathway and results for in-country strategic planning, monitoring of progress and implementation strength, and evaluation.

Conclusions

The Pathway provides a cohesive health systems thinking framework that facilitates assessment and coordinated action to achieve high coverage and impact. Experiences of its application show its utility in guiding strategic planning and in more comprehensive and effective monitoring and evaluation as well as its potential adaptability for use in other health areas and sectors.

Epidemiologic Profile of Overweight and Obesity in Abidjan, Ivory Coast: A Cross-Sectional Study

Background:

In sub-Saharan Africa, the prevalence of overweight and obesity is high, and it is estimated to increase within the next ten years. In Ivory Coast, the rise in and public health consequences of overweight and obese people are evident. Moreover, data concerning this status are scarce, old, local, and describe only a small sample of the population.

Objective:

This study has been conducted in order to describe the epidemiologic profile of overweight and obese people in Ivory Coast and identify the potential risk factors of obesity.

Methods:

From January 2014 to July 2017, 2,643 patients aged 17–70 years old from Abidjan of Ivory Coast were recruited. Statistical analysis was carried out using SPSS 20.0. Chi-square test and binary logistic regression analysis were used to identify risk factors for overweight and obesity.

Results:

Most of our patients were females (86.3%) with an estimated average age of 43.7 ± 12.19 years. Among 2,643 patients recruited in this study, 83.3% were obese and 87.2% were affected by central abdominal obesity. Binary logistic regression analysis identified seven factors significantly associated with overweight and obesity, including females (OR: 2.06; 95% CI [1.58–2.68]), >54 years old of age (OR: 3.71; 95% CI [1.84–7.50]), occupation of salesperson and traders (OR: 2.42; 95% CI [1.78–3.29]), ethnic group of North Mande ethnicity (OR: 1.47; 95% CI [1.07–2.02]), family history of obesity (OR: 1.96; 95% CI [1.46–2.63]), ≥150 minutes of sport practice (OR:0.72; 95% CI [0.55–0.96]), and parous females (OR: 1.63; 95% CI [1.11–2.38]).

Conclusions:

Overall, gender (female), older age, and occupation were associated with greater risks of overweight and obesity in patients. Ethnic group, pregnancy and family history of obesity posed a lower but significant risk for obesity. More sport practice played a protective role against the acquisition of overweight and obesity.

Variance estimation for effective coverage measures: A simulation study

Background

Effective coverage research is increasing rapidly in global health and development, as researchers use a range of measures and combine data sources to adjust coverage for the quality of services received. However, most estimates of effective coverage that combine data sources are reported only as point estimates, which may be due to the challenge of calculating the variance for a composite measure. In this paper, we evaluate three methods to quantify the uncertainty in the estimation of effective coverage.

Methods

We conducted a simulation study to evaluate the performance of the exact, delta, and parametric bootstrap methods for constructing confidence intervals around point estimates that are calculated from combined data on coverage and quality. We assessed performance by computing the number of nominally 95% confidence intervals that contain the truth for a range of coverage and quality values and data source sample sizes. To illustrate these approaches, we applied the delta and exact methods to estimates of adjusted coverage of antenatal care (ANC) in Senegal. We used household survey data for coverage and health facility assessments for readiness to provide services.

Results

With small sample sizes, when the true effective coverage value was close to the boundaries 0 or 1, the exact and parametric bootstrap methods resulted in substantial over or undercoverage and, for the exact method, a high proportion of invalid confidence intervals, while the delta method yielded modest overcoverage. The proportion of confidence intervals containing the truth in all three methods approached the intended 95% with larger sample sizes and as the true effective coverage value moved away from the 0 or 1 boundary. Confidence intervals for adjusted ANC in Senegal were largely overlapping across the delta and exact methods, although at the sub-national level, the exact method produced invalid confidence intervals for estimates near 0 or 1. We provide the code to implement these methods.

Conclusions

The uncertainty around an effective coverage estimate can be characterized; this should become standard practice if effective coverage estimates are to become part of national and global health monitoring. The delta method approach outperformed the other methods in this study; we recommend its use for appropriate inference from effective coverage estimates that combine data sources, particularly when either sample size is small. When used for estimates created from facility type or regional strata, these methods require assumptions of independence that must be considered in each example.

Effective coverage of facility delivery in Bangladesh, Haiti, Malawi, Nepal, Senegal, and Tanzania

BACKGROUND

The persistence of preventable maternal and newborn deaths highlights the importance of quality of care as an essential element in coverage interventions. Moving beyond the conventional measurement of crude coverage, we estimated effective coverage of facility delivery by adjusting for facility preparedness to provide delivery services in Bangladesh, Haiti, Malawi, Nepal, Senegal, and Tanzania.

METHODS

The study uses data from Demographic and Health Surveys (DHS) and Service Provision Assessments (SPA) in Bangladesh (2014 DHS and 2014 SPA), Haiti (2012 DHS and 2013 SPA), Malawi (2015-16 DHS and 2013-14 SPA), Nepal (2016 DHS and 2015 SPA), Senegal (2016 DHS and 2015 SPA), and Tanzania (2015-16 DHS and 2014-15 SPA). We defined effective coverage as the mathematical product of crude coverage and quality of care. The coverage of facility delivery was measured with DHS data and quality of care was measured with facility data from SPA. We estimated effective coverage at both the regional and the national level and accounted for type of facility where delivery care was sought.

FINDINGS

The findings from the six countries indicate the effective coverage ranges from 24% in Haiti to 66% in Malawi, representing substantial reductions (20% to 39%) from crude coverage rates. Although Malawi has achieved almost universal coverage of facility delivery (93%), effective coverage was only 66%.vSuch gaps between the crude coverage and the effective coverage suggest that women delivered in health facility but did not necessarily receive an adequate quality of care. In all countries except Malawi, effective coverage differed substantially among the country's regions of the country, primarily due to regional variability in coverage.

INTERPRETATION

Our findings reinforce the importance of quality of obstetric and newborn care to achieve further reduction of maternal and newborn mortality. Continued efforts are needed to increase the use of facility delivery service in countries or regions where coverage remains low.