Evaluation of methods for linking household and health care provider data to estimate effective coverage of management of child illness: results of a pilot study in Southern Province, Zambia

Private sector quality of care for maternal, new-born, and child health in low-and-middle-income countries: a secondary review

The private sector has emerged as a crucial source of maternal, newborn, and child health (MNCH) care in many low- and middle-income countries (LMICs). Quality within the MNCH private sector varies and has not been established systematically. This study systematically reviews findings on private-sector delivery of quality MNCH care in LMICs through the six domains of quality care (QoC) (i.e., efficiency, equity, effectiveness, people-centered care, safety, and timeliness). We registered the systematic review with PROSPERO international prospective register of systematic reviews (registration number CRD42019143383) and followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement for clear and transparent reporting of systematic reviews and meta-analyses. Searches were conducted in eight electronic databases and two websites. For inclusion, studies in LMICs must have examined at least one of the following outcomes using qualitative, quantitative, and/or mixed-methods: maternal morbidity, maternal mortality, newborn morbidity, newborn mortality, child morbidity, child mortality, service utilization, quality of care, and/or experience of care including respectful care. Outcome data was extracted for descriptive statistics and thematic analysis. Of the 139 included studies, 110 studies reported data on QoC. Most studies reporting on QoC occurred in India (19.3%), Uganda (12.3%), and Bangladesh (8.8%). Effectiveness was the most widely measured quality domain with 55 data points, followed by people-centered care (n = 52), safety (n = 47), timeliness (n = 31), equity (n = 24), and efficiency (n = 4). The review showed inconsistencies in care quality across private and public facilities, with quality varying across the six domains. Factors such as training, guidelines, and technical competence influenced the quality. There were also variations in how domains like "people-centered care" have been understood and measured over time. The review underscores the need for clearer definitions of "quality" and practical QoC measures, central to the success of Sustainable Development Goals (SDGs) and equitable health outcomes. This research addresses how quality MNCH care has been defined and operationalized to understand how quality is delivered across the private health sector and the larger health system. Numerous variables and metrics under each QoC domain highlight the difficulty in systematizing QoC. These findings have practical significance to both researchers and policymakers.

Systematic Review Registration

https://bmjopen.bmj.com/content/10/2/e033141.long, Identifier [CRD42019143383].

Delays in accessing high-quality care for newborns in East Africa: An analysis of survey data in Malawi, Mozambique, and Tanzania

Background

Despite the existence of evidence-based interventions, substantial progress in reducing neonatal mortality is lagging, indicating that small and sick newborns (SSNs) are likely not receiving the care they require to survive and thrive. The 'three delays model' provides a framework for understanding the challenges in accessing care for SSNs. However, the extent to which each delay impacts access to care for SSNs is not well understood. To fill this evidence gap, we explored the impact of each of the three delays on access to care for SSNs in Malawi, Mozambique, and Tanzania.

Methods

Secondary analyses of data from three different surveys served as the foundation of this study. To understand the impact of delays in the decision to seek care (delay 1) and the ability to reach an appropriate point of care (delay 2), we investigated time trends in place of birth disaggregated by facility type. We also explored care-seeking behaviours for newborns who died. To understand the impact of delays in accessing high-quality care after reaching a facility (delay 3), we measured facility readiness to manage care for SSNs. We used this measure to adjust institutional delivery coverage for SSN care readiness.

Results

Coverage of institutional deliveries was substantially lower after adjusting for facility readiness to manage SSN care, with decreases of 30 percentage points (pp) in Malawi, 14 pp in Mozambique, and 24 pp in Tanzania. While trends suggest more SSNs are born in facilities, substantial gaps remain in facilities' capacities to provide lifesaving interventions. In addition, exploration of care-seeking pathways revealed that a substantial proportion of newborn deaths occurred outside of health facilities, indicating barriers in the decision to seek care or the ability to reach an appropriate source of care may also prevent SSNs from receiving these interventions.

Conclusions

Investments are needed to overcome delays in accessing high-quality care for the most vulnerable newborns, those who are born small or sick. As more mothers and newborns access health services in low- and middle-income countries, ensuring that life-saving interventions for SSNs are available at the locations where newborns are born and seek care after birth is critical.

Association between antenatal care facility readiness and provision of care at the client level and facility level in five low- and middle-income countries

BACKGROUND

Despite growing interest in monitoring improvements in quality of care, data on service quality in low-income and middle-income countries (LMICs) is limited. While health systems researchers have hypothesized the relationship between facility readiness and provision of care, there have been few attempts to quantify this relationship in LMICs. This study assesses the association between facility readiness and provision of care for antenatal care at the client level and facility level.

METHODS

To assess the association between provision of care and various facility readiness indices for antenatal care, we used multilevel, multivariable random-effects linear regression models. We tested an inflection point on readiness scores by fitting linear spline models. To compare the coefficients between models, we used a bootstrapping approach and calculated the mean difference between all pairwise comparisons. Analyses were conducted at client and facility levels.

RESULTS

Our results showed a small, but significant association between facility readiness and provision of care across countries and most index constructions. The association was most evident in the client-level analyses that had a larger sample size and were adjusted for factors at the facility, health worker, and individual levels. In addition, spline models at a facility readiness score of 50 better fit the data, indicating a plausible threshold effect.

CONCLUSIONS

The results of this study suggest that facility readiness is not a proxy for provision of care, but that there is an important association between facility readiness and provision of care. Data on facility readiness is necessary for understanding the foundations of health systems particularly in countries with the lowest levels of service quality. However, a comprehensive view of quality of care should include both facility readiness and provision of care measures.

Evaluations of effective coverage of maternal and child health services: A systematic review

Conventionally used coverage measures do not reflect the quality of care. Effective coverage (EC) assesses the extent to which health care services deliver potential health gains to the population by integrating concepts of utilization, need and quality. We aimed to conduct a systematic review of studies evaluating EC of maternal and child health services, quality measurement strategies and disparities across wealth quantiles. A systematic search was performed in six electronic databases [MEDLINE, EMBASE, Cumulative Index of Nursing and Allied Health (CINAHL), Scopus, Web of Science and Maternity and Infant Care] and grey literature. We also undertook a hand search of references. We developed search terms having no restrictions based on publication period, country or language. We included studies which reported EC estimates based on the World Health Organization framework of measuring EC. Twenty-seven studies, all from low- and middle-income settings (49 countries), met the criteria and were included in the narrative synthesis of the results. Maternal and child health intervention(s) and programme(s) were assessed either at an individual level or as an aggregated measure of health system performance or both. The EC ranged from 0% for post-partum care to 95% for breastfeeding. When crude coverage measures were adjusted to account for the quality of care, the EC values turned lower. The gap between crude coverage and EC was as high as 86%, and it signified a low quality of care. The assessment of the quality of care addressed structural, process and outcome domains individually or combined. The wealthiest 20% had higher EC of services than the poorest 20%, an inequitable distribution of coverage. More efforts are needed to improve the quality of maternal and child health services and to eliminate the disparities. Moreover, considering multiple dimensions of quality and the use of standard measurements are recommended to monitor coverage effectively.

A rapid systematic review and evidence synthesis of effective coverage measures and cascades for childbirth, newborn and child health in low- and middle-income countries

BACKGROUND

Effective coverage measures aim to estimate the proportion of a population in need of a service that received a positive health outcome. In 2020, the Effective Coverage Think Tank Group recommended using a 'coverage cascade' for maternal, newborn, child and adolescent health and nutrition (MNCAHN), which organises components of effective coverage in a stepwise fashion, with each step accounting for different aspects of quality of care (QoC), applied at the population level. The cascade outlines six steps that increase the likelihood that the population in need experience the intended health benefit: 1) the population in need (target population) who contact a health service; 2) that has the inputs available to deliver the service; 3) who receive the health service; 4) according to quality standards; 5) and adhere to prescribed medication(s) or health workers instructions; and 6) experience the expected health outcome. We examined how effective coverage of life-saving interventions from childbirth to children aged nine has been defined and assessed which steps of the cascade are captured by existing measures.

METHODS

We undertook a rapid systematic review. Seven scientific literature databases were searched covering the period from May 1, 2017 to July, 8 2021. Reference lists from reviews published in 2018 and 2019 were examined to identify studies published prior to May 2017. Eligible studies reported population-level contact coverage measures adjusted for at least one dimension of QoC.

RESULTS

Based on these two search approaches this review includes literature published from 2010 to 2021. From 16 662 records reviewed, 33 studies were included, reporting 64 effective coverage measures. The most frequently examined measures were for childbirth and immediate newborn care (n = 24). No studies examined measures among children aged five to nine years. Definitions of effective coverage varied across studies. Key sources of variability included (i) whether a single effective coverage measure was reported for a package of interventions or separate measures were calculated for each intervention; (ii) the number and type of coverage cascade steps applied to adjust for QoC; and (iii) the individual items included in the effective coverage definition and the methods used to generate a composite quality measure.

CONCLUSION

In the MNCAHN literature there is substantial heterogeneity in both definitions and construction of effective coverage, limiting the comparability of measures over time and place. Current measurement approaches are not closely aligned with the proposed cascade. For widespread adoption, there is a need for greater standardisation of indicator definitions and transparency in reporting, so governments can use these measures to improve investments in MNACHN and implement life-saving health policies and programs.

Operationalising effective coverage measurement of facility based childbirth in Gombe State; a comparison of data sources

Estimating effective coverage of childbirth care requires linking population based data sources to health facility data. For effective coverage to gain widespread adoption there is a need to focus on the feasibility of constructing these measures using data typically available to decision makers in low resource settings. We estimated effective coverage of childbirth care in Gombe State, northeast Nigeria, using two different combinations of facility data sources and examined their strengths and limitations for decision makers. Effective coverage captures information on four steps: access, facility inputs, receipt of interventions and process quality. We linked data from the 2018 Nigerian Demographic and Health Survey (NDHS) to two sources of health facility data: (1) comprehensive health facility survey data generated by a research project; and (2) District Health Information Software 2 (DHIS2). For each combination of data sources, we examined which steps were feasible to calculate, the size of the drop in coverage between steps and the resulting estimate of effective coverage. Analysis included 822 women with a recent live birth, 30% of whom attended a facility for childbirth. Effective coverage was low: 2% based on the project data and less than 1% using the DHIS2. Linking project data with NDHS, it was feasible to measure all four steps; using DHIS2 it was possible to estimate three steps: no data was available to measure process quality. The provision of high quality care is suboptimal in this high mortality setting where access and facility readiness to provide care, crucial foundations to the provision of high quality of care, have not yet been met. This study demonstrates that partial effective coverage measures can be constructed from routine data combined with nationally representative surveys. Advocacy to include process of care indicators in facility summary reports could optimise this data source for decision making.

Methodological considerations for linking household and healthcare provider data for estimating effective coverage: a systematic review

OBJECTIVE

To assess existing knowledge related to methodological considerations for linking population-based surveys and health facility data to generate effective coverage estimates. Effective coverage estimates the proportion of individuals in need of an intervention who receive it with sufficient quality to achieve health benefit.

DESIGN

Systematic review of available literature.

DATA SOURCES

Medline, Carolina Population Health Center and Demographic and Health Survey publications and handsearch of related or referenced works of all articles included in full text review. The search included publications from 1 January 2000 to 29 March 2021.

ELIGIBILITY CRITERIA

Publications explicitly evaluating (1) the suitability of data, (2) the implications of the design of existing data sources and (3) the impact of choice of method for combining datasets to obtain linked coverage estimates.

RESULTS

Of 3805 papers reviewed, 70 publications addressed relevant issues. Limited data suggest household surveys can be used to identify sources of care, but their validity in estimating intervention need was variable. Methods for collecting provider data and constructing quality indices were diverse and presented limitations. There was little empirical data supporting an association between structural, process and outcome quality. Few studies addressed the influence of the design of common data sources on linking analyses, including imprecise household geographical information system data, provider sampling design and estimate stability. The most consistent evidence suggested under certain conditions, combining data based on geographical proximity or administrative catchment (ecological linking) produced similar estimates to linking based on the specific provider utilised (exact match linking).

CONCLUSIONS

Linking household and healthcare provider data can leverage existing data sources to generate more informative estimates of intervention coverage and care. However, existing evidence on methods for linking data for effective coverage estimation are variable and numerous methodological questions remain. There is need for additional research to develop evidence-based, standardised best practices for these analyses.

Impact of imprecise household location on effective coverage estimates generated through linking household and health provider data by geographic proximity: a simulation study

Background

Geographic proximity is often used to link household and health provider data to estimate effective coverage of health interventions. Existing household surveys often provide displaced data on the central point within household clusters rather than household location. This may introduce error into analyses based on the distance between households and providers.

Methods

We assessed the effect of imprecise household location on quality-adjusted effective coverage of child curative services estimated by linking sick children to providers based on geographic proximity. We used data on care-seeking for child illness and health provider quality in Southern Province, Zambia. The dataset included the location of respondent households, a census of providers, and data on the exact outlets utilized by sick children included in the study. We displaced the central point of each household cluster point five times. We calculated quality-adjusted coverage by assigning each sick child to a provider’s care based on three measures of geographic proximity (Euclidean distance, travel time, and geographic radius) from the household location, cluster point, and displaced cluster locations. We compared the estimates of quality-adjusted coverage to each other and estimates using each sick child’s true source of care. We performed sensitivity analyses with simulated preferential care-seeking from higher-quality providers and randomly generated provider quality scores.

Results

Fewer children were linked to their true source of care using cluster locations than household locations. Effective coverage estimates produced using undisplaced or displaced cluster points did not vary significantly from estimates produced using household location data or each sick child’s true source of care. However, the sensitivity analyses simulating greater variability in provider quality showed bias in effective coverage estimates produced with the geographic radius and travel time method using imprecise location data in some scenarios.

Conclusions

Use of undisplaced or displaced cluster location reduced the proportion of children that linked to their true source of care. In settings with minimal variability in quality within provider categories, the impact on effective coverage estimates is limited. However, use of imprecise household location and choice of geographic linking method can bias estimates in areas with high variability in provider quality or preferential care-seeking.

Supply-Side Barriers to the Use of Public Healthcare Facilities for Childhood Illness Care in Rural Zambia: A Cross-Sectional Study Linking Data from a Healthcare Facility Census to a Household Survey

Child mortality due to malaria and diarrhea can be reduced if proper treatment is received timely at healthcare facilities, but various factors hinder this. The present study assessed the associations between the use of public healthcare facilities among febrile/diarrheal children in rural Zambia and supply-side factors (i.e., the distance from the village to the nearest facility and the availability of essential human resources and medical equipment at the facility). Data from the Demographic and Health Survey 2018 and the Health Facility Census 2017 were linked. Generalized linear mixed models were used to assess the associations, controlling for clustering and other variables. The median distances to the nearest facility were 4.5 km among 854 febrile children and 4.6 km among 813 diarrheal children. Children who were over 10 km away from the facility were significantly less likely to use it, compared to those within 5 km (fever group: odds ratio (OR) = 0.36, 95% confidence interval (CI) = 0.20-0.66; diarrhea group: OR = 0.30, 95% CI = 0.18-0.51). The availability of human resources and equipment was, however, not significantly associated with facility use. Poor geographic access could be a critical barrier to facility use among children in rural Zambia.

Uterotonics for prevention of postpartum haemorrhage: EN-BIRTH multi-country validation study

BACKGROUND

Postpartum haemorrhage (PPH) is a leading cause of preventable maternal mortality worldwide. The World Health Organization (WHO) recommends uterotonic administration for every woman after birth to prevent PPH. There are no standardised data collected in large-scale measurement platforms. The Every Newborn Birth Indicators Research Tracking in Hospitals (EN-BIRTH) is an observational study to assess the validity of measurement of maternal and newborn indicators, and this paper reports findings regarding measurement of coverage and quality for uterotonics.

METHODS

The EN-BIRTH study took place in five hospitals in Bangladesh, Nepal and Tanzania, from July 2017 to July 2018. Clinical observers collected tablet-based, time-stamped data. We compared observation data for uterotonics to routine hospital register-records and women's report at exit-interview survey. We analysed the coverage and quality gap for timing and dose of administration. The register design was evaluated against gap analyses and qualitative interview data assessing the barriers and enablers to data recording and use.

RESULTS

Observed uterotonic coverage was high in all five hospitals (> 99%, 95% CI 98.7-99.8%). Survey-report underestimated coverage (79.5 to 91.7%). "Don't know" replies varied (2.1 to 14.4%) and were higher after caesarean (3.7 to 59.3%). Overall, there was low accuracy in survey data for details of uterotonic administration (type and timing). Register-recorded coverage varied in four hospitals capturing uterotonics in a specific column (21.6, 64.5, 97.6, 99.4%). The average coverage measurement gap was 18.1% for register-recorded and 6.0% for survey-reported coverage. Uterotonics were given to 15.9% of women within the "right time" (1 min) and 69.8% within 3 min. Women's report of knowing the purpose of uterotonics after birth ranged from 0.4 to 64.9% between hospitals. Enabling register design and adequate staffing were reported to improve routine recording.

CONCLUSIONS

Routine registers have potential to track uterotonic coverage - register data were highly accurate in two EN-BIRTH hospitals, compared to consistently underestimated coverage by survey-report. Although uterotonic coverage was high, there were gaps in observed quality for timing and dose. Standardisation of register design and implementation could improve data quality and data flow from registers into health management information reporting systems, and requires further assessment.

Linking household surveys and facility assessments: a comparison of geospatial methods using nationally representative data from Malawi

BACKGROUND

Linking facility and household surveys through geographic methods is a popular technique to draw conclusions about the relationship between health services and population health outcomes at local levels. These methods are useful tools for measuring effective coverage and tracking progress towards Universal Health Coverage, but are understudied. This paper compares the appropriateness of several geospatial methods used for linking individuals (within displaced survey cluster locations) to their source of family planning (at undisplaced health facilities) at a national level.

METHODS

In Malawi, geographic methods linked a population health survey, rural clusters from the Woman's Questionnaire of the 2015 Malawi Demographic and Health Survey (MDHS 2015), to Malawi's national health facility census to understand the service environment where women receive family planning services. Individuals from MDHS 2015 clusters were linked to health facilities through four geographic methods: (i) closest facility, (ii) buffer (5 km), (iii) administrative boundary, and (iv) a newly described theoretical catchment area method. Results were compared across metrics to assess the number of unlinked clusters (data lost), the number of linkages per cluster (precision of linkage), and the number of women linked to their last source of modern contraceptive (appropriateness of linkage).

RESULTS

The closest facility and administrative boundary methods linked every cluster to at least one facility, while the 5-km buffer method left 288 clusters (35.3%) unlinked. The theoretical catchment area method linked all but one cluster to at least one facility (99.9% linked). Closest facility, 5-km buffer, administrative boundary, and catchment methods linked clusters to 1.0, 1.4, 21.1, and 3.3 facilities on average, respectively. Overall, the closest facility, 5-km buffer, administrative boundary, and catchment methods appropriately linked 64.8%, 51.9%, 97.5%, and 88.9% of women to their last source of modern contraceptive, respectively.

CONCLUSIONS

Of the methods studied, the theoretical catchment area linking method loses a marginal amount of population data, links clusters to a relatively low number of facilities, and maintains a high level of appropriate linkages. This linking method is demonstrated at scale and can be used to link individuals to qualities of their service environments and better understand the pathways through which interventions impact health.

Operationalising health systems thinking: a pathway to high effective coverage

Background

The global health community has recognised the importance of defining and measuring the effective coverage of health interventions and their implementation strength to monitor progress towards global mortality and morbidity targets. Existing health system models and frameworks guide thinking around these measurement areas; however, they fall short of adequately capturing the dynamic and multi-level relationships between different components of the health system. These relationships must be articulated for measurement and managed to effectively deliver health interventions of sufficient quality to achieve health impacts. Save the Children’s Saving Newborn Lives programme and EnCompass LLC, its evaluation partner, developed and applied the Pathway to High Effective Coverage as a health systems thinking framework (hereafter referred to as the Pathway) in its strategic planning, monitoring and evaluation.

Methods

We used an iterative approach to develop, test and refine thinking around the Pathway. The initial framework was developed based on existing literature, then shared and vetted during consultations with global health thought leaders in maternal and newborn health.

Results

The Pathway is a robust health systems thinking framework that unpacks system, policy and point of intervention delivery factors, thus encouraging specific actions to address gaps in implementation and facilitate the achievement of high effective coverage. The Pathway includes six main components – (1) national readiness; (2) system structures; (3) management capacity; (4) implementation strength; (5) effective coverage; and (6) impact. Each component is comprised of specific elements reflecting the range of facility-, community- and home-based interventions. We describe applications of the Pathway and results for in-country strategic planning, monitoring of progress and implementation strength, and evaluation.

Conclusions

The Pathway provides a cohesive health systems thinking framework that facilitates assessment and coordinated action to achieve high coverage and impact. Experiences of its application show its utility in guiding strategic planning and in more comprehensive and effective monitoring and evaluation as well as its potential adaptability for use in other health areas and sectors.

Linking household survey and health facility data for effective coverage measures: a comparison of ecological and individual linking methods using the Multiple Indicator Cluster Survey in Côte d'Ivoire

Background

Population-based measures of intervention coverage are used in low- and middle-income countries for program planning, prioritization, and evaluation. There is increased interest in effective coverage, which integrates information about service quality or health outcomes. Approaches proposed for quality-adjusted effective coverage include linking data on need and service contact from population-based surveys with data on service quality from health facility surveys. However, there is limited evidence about the validity of different linking methods for effective coverage estimation.

Methods

We collaborated with the 2016 Côte d'Ivoire Multiple Indicator Cluster Survey (MICS) to link data from a health provider assessment to care-seeking data collected by the MICS in the Savanes region of Côte d'Ivoire. The provider assessment was conducted in a census of public and non-public health facilities and pharmacies in Savanes in May-June 2016. We also included community health workers managing sick children who served the clusters sampled for the MICS. The provider assessment collected information on structural and process quality for antenatal care, delivery and immediate newborn care, postnatal care, and sick child care. We linked the MICS and provider data using exact-match and ecological linking methods, including aggregate linking and geolinking methods. We compared the results obtained from exact-match and ecological methods.

Results

We linked 731 of 786 care-seeking episodes (93%) from the MICS to a structural quality score for the provider named by the respondent. Effective coverage estimates computed using exact-match methods were 13%-63% lower than the care-seeking estimates from the MICS. Absolute differences between exact match and ecological linking methods were ±7 percentage points for all ecological methods. Incorporating adjustments for provider category and weighting by service-specific utilization into the ecological methods generally resulted in better agreement between ecological and exact match estimates.

Conclusions

Ecological linking may be a feasible and valid approach for estimating quality-adjusted effective coverage when a census of providers is used. Adjusting for provider type and caseload may improve agreement with exact match results. There remain methodological questions to be addressed to develop guidance on using linking methods for estimating quality-adjusted effective coverage, including the effect of facility sampling and time displacement.