Quantitative and qualitative verification of data quality in the childbirth registers of two rural district hospitals in Western Kenya

Travelling numbers and broken loops: A qualitative systematic review on collecting and reporting maternal and neonatal health data in low-and lower-middle income countries

Data and indicator estimates are considered vital to document persisting challenges in maternal and newborn health and track progress towards global goals. However, prioritization of standardised, comparable quantitative data can preclude the collection of locally relevant information and pose overwhelming burdens in low-resource settings, with negative effects on the provision of quality of care. A growing body of qualitative studies aims to provide a place-based understanding of the complex processes and human experiences behind the generation and use of maternal and neonatal health data. We conducted a qualitative systematic review exploring how national or international requirements to collect and report data on maternal and neonatal health indicators are perceived and experienced at the sub-national and country level in low-income and lower-middle income countries. We systematically searched six electronic databases for qualitative and mixed-methods studies published between January 2000 and March 2023. Following screening of 4084 records by four reviewers, 47 publications were included in the review. Data were analysed thematically and synthesised from a Complex Adaptive Systems (CAS) theoretical perspective. Our findings show maternal and neonatal health data and indicators are not fixed, neutral entities, but rather outcomes of complex processes. Their collection and uptake is influenced by a multitude of system hardware elements (human resources, relevancy and adequacy of tools, infrastructure, and interoperability) and software elements (incentive systems, supervision and feedback, power and social relations, and accountability). When these components are aligned and sufficiently supportive, data and indicators can be used for positive system adaptivity through performance evaluation, prioritization, learning, and advocacy. Yet shortcomings and broken loops between system components can lead to unforeseen emergent behaviors such as blame, fear, and data manipulation. This review highlights the importance of measurement approaches that prioritize local relevance and feasibility, necessitating participatory approaches to define context-specific measurement objectives and strategies.

Introducing a standardised register for strengthening the inpatient management of newborns and sick children: Implementation research in selected health facilities of Bangladesh

Background

It is imperative to maintain accurate documentation of clinical interventions aimed at enhancing the quality of care for newborns and sick children. The National Newborn Health and IMCI programme of Bangladesh led the development of a standardised register for managing newborns and sick children under five years of age during inpatient care through stakeholder engagement. We aimed to assess the implementation outcomes of the standardised register in the inpatient department.

Methods

We conducted implementation research in two district hospitals and two sub-district hospitals of Kushtia and Dinajpur districts from November 2022 to January 2023 to assess the implementation outcomes of the standardised register. We assessed the following World Health Organization implementation outcome variables: usability, acceptability, adoption (actual use), fidelity (completeness and accuracy), and utility (quality of care) of the register against preset benchmarks. We collected data through structured interviews with health care providers; participant enrolment; and data extraction from inpatient registers and case record forms.

Results

The average usability and acceptability scores among health care providers were 73 (standard deviation (SD) = 14) and 82 (SD = 14) out of 100, respectively. The inpatient register recorded 96% (95% confidence interval (CI) = 95-97) of under-five children who were admitted to the inpatient department (adoption - actual use). The proportions of completed data elements in the inpatient register were above the preset benchmark of 70% for all the assessed data elements except 'investigation done' (24%; 95% CI = 23-26) (fidelity - completeness). The percentage agreements between government-appointed nurses posted and study-appointed nurses were above the preset benchmark of 70% for all the reported variables (fidelity - accuracy). The kappa coefficient for the overall level of agreement between these two groups regarding reported variables indicated moderate to substantial agreement. The proportion of newborns with sepsis receiving injectable antibiotics was 62% (95% CI = 47-75) (utility - quality of care). We observed some variability in the completeness and accuracy of the inpatient register by district and facility type.

Conclusions

The inpatient register was positively received by health care providers, with evaluations of implementation outcome variables showing encouraging results. Our findings could inform evidence-based decision-making on the implementation and scale-up of the inpatient register in Bangladesh, as well as other low- and middle-income countries.

Improving birth weight measurement and recording practices in Kenya and Tanzania: a prospective intervention study with historical controls

BACKGROUND

Low birth weight (LBW) is a significant public health concern given its association with early-life mortality and other adverse health consequences that can impact the entire life cycle. In many countries, accurate estimates of LBW prevalence are lacking due to inaccuracies in collection and gaps in available data. Our study aimed to determine LBW prevalence among facility-born infants in selected areas of Kenya and Tanzania and to assess whether the introduction of an intervention to improve the accuracy of birth weight measurement would result in a meaningfully different estimate of LBW prevalence than current practice.

METHODS

We carried out a historically controlled intervention study in 22 health facilities in Kenya and three health facilities in Tanzania. The intervention included: provision of high-quality digital scales, training of nursing staff on accurate birth weight measurement, recording and scale calibration practices, and quality maintenance support that consisted of enhanced supervision and feedback (prospective arm). The historically controlled data were birth weights from the same facilities recorded in maternity registers for the same calendar months from the previous year measured using routine practices and manual scales. We calculated mean birth weight (95% confidence interval CI), mean difference in LBW prevalence, and respective risk ratio (95% CI) between study arms.

RESULTS

Between October 2019 and February 2020, we prospectively collected birth weights from 8441 newborns in Kenya and 4294 in Tanzania. Historical data were available from 9318 newborns in Kenya and 12,007 in Tanzania. In the prospective sample, the prevalence of LBW was 12.6% (95% confidence intervals [CI]: 10.9%-14.4%) in Kenya and 18.2% (12.2%-24.2%) in Tanzania. In the historical sample, the corresponding prevalence estimates were 7.8% (6.5%-9.2%) and 10.0% (8.6%-11.4%). Compared to the retrospective sample, the LBW prevalence in the prospective sample was 4.8% points (3.2%-6.4%) higher in Kenya and 8.2% points (2.3%-14.0%) higher in Tanzania, corresponding to a risk ratio of 1.61 (1.38-1.88) in Kenya and 1.81 (1.30-2.52) in Tanzania.

CONCLUSION

Routine birth weight records underestimate the risk of LBW among facility-born infants in Kenya and Tanzania. The quality of birth weight data can be improved by a simple intervention consisting of provision of digital scales and supportive training.

Quality of routine health facility data used for newborn indicators in low- and middle-income countries: A systematic review

Background

High-quality data are fundamental for effective monitoring of newborn morbidity and mortality, particularly in high burden low- and middle-income countries (LMIC).

Methods

We conducted a systematic review on the quality of routine health facility data used for newborn indicators in LMIC, including measures employed. Five databases were searched from inception to February 2021 for relevant observational studies (excluding case-control studies, case series, and case reports) and baseline or control group data from interventional studies, with no language limits. An adapted version (19-point scale) of the Critical Appraisal Tool to assess the Quality of Cross-Sectional Studies (AXIS) was used to assess methodological quality, and results were synthesized using descriptive analysis.

Results

From the 19 572 records retrieved, 34 studies in 16 LMIC countries were included. Methodological quality was high (>14/19) in 32 studies and moderate (10-14/19) in two. Studies were mostly from African (n = 30, 88.2%) and South-East Asian (n = 24, 70.6%) World Health Organization (WHO) regions, with very few from Eastern Mediterranean (n = 2, 5.9%) and Western Pacific (n = 1, 2.9%) ones. We found that only data elements used to calculate neonatal indicators had been assessed, not the indicators themselves. 41 data elements were assessed, most frequently birth outcome. 20 measures of data quality were used, most along three dimensions: 1) completeness and timeliness, 2) internal consistency, and 3) external consistency. Data completeness was very heterogeneous across 26 studies, ranging from 0%-100% in routine facility registers, 0%-100% in patient case notes, and 20%-68% in aggregate reports. One study reported on the timeliness of aggregate reports. Internal consistency ranged from 0% to 96.2% in four studies. External consistency (21 studies) varied widely in measurement and findings, with specificity (6.4%-100%), sensitivity (23.6%-97.6%), and percent agreement (24.6%-99.4%) most frequently reported.

Conclusions

This systematic review highlights a gap in the published literature on the quality of routine LMIC health facility data for newborn indicators. Robust evidence is crucial in driving data quality initiatives at national and international levels. The findings of this review indicate that good quality data collection is achievable even in high-burden LMIC settings, but more efforts are needed to ensure uniformly high data quality for neonatal indicators.

Health management information system (HMIS) data quality and associated factors in Massaguet district, Chad

Background

Quality data from Health Management Information Systems (HMIS) are important for tracking the effectiveness of malaria control interventions. However, HMIS data in many resource-limited settings do not currently meet standards set by the World Health Organization (WHO). We aimed to assess HMIS data quality and associated factors in Chad.

Methods

A cross-sectional study was conducted in 14 health facilities in Massaguet district. Data on children under 15 years were obtained from the HMIS and from the external patient register covering the period January–December 2018. An additional questionnaire was administered to 16 health centre managers to collect data on contextual variables. Patient registry data were aggregated and compared with the HMIS database at district and health centre level. Completeness and accuracy indicators were calculated as per WHO guidelines. Multivariate logistic regressions were performed on the Verification Factor for attendance, suspected and confirmed malaria cases for three age groups (1 to < 12 months, 1 to < 5 years and 5 to < 15 years) to identify associations between health centre characteristics and data accuracy.

Results

Health centres achieved a high level of data completeness in HMIS. Malaria data were over-reported in HMIS for children aged under 15 years. There was an association between workload and higher odds of inaccuracy in reporting of attendance among children aged 1 to < 5 years (Odds ratio [OR]: 10.57, 95% CI 2.32–48.19) and 5– < 15 years (OR: 6.64, 95% CI 1.38–32.04). Similar association was found between workload and stock-outs in register books, and inaccuracy in reporting of malaria confirmed cases. Meanwhile, we found that presence of a health technician, and of dedicated staff for data management, were associated with lower inaccuracy in reporting of clinic attendance in children aged under five years.

Conclusion

Data completeness was high while the accuracy was low. Factors associated with data inaccuracy included high workload and the unavailability of required data collection tools. The results suggest that improvement in working conditions for clinic personnel may improve HMIS data quality. Upgrading from paper-based forms to a web-based HMIS may provide a solution for improving data accuracy and its utility for future evaluations of health interventions. Results from this study can inform the Ministry of Health and it partners on the precautions to be taken in the use of HMIS data and inform initiatives for improving its quality.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01684-7.

Birth and death notification via mobile devices: a mixed methods systematic review

BACKGROUND

Ministries of health, donors, and other decision-makers are exploring how they can use mobile technologies to acquire accurate and timely statistics on births and deaths. These stakeholders have called for evidence-based guidance on this topic. This review was carried out to support World Health Organization (WHO) recommendations on digital interventions for health system strengthening.

OBJECTIVES

Primary objective: To assess the effects of birth notification and death notification via a mobile device, compared to standard practice. Secondary objectives: To describe the range of strategies used to implement birth and death notification via mobile devices and identify factors influencing the implementation of birth and death notification via mobile devices.

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, the Global Health Library, and POPLINE (August 2, 2019). We searched two trial registries (August 2, 2019). We also searched Epistemonikos for related systematic reviews and potentially eligible primary studies (August 27, 2019). We conducted a grey literature search using mHealthevidence.org (August 15, 2017) and issued a call for papers through popular digital health communities of practice. Finally, we conducted citation searches of included studies in Web of Science and Google Scholar (May 15, 2020). We searched for studies published after 2000 in any language.  SELECTION CRITERIA: For the primary objective, we included individual and cluster-randomised trials; cross-over and stepped-wedge study designs; controlled before-after studies, provided they have at least two intervention sites and two control sites; and interrupted time series studies. For the secondary objectives, we included any study design, either quantitative, qualitative, or descriptive, that aimed to describe current strategies for birth and death notification via mobile devices; or to explore factors that influence the implementation of these strategies, including studies of acceptability or feasibility. For the primary objective, we included studies that compared birth and death notification via mobile devices with standard practice. For the secondary objectives, we included studies of birth and death notification via mobile device as long as we could extract data relevant to our secondary objectives. We included studies of all cadres of healthcare providers, including lay health workers; administrative, managerial, and supervisory staff; focal individuals at the village or community level; children whose births were being notified and their parents/caregivers; and individuals whose deaths were being notified and their relatives/caregivers.

DATA COLLECTION AND ANALYSIS

For the primary objective, two authors independently screened all records, extracted data from the included studies and assessed risk of bias. For the analyses of the primary objective, we reported means and proportions, where appropriate. We used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to assess the certainty of the evidence and we prepared a 'Summary of Findings' table. For the secondary objectives, two authors screened all records, one author extracted data from the included studies and assessed methodological limitations using the WEIRD tool and a second author checked the data and assessments. We carried out a framework analysis using the Supporting the Use of Research Evidence (SURE) framework to identify themes in the data. We used the GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach to assess our confidence in the evidence and we prepared a 'Summary of Qualitative Findings' table.

MAIN RESULTS

For the primary objective, we included one study, which used a controlled before-after study design. The study was conducted in Lao People's Democratic Republic and assessed the effect of using mobile devices for birth notification on outcomes related to coverage and timeliness of Hepatitis B vaccination. However, we are uncertain of the effect of this approach on these outcomes because the certainty of this evidence was assessed as very low. The included study did not assess resource use or unintended consequences. For the primary objective, we did not identify any studies using mobile devices for death notification. For the secondary objective, we included 21 studies. All studies were conducted in low- or middle-income settings. They focussed on identification of births and deaths in rural, remote, or marginalised populations who are typically under-represented in civil registration processes or traditionally seen as having poor access to health services. The review identified several factors that could influence the implementation of birth-death notification via mobile device. These factors were tied to the health system, the person responsible for notifying, the community and families; and include: - Geographic barriers that could prevent people's access to birth-death notification and post-notification services - Access to health workers and other notifiers with enough training, supervision, support, and incentives - Monitoring systems that ensure the quality and timeliness of the birth and death data - Legal frameworks that allow births and deaths to be notified by mobile device and by different types of notifiers - Community awareness of the need to register births and deaths - Socio-cultural norms around birth and death - Government commitment - Cost to the system, to health workers and to families - Access to electricity and network connectivity, and compatibility with existing systems - Systems that protect data confidentiality We have low to moderate confidence in these findings. This was mainly because of concerns about methodological limitations and data adequacy.

AUTHORS' CONCLUSIONS

We need more, well-designed studies of the effect of birth and death notification via mobile devices and on factors that may influence its implementation.

Assessment of the validity of the measurement of newborn and maternal health-care coverage in hospitals (EN-BIRTH): an observational study

BACKGROUND

Progress in reducing maternal and neonatal deaths and stillbirths is impeded by data gaps, especially regarding coverage and quality of care in hospitals. We aimed to assess the validity of indicators of maternal and newborn health-care coverage around the time of birth in survey data and routine facility register data.

METHODS

Every Newborn-BIRTH Indicators Research Tracking in Hospitals was an observational study in five hospitals in Bangladesh, Nepal, and Tanzania. We included women and their newborn babies who consented on admission to hospital. Exclusion critiera at admission were no fetal heartbeat heard or imminent birth. For coverage of uterotonics to prevent post-partum haemorrhage, early initiation of breastfeeding (within 1 h), neonatal bag-mask ventilation, kangaroo mother care (KMC), and antibiotics for clinically defined neonatal infection (sepsis, pneumonia, or meningitis), we collected time-stamped, direct observation or case note verification data as gold standard. We compared data reported via hospital exit surveys and via hospital registers to the gold standard, pooled using random effects meta-analysis. We calculated population-level validity ratios (measured coverage to observed coverage) plus individual-level validity metrics.

FINDINGS

We observed 23 471 births and 840 mother-baby KMC pairs, and verified the case notes of 1015 admitted newborn babies regarding antibiotic treatment. Exit-survey-reported coverage for KMC was 99·9% (95% CI 98·3-100) compared with observed coverage of 100% (99·9-100), but exit surveys underestimated coverage for uterotonics (84·7% [79·1-89·5]) vs 99·4% [98·7-99·8] observed), bag-mask ventilation (0·8% [0·4-1·4]) vs 4·4% [1·9-8·1]), and antibiotics for neonatal infection (74·7% [55·3-90·1] vs 96·4% [94·0-98·6] observed). Early breastfeeding coverage was overestimated in exit surveys (53·2% [39·4-66·8) vs 10·9% [3·8-21·0] observed). "Don't know" responses concerning clinical interventions were more common in the exit survey after caesarean birth. Register data underestimated coverage of uterotonics (77·9% [37·8-99·5] vs 99·2% [98·6-99·7] observed), bag-mask ventilation (4·3% [2·1-7·3] vs 5·1% [2·0-9·6] observed), KMC (92·9% [84·2-98·5] vs 100% [99·9-100] observed), and overestimated early breastfeeding (85·9% (58·1-99·6) vs 12·5% [4·6-23·6] observed). Inter-hospital heterogeneity was higher for register-recorded coverage than for exit survey report. Even with the same register design, accuracy varied between hospitals.

INTERPRETATION

Coverage indicators for newborn and maternal health care in exit surveys had low accuracy for specific clinical interventions, except for self-report of KMC, which had high sensitivity after admission to a KMC ward or corner and could be considered for further assessment. Hospital register design and completion are less standardised than surveys, resulting in variable data quality, with good validity for the best performing sites. Because approximately 80% of births worldwide take place in facilities, standardising register design and information systems has the potential to sustainably improve the quality of data on care at birth.

FUNDING

Children's Investment Fund Foundation and Swedish Research Council.

Improving quality and use of routine health information system data in low- and middle-income countries: A scoping review

BACKGROUND

A routine health information system is one of the essential components of a health system. Interventions to improve routine health information system data quality and use for decision-making in low- and middle-income countries differ in design, methods, and scope. There have been limited efforts to synthesise the knowledge across the currently available intervention studies. Thus, this scoping review synthesised published results from interventions that aimed at improving data quality and use in routine health information systems in low- and middle-income countries.

METHOD

We included articles on intervention studies that aimed to improve data quality and use within routine health information systems in low- and middle-income countries, published in English from January 2008 to February 2020. We searched the literature in the databases Medline/PubMed, Web of Science, Embase, and Global Health. After a meticulous screening, we identified 20 articles on data quality and 16 on data use. We prepared and presented the results as a narrative.

RESULTS

Most of the studies were from Sub-Saharan Africa and designed as case studies. Interventions enhancing the quality of data targeted health facilities and staff within districts, and district health managers for improved data use. Combinations of technology enhancement along with capacity building activities, and data quality assessment and feedback system were found useful in improving data quality. Interventions facilitating data availability combined with technology enhancement increased the use of data for planning.

CONCLUSION

The studies in this scoping review showed that a combination of interventions, addressing both behavioural and technical factors, improved data quality and use. Interventions addressing organisational factors were non-existent, but these factors were reported to pose challenges to the implementation and performance of reported interventions.

A, Salim N, Zaman SB, Shamba D, Blencowe H, Ruysen H, El Arifeen S, Boggs D, Gordeev VS, Rahman QSU, Hossain T, Joshi E, Thapa S, Poudel RP, Poudel D, Chaudhary P, Karki R, Chitrakar B, Mkopi N, Wisiko A, Kitende AP, Shirati MR, Chingalo C, Semhando AO, Mtei C, Mwenisongole V, Bakuza JM, Kombo J, Mbaruku G, Lawn JE. Labour and delivery ward register data availability, quality, and utility - Every Newborn - birth indicators research tracking in hospitals (EN-BIRTH) study baseline analysis in three countries

BACKGROUND

Countries with the highest burden of maternal and newborn deaths and stillbirths often have little information on these deaths. Since over 81% of births worldwide now occur in facilities, using routine facility data could reduce this data gap. We assessed the availability, quality, and utility of routine labour and delivery ward register data in five hospitals in Bangladesh, Nepal, and Tanzania. This paper forms the baseline register assessment for the Every Newborn-Birth Indicators Research Tracking in Hospitals (EN-BIRTH) study.

METHODS

We extracted 21 data elements from routine hospital labour ward registers, useful to calculate selected maternal and newborn health (MNH) indicators. The study sites were five public hospitals during a one-year period (2016-17). We measured 1) availability: completeness of data elements by register design, 2) data quality: implausibility, internal consistency, and heaping of birthweight and explored 3) utility by calculating selected MNH indicators using the available data.

RESULTS

Data were extracted for 20,075 births. Register design was different between the five hospitals with 10-17 of the 21 selected MNH data elements available. More data were available for health outcomes than interventions. Nearly all available data elements were > 95% complete in four of the five hospitals and implausible values were rare. Data elements captured in specific columns were 85.2% highly complete compared to 25.0% captured in non-specific columns. Birthweight data were less complete for stillbirths than live births at two hospitals, and significant heaping was found in all sites, especially at 2500g and 3000g. All five hospitals recorded count data required to calculate impact indicators including; stillbirth rate, low birthweight rate, Caesarean section rate, and mortality rates.

CONCLUSIONS

Data needed to calculate MNH indicators are mostly available and highly complete in EN-BIRTH study hospital routine labour ward registers in Bangladesh, Nepal and Tanzania. Register designs need to include interventions for coverage measurement. There is potential to improve data quality if Health Management Information Systems utilization with feedback loops can be strengthened. Routine health facility data could contribute to reduce the coverage and impact data gap around the time of birth.

Producing routine malaria data: an exploration of the micro-practices and processes shaping routine malaria data quality in frontline health facilities in Kenya

BACKGROUND

Routine health information systems can provide near real-time data for malaria programme management, monitoring and evaluation, and surveillance. There are widespread concerns about the quality of the malaria data generated through routine information systems in many low-income countries. However, there has been little careful examination of micro-level practices of data collection which are central to the production of routine malaria data.

METHODS

Drawing on fieldwork conducted in two malaria endemic sub-counties in Kenya, this study examined the processes and practices that shape routine malaria data generation at frontline health facilities. The study employed ethnographic methods-including observations, records review, and interviews-over 18-months in four frontline health facilities and two sub-county health records offices. Data were analysed using a thematic analysis approach.

RESULTS

Malaria data generation was influenced by a range of factors including human resource shortages, tool design, and stock-out of data collection tools. Most of the challenges encountered by health workers in routine malaria data generation had their roots in wider system issues and at the national level where the framing of indicators and development of data collection tools takes place. In response to these challenges, health workers adopted various coping mechanisms such as informal task shifting and use of improvised tools. While these initiatives sustained the data collection process, they also had considerable implications for the data recorded and led to discrepancies in data that were recorded in primary registers. These discrepancies were concealed in aggregated monthly reports that were subsequently entered into the District Health Information Software 2.

CONCLUSION

Challenges to routine malaria data generation at frontline health facilities are not malaria or health information systems specific; they reflect wider health system weaknesses. Any interventions seeking to improve routine malaria data generation must look beyond just malaria or health information system initiatives and include consideration of the broader contextual factors that shape malaria data generation.

The burden of and factors associated with misclassification of intrapartum stillbirth: Evidence from a large scale multicentric observational study

INTRODUCTION

Global estimates suggest 2.6 million stillbirths and 2.5 million neonatal deaths occur annually worldwide. The majority of these deaths occur in low resource settings where analysis of health metrics and outcomes measurements may be challenging. We examined the misclassification of documented intrapartum stillbirth and factors associated with misclassification.

MATERIAL AND METHODS

A prospective observational study was performed in 12 public hospitals in Nepal. Data were extracted from the medical records of all births that occurred during the 6-month period of the study. For the study purpose, we classified birth outcome based on the presence of fetal heart sound (FHS) at admission and use of neonatal resuscitation. The health worker-documented intrapartum stillbirths were considered potentially misclassified when there were FHS present at admission and no resuscitation initiated after birth. The association between potentially misclassified intrapartum stillbirth and complications during labor, birthweight and gestational age was assessed using Pearson's chi-square test, bivariate and multivariate logistic regression.

RESULTS

A total of 39 562 mother-infant dyads were enrolled in the study, all of whom had FHS at admission. Among the 391 intrapartum stillbirths recorded during the study, 180 (46.0%) of them had FHS at admission with no resuscitation initiated after birth and were considered potentially misclassified intrapartum stillbirths. Among these potentially misclassified intrapartum stillbirths, 170 (43.5%) had FHS present 15 minutes before birth and 10 had no FHS 15 minutes before birth Among the potentially misclassified intrapartum stillbirths, 23.3% had complications during labor, 93.3% had birthweight less than 2500 g and 90.0% were born preterm. The risk of intrapartum misclassification was nearly four times higher among low birthweight babies (adjusted odds ratio [aOR] 3.5, 95% confidence interval [CI] 1.8 to 7.0, P < 0.001) and five times higher among preterm babies (aOR 5.3, 95% CI 3.0 to 9.3, P < 0.001).

CONCLUSIONS

We estimate that 46% of intrapartum stillbirths were potentially misclassified intrapartum stillbirths. Improving quality of both FHS monitoring and neonatal resuscitation as well as measurement of the care will reduce the risk of potentially misclassified intrapartum stillbirth and consequently intrapartum stillbirth.

Initializing a hospital-wide data quality program. The AP-HP experience

BACKGROUND AND OBJECTIVES

Data Quality (DQ) programs are recognized as a critical aspect of new-generation research platforms using electronic health record (EHR) data for building Learning Healthcare Systems. The AP-HP Clinical Data Repository aggregates EHR data from 37 hospitals to enable large-scale research and secondary data analysis. This paper describes the DQ program currently in place at AP-HP and the lessons learned from two DQ campaigns initiated in 2017.

MATERIALS AND METHODS

As part of the AP-HP DQ program, two domains - patient identification (PI) and healthcare services (HS) - were selected for conducting DQ campaigns consisting of 5 phases: defining the scope, measuring, analyzing, improving and controlling DQ. Semi-automated DQ profiling was conducted in two data sets - the PI data set containing 8.8 M patients and the HS data set containing 13,099 consultation agendas and 2122 care units. Seventeen DQ measures were defined and DQ issues were classified using a unified DQ reporting framework. For each domain, actions plans were defined for improving and monitoring prioritized DQ issues.

RESULTS

Eleven identified DQ issues (8 for the PI data set and 3 for the HS data set) were categorized into completeness (n = 6), conformance (n = 3) and plausibility (n = 2) DQ issues. DQ issues were caused by errors from data originators, ETL issues or limitations of the EHR data entry tool. The action plans included sixteen actions (9 for the PI domain and 7 for the HS domain). Though only partial implementation, the DQ campaigns already resulted in significant improvement of DQ measures.

CONCLUSION

DQ assessments of hospital information systems are largely unpublished. The preliminary results of two DQ campaigns conducted at AP-HP illustrate the benefit of the engagement into a DQ program. The adoption of a unified DQ reporting framework enables the communication of DQ findings in a well-defined manner with a shared vocabulary. Dedicated tooling is needed to automate and extend the scope of the generic DQ program. Specific DQ checks will be additionally defined on a per-study basis to evaluate whether EHR data fits for specific uses.

"Every Newborn-BIRTH" protocol: observational study validating indicators for coverage and quality of maternal and newborn health care in Bangladesh, Nepal and Tanzania

Background

To achieve Sustainable Development Goals and Universal Health Coverage, programmatic data are essential. The Every Newborn Action Plan, agreed by all United Nations member states and >80 development partners, includes an ambitious Measurement Improvement Roadmap. Quality of care at birth is prioritised by both Every Newborn and Ending Preventable Maternal Mortality strategies, hence metrics need to advance from health service contact alone, to content of care. As facility births increase, monitoring using routine facility data in DHIS2 has potential, yet validation research has mainly focussed on maternal recall surveys. The Every Newborn – Birth Indicators Research Tracking in Hospitals (EN-BIRTH) study aims to validate selected newborn and maternal indicators for routine tracking of coverage and quality of facility-based care for use at district, national and global levels.

Methods

EN-BIRTH is an observational study including >20 000 facility births in three countries (Tanzania, Bangladesh and Nepal) to validate selected indicators. Direct clinical observation will be compared with facility register data and a pre-discharge maternal recall survey for indicators including: uterotonic administration, immediate newborn care, neonatal resuscitation and Kangaroo mother care. Indicators including neonatal infection management and antenatal corticosteroid administration, which cannot be easily observed, will be validated using inpatient records. Trained clinical observers in Labour/Delivery ward, Operation theatre, and Kangaroo mother care ward/areas will collect data using a tablet-based customised data capturing application. Sensitivity will be calculated for numerators of all indicators and specificity for those numerators with adequate information. Other objectives include comparison of denominator options (ie, true target population or surrogates) and quality of care analyses, especially regarding intervention timing. Barriers and enablers to routine recording and data usage will be assessed by data flow assessments, quantitative and qualitative analyses.

Conclusions

To our knowledge, this is the first large, multi-country study validating facility-based routine data compared to direct observation for maternal and newborn care, designed to provide evidence to inform selection of a core list of indicators recommended for inclusion in national DHIS2. Availability and use of such data are fundamental to drive progress towards ending the annual 5.5 million preventable stillbirths, maternal and newborn deaths.

Assessing primary care data quality

Purpose The purpose of this paper is to assess National Medical Care Survey data quality. Design/methodology/approach Data completeness and representativeness were computed for all observations while other data quality measures were assessed using a 10 per cent sample from the National Medical Care Survey database; i.e., 12,569 primary care records from 189 public and private practices were included in the analysis. Findings Data field completion ranged from 69 to 100 per cent. Error rates for data transfer from paper to web-based application varied between 0.5 and 6.1 per cent. Error rates arising from diagnosis and clinical process coding were higher than medication coding. Data fields that involved free text entry were more prone to errors than those involving selection from menus. The authors found that completeness, accuracy, coding reliability and representativeness were generally good, while data timeliness needs to be improved. Research limitations/implications Only data entered into a web-based application were examined. Data omissions and errors in the original questionnaires were not covered. Practical implications Results from this study provided informative and practicable approaches to improve primary health care data completeness and accuracy especially in developing nations where resources are limited. Originality/value Primary care data quality studies in developing nations are limited. Understanding errors and missing data enables researchers and health service administrators to prevent quality-related problems in primary care data.

"Every day they keep adding new tools but they don't take any away": Producing indicators for intermittent preventive treatment for malaria in pregnancy (IPTp) from routine data in Kenya

BACKGROUND

Intermittent preventive treatment for malaria in pregnancy (IPTp) is part of a multi-pronged strategy aimed at preventing malaria in pregnancy in areas of moderate to high transmission in sub-Saharan Africa. Despite being formally adopted as a malaria prevention policy over a decade ago, IPTp coverage has remained low. Recent demands for action have incorporated calls to strengthen IPTp monitoring and evaluation systems, including the use of routine data, to measure coverage, track implementation and identify roadblocks to improving uptake. Concerns about the quality of malaria indicators reported through routine information systems are well recognized, but there are few data on the realities of IPTp recording practices in frontline facilities or their entry into District Health Information Software (DHIS2).

METHODS

Drawing on fieldwork conducted in two malaria endemic sub-counties in Kenya, we explore how local adaptations and innovations employed by health workers and sub-country managers to cope with a range of health system constraints, shape recording practices and in turn, the measurement of IPTp. Data were collected through observations, interviews, and document reviews. Data analysis and interpretation was guided by thematic analysis approach.

RESULTS

Measurement of IPTp was undermined by health system constraints such as stock-out of drugs and human resource shortages. Coping strategies adopted by health workers to address these challenges ensured continuity in service delivery and IPTp data generation but had variable consequences on IPTp data quality. Unclear recording and reporting instructions also led to lack of standardization in IPTp data generation. The use of redundant tools created significant data burdens which undermined service delivery in general.

CONCLUSIONS

There is need to integrate monthly reporting forms so as to remove redundancies which exacerbates workload for health workers and disrupts service delivery. Similarly, data collection instructions in registers and reporting forms need to be clarified to standardize IPTp data generation across health facilities. There is also need to address broader contextual factors such as stock-out of commodities and human resource shortages which undermine IPTp data generation process.

Barriers to data quality resulting from the process of coding health information to administrative data: a qualitative study

BACKGROUND

Administrative health data are increasingly used for research and surveillance to inform decision-making because of its large sample sizes, geographic coverage, comprehensivity, and possibility for longitudinal follow-up. Within Canadian provinces, individuals are assigned unique personal health numbers that allow for linkage of administrative health records in that jurisdiction. It is therefore necessary to ensure that these data are of high quality, and that chart information is accurately coded to meet this end. Our objective is to explore the potential barriers that exist for high quality data coding through qualitative inquiry into the roles and responsibilities of medical chart coders.

METHODS

We conducted semi-structured interviews with 28 medical chart coders from Alberta, Canada. We used thematic analysis and open-coded each transcript to understand the process of administrative health data generation and identify barriers to its quality.

RESULTS

The process of generating administrative health data is highly complex and involves a diverse workforce. As such, there are multiple points in this process that introduce challenges for high quality data. For coders, the main barriers to data quality occurred around chart documentation, variability in the interpretation of chart information, and high quota expectations.

CONCLUSIONS

This study illustrates the complex nature of barriers to high quality coding, in the context of administrative data generation. The findings from this study may be of use to data users, researchers, and decision-makers who wish to better understand the limitations of their data or pursue interventions to improve data quality.

Health Promotion Through Existing Community Structures: A Case of Churches' Roles in Promoting Rotavirus Vaccination in Rural Zambia

INTRODUCTION

Rural populations, particularly in Africa, suffer worse health outcomes from poor health services access. Community health workers (CHWs) effectively improve health outcomes, but the best means for CHWs reaching rural populations is unknown. Since Zambia is predominantly Christian, this study explored the use of CHWs through churches as an existing community structure for promoting preventive health behaviors, specifically rotavirus vaccine uptake.

METHODS

A noncontrolled cross-sectional study of 32 churches receiving a packaged intervention of diarrhea prevention and treatment messaging was conducted with repeated time points of data collection over 13 months (2013-2014) in the Kafue District of Zambia. Two churches were selected for each of the 17 catchment areas, and CHWs were identified and trained in the intervention of promoting 4 key messages related to diarrhea prevention and treatment: hand washing with soap, exclusive breast-feeding, rotavirus vaccination, and treating diarrhea with oral rehydration solution and zinc. The intervention was conducted within existing church's women's groups, and data was collected on attendance and the distribution of Rota Cards for tracking rotavirus immunizations.

RESULTS

Nineteen (59%) churches completed the study, and CHWs delivered health messages at a total of 890 women's group meetings. The overall reach of the intervention was to 37.0% of church-attending women, and the efficacy was 67.7% (317 of 468 Rota Cards collected at health centers).

DISCUSSION

Implementing community health programs is often expensive and unsustainable, but the reach and efficacy levels achieved through existing structures like churches are encouraging in resource-constrained countries. Churches can be effective channels for delivering health prevention strategies to often difficult-to-reach rural populations. Further research is needed to investigate the impact of the intervention on health outcomes.

What's Past is Prologue: A Scoping Review of Recent Public Health and Global Health Informatics Literature

OBJECTIVE

To categorize and describe the public health informatics (PHI) and global health informatics (GHI) literature between 2012 and 2014.

METHODS

We conducted a semi-systematic review of articles published between January 2012 and September 2014 where information and communications technologies (ICT) was a primary subject of the study or a main component of the study methodology. Additional inclusion and exclusion criteria were used to filter PHI and GHI articles from the larger biomedical informatics domain. Articles were identified using MEDLINE as well as personal bibliographies from members of the American Medical Informatics Association PHI and GHI working groups.

RESULTS

A total of 85 PHI articles and 282 GHI articles were identified. While systems in PHI continue to support surveillance activities, we identified a shift towards support for prevention, environmental health, and public health care services. Furthermore, articles from the U.S. reveal a shift towards PHI applications at state and local levels. GHI articles focused on telemedicine, mHealth and eHealth applications. The development of adequate infrastructure to support ICT remains a challenge, although we identified a small but growing set of articles that measure the impact of ICT on clinical outcomes.

DISCUSSION

There is evidence of growth with respect to both implementation of information systems within the public health enterprise as well as a widening of scope within each informatics discipline. Yet the articles also illuminate the need for more primary research studies on what works and what does not as both searches yielded small numbers of primary, empirical articles.

CONCLUSION

While the body of knowledge around PHI and GHI continues to mature, additional studies of higher quality are needed to generate the robust evidence base needed to support continued investment in ICT by governmental health agencies.

Development and Validation of an Index to Measure the Quality of Facility-Based Labor and Delivery Care Processes in Sub-Saharan Africa

Background

High quality care is crucial in ensuring that women and newborns receive interventions that may prevent and treat birth-related complications. As facility deliveries increase in developing countries, there are concerns about service quality. Observation is the gold standard for clinical quality assessment, but existing observation-based measures of obstetric quality of care are lengthy and difficult to administer. There is a lack of consensus on quality indicators for routine intrapartum and immediate postpartum care, including essential newborn care. This study identified key dimensions of the quality of the process of intrapartum and immediate postpartum care (QoPIIPC) in facility deliveries and developed a quality assessment measure representing these dimensions.

Methods and Findings

Global maternal and neonatal care experts identified key dimensions of QoPIIPC through a modified Delphi process. Experts also rated indicators of these dimensions from a comprehensive delivery observation checklist used in quality surveys in sub-Saharan African countries. Potential QoPIIPC indices were developed from combinations of highly-rated indicators. Face, content, and criterion validation of these indices was conducted using data from observations of 1,145 deliveries in Kenya, Madagascar, and Tanzania (including Zanzibar). A best-performing index was selected, composed of 20 indicators of intrapartum/immediate postpartum care, including essential newborn care. This index represented most dimensions of QoPIIPC and effectively discriminated between poorly and well-performed deliveries.

Conclusions

As facility deliveries increase and the global community pays greater attention to the role of care quality in achieving further maternal and newborn mortality reduction, the QoPIIPC index may be a valuable measure. This index complements and addresses gaps in currently used quality assessment tools. Further evaluation of index usability and reliability is needed. The availability of a streamlined, comprehensive, and validated index may enable ongoing and efficient observation-based assessment of care quality during labor and delivery in sub-Saharan Africa, facilitating targeted quality improvement.

Data quality monitoring and performance metrics of a prospective, population-based observational study of maternal and newborn health in low resource settings

Background

To describe quantitative data quality monitoring and performance metrics adopted by the Global Network’s (GN) Maternal Newborn Health Registry (MNHR), a maternal and perinatal population-based registry (MPPBR) based in low and middle income countries (LMICs).

Methods

Ongoing prospective, population-based data on all pregnancy outcomes within defined geographical locations participating in the GN have been collected since 2008. Data quality metrics were defined and are implemented at the cluster, site and the central level to ensure data quality. Quantitative performance metrics are described for data collected between 2010 and 2013.

Results

Delivery outcome rates over 95% illustrate that all sites are successful in following patients from pregnancy through delivery. Examples of specific performance metric reports illustrate how both the metrics and reporting process are used to identify cluster-level and site-level quality issues and illustrate how those metrics track over time. Other summary reports (e.g. the increasing proportion of measured birth weight compared to estimated and missing birth weight) illustrate how a site has improved quality over time.

Conclusion

High quality MPPBRs such as the MNHR provide key information on pregnancy outcomes to local and international health officials where civil registration systems are lacking. The MNHR has measures in place to monitor data collection procedures and improve the quality of data collected. Sites have increasingly achieved acceptable values of performance metrics over time, indicating improvements in data quality, but the quality control program must continue to evolve to optimize the use of the MNHR to assess the impact of community interventions in research protocols in pregnancy and perinatal health.

Trial registration number

NCT01073475

Indicators for Universal Health Coverage: can Kenya comply with the proposed post-2015 monitoring recommendations?

INTRODUCTION

Universal Health Coverage (UHC), referring to access to healthcare without financial burden, has received renewed attention in global health spheres. UHC is a potential goal in the post-2015 development agenda. Monitoring of progress towards achieving UHC is thus critical at both country and global level, and a monitoring framework for UHC was proposed by a joint WHO/World Bank discussion paper in December 2013. The aim of this study was to determine the feasibility of the framework proposed by WHO/World Bank for global UHC monitoring framework in Kenya.

METHODS

The study utilised three documents--the joint WHO/World Bank UHC monitoring framework and its update, and the Bellagio meeting report sponsored by WHO and the Rockefeller Foundation--to conduct the research. These documents informed the list of potential indicators that were used to determine the feasibility of the framework. A purposive literature search was undertaken to identify key government policy documents and relevant scholarly articles. A desk review of the literature was undertaken to answer the research objectives of this study.

RESULTS

Kenya has yet to establish an official policy on UHC that provides a clear mandate on the goals, targets and monitoring and evaluation of performance. However, a significant majority of Kenyans continue to have limited access to health services as well as limited financial risk protection. The country has the capacity to reasonably report on five out of the seven proposed UHC indicators. However, there was very limited capacity to report on the two service coverage indicators for the chronic condition and injuries (CCIs) interventions. Out of the potential tracer indicators (n = 27) for aggregate CCI-related measures, four tracer indicators were available. Moreover the country experiences some wider challenges that may impact on the implementation and feasibility of the WHO/World Bank framework.

CONCLUSION

The proposed global framework for monitoring UHC will only be feasible in Kenya if systemic challenges are addressed. While the infrastructure for reporting the MDG related indicators is in place, Kenya will require continued international investment to extend its capacity to meet the data requirements of the proposed UHC monitoring framework, particularly for the CCI-related indicators.

A review of data quality assessment methods for public health information systems

High quality data and effective data quality assessment are required for accurately evaluating the impact of public health interventions and measuring public health outcomes. Data, data use, and data collection process, as the three dimensions of data quality, all need to be assessed for overall data quality assessment. We reviewed current data quality assessment methods. The relevant study was identified in major databases and well-known institutional websites. We found the dimension of data was most frequently assessed. Completeness, accuracy, and timeliness were the three most-used attributes among a total of 49 attributes of data quality. The major quantitative assessment methods were descriptive surveys and data audits, whereas the common qualitative assessment methods were interview and documentation review. The limitations of the reviewed studies included inattentiveness to data use and data collection process, inconsistency in the definition of attributes of data quality, failure to address data users’ concerns and a lack of systematic procedures in data quality assessment. This review study is limited by the coverage of the databases and the breadth of public health information systems. Further research could develop consistent data quality definitions and attributes. More research efforts should be given to assess the quality of data use and the quality of data collection process.

Where is information quality lost at clinical level? A mixed-method study on information systems and data quality in three urban Kenyan ANC clinics

BACKGROUND

Well-working health information systems are considered vital with the quality of health data ranked of highest importance for decision making at patient care and policy levels. In particular, health facilities play an important role, since they are not only the entry point for the national health information system but also use health data (and primarily) for patient care.

DESIGN

A multiple case study was carried out between March and August 2012 at the antenatal care (ANC) clinics of two private and one public Kenyan hospital to describe clinical information systems and assess the quality of information. The following methods were developed and employed in an iterative process: workplace walkthroughs, structured and in-depth interviews with staff members, and a quantitative assessment of data quality (completeness and accurate transmission of clinical information and reports in ANC). Views of staff and management on the quality of employed information systems, data quality, and influencing factors were captured qualitatively.

RESULTS

Staff rated the quality of information higher in the private hospitals employing computers than in the public hospital which relies on paper forms. Several potential threats to data quality were reported. Limitations in data quality were common at all study sites including wrong test results, missing registers, and inconsistencies in reports. Feedback was seldom on content or quality of reports and usage of data beyond individual patient care was low.

CONCLUSIONS

We argue that the limited data quality has to be seen in the broader perspective of the information systems in which it is produced and used. The combination of different methods has proven to be useful for this. To improve the effectiveness and capabilities of these systems, combined measures are needed which include technical and organizational aspects (e.g. regular feedback to health workers) and individual skills and motivation.