Validity and reliability of criterion based clinical audit to assess obstetrical quality of care in West Africa

Development of measures for assessing mistreatment of women during facility-based childbirth based on labour observations

INTRODUCTION

Mistreatment of women during childbirth is increasingly recognised as a significant issue globally. Research and programmatic efforts targeting this phenomenon have been limited by a lack of validated measurement tools. This study aimed to develop a set of concise, valid and reliable multidimensional measures for mistreatment using labour observations applicable across multiple settings.

METHODS

Data from continuous labour observations of 1974 women in Nigeria (n=407), Ghana (n=912) and Guinea (n=655) were used from the cross-sectional WHO's multicountry study 'How women are treated during facility-based childbirth' (2016-2018). Exploratory factor analysis was conducted to develop a scale measuring interpersonal abuse. Two indexes were developed through a modified Organisation for Economic Co-operation and Development approach for generating composite indexes. Measures were evaluated for performance, validity and internal reliability.

RESULTS

Three mistreatment measures were developed: a 7-item Interpersonal Abuse Scale, a 3-item Exams & Procedures Index and a 12-item Unsupportive Birth Environment Index. Factor analysis results showed a consistent unidimensional factor structure for the Interpersonal Abuse Scale in all three countries based on factor loadings and interitem correlations, indicating good structural construct validity. The scale had a reliability coefficient of 0.71 in Nigeria and approached 0.60 in Ghana and Guinea. Low correlations (Spearman correlation range: -0.06-0.19; p≥0.05) between mistreatment measures supported our decision to develop three separate measures. Predictive criterion validation yielded mixed results across countries. Both items within measures and measure scores were internally consistent across countries; each item co-occurred with other items in a measure, and scores consistently distinguished between 'high' and 'low' mistreatment levels.

CONCLUSION

The set of concise, comprehensive multidimensional measures of mistreatment can be used in future research and quality improvement initiatives targeting mistreatment to quantify burden, identify risk factors and determine its impact on health and well-being outcomes. Further validation and reliability testing of the measures in other contexts is needed.

The reliability of maternal audit instruments to assign cause of death in maternal deaths review process: a systematic review and meta-analysis

BACKGROUND

Maternal Death Reviews (MDR) can assist in formulating prevention strategies to reduce maternal mortality. To support MDR, an adequate MDR instrument is required to accurately identify the underlying causes of maternal deaths. We conducted a systematic review and meta-analysis to determine the reliability of maternal death instruments for conducting the MDR process.

METHOD

Three databases: PubMed, ProQuest and EBSCO were systematically searched to identify related research articles published between January 2004 and July 2019. The review and meta-analysis involved identification of measurement tools to conduct MDR in all or part of maternal audit. Eligibiliy and quality of studies were evaluated using the Modified Quality Appraisal of Diagnostic Reliability (QAREL) Checklist: Reliability Studies.

RESULTS

Overall, 242 articles were identified. Six articles examining the instrument used for MDR in 4 countries (4 articles on verbal autopsy (VA) and 2 articles on facility-based MDR) were included. None of studies identified reliability in evaluation instruments assessing maternal audit cycle as a comprehensive approach. The pooled kappa for the MDR instruments was 0.72 (95%CI:0.43-0.99; p < 0.001) with considerable heterogeneity (I² = 96.19%; p < 0.001). Subgroup analysis of MDR instruments showed pooled kappa in VA of 0.89 (95%CI:0.52-1.25) and facility-based MDR of 0.48 (95%CI:0.15-0.82). Meta-regression analysis tended to show the high heterogeneity was likely associated with sample sizes, regions, and year of publications.

CONCLUSIONS

The MDR instruments appear feasible. Variation of the instruments suggest the need for judicious selection of MDR instruments by considering the study population and assessment during the target periods.

Evaluating the effect of care around labor and delivery practices on early neonatal mortality in the Global Network's Maternal and Newborn Health Registry

BACKGROUND

Neonatal deaths in first 28-days of life represent 47% of all deaths under the age of five years globally and are a focus of the United Nation's (UN's) Sustainable Development Goals. Pregnant women are delivering in facilities but that does not indicate quality of care during delivery and the postpartum period. The World Health Organization's Essential Newborn Care (ENC) package reduces neonatal mortality, but lacks a simple and valid composite index that measures its effectiveness.

METHODS

Data on 5 intra-partum and 3 post-partum practices (indicators) recommended as part of ENC, routinely collected in NICHD's Global Network's (GN) Maternal Newborn Health Registry (MNHR) between 2010 and 2013, were included. We evaluated if all 8 practices (Care around Delivery - CAD), combined as an index was associated with reduced early neonatal mortality rates (days 0-6 of life).

RESULTS

A total of 150,848 live births were included in the analysis. The individual indicators varied across sites. All components were present in 19.9% births (range 0.4 to 31% across sites). Present indicators (8 components) were associated with reduced early neonatal mortality [adjusted RR (95% CI):0.81 (0.77, 0.85); p < 0.0001]. Despite an overall association between CAD and early neonatal mortality (RR < 1.0 for all early mortality): delivery by skilled birth attendant; presence of fetal heart and delayed bathing were associated with increased early neonatal mortality.

CONCLUSIONS

Present indicators (8 practices) of CAD were associated with a 19% reduction in the risk of neonatal death in the diverse health facilities where delivery occurred within the GN MNHR. These indicators could be monitored to identify facilities that need to improve compliance with ENC practices to reduce preventable neonatal deaths. Three of the 8 indicators were associated with increased neonatal mortality, due to baby being sick at birth. Although promising, this composite index needs refinement before use to monitor facility-based quality of care in association with early neonatal mortality. Trial registration The identifier of the Maternal Newborn Health Registry at ClinicalTrials.gov is NCT01073475.

Death audits and reviews for reducing maternal, perinatal and child mortality

BACKGROUND

The United Nations' Sustainable Development Goals (SDGs) include reducing the global maternal mortality rate to less than 70 per 100,000 live births and ending preventable deaths of newborns and children under five years of age, in every country, by 2030. Maternal and perinatal death audit and review is widely recommended as an intervention to reduce maternal and perinatal mortality, and to improve quality of care, and could be key to attaining the SDGs. However, there is uncertainty over the most cost-effective way of auditing and reviewing deaths: community-based audit (verbal and social autopsy), facility-based audits (significant event analysis (SEA)) or a combination of both (confidential enquiry).

OBJECTIVES

To assess the impact and cost-effectiveness of different types of death audits and reviews in reducing maternal, perinatal and child mortality.

SEARCH METHODS

We searched the following from inception to 16 January 2019: CENTRAL, Ovid MEDLINE, Embase OvidSP, and five other databases. We identified ongoing studies using ClinicalTrials.gov and the World Health Organization (WHO) International Clinical Trials Registry Platform, and searched reference lists of included articles.

SELECTION CRITERIA

Cluster-randomised trials, cluster non-randomised trials, controlled before-and-after studies and interrupted time series studies of any form of death audit or review that involved reviewing individual cases of maternal, perinatal or child deaths, identifying avoidable factors, and making recommendations. To be included in the review, a study needed to report at least one of the following outcomes: perinatal mortality rate; stillbirth rate; neonatal mortality rate; mortality rate in children under five years of age or maternal mortality rate.

DATA COLLECTION AND ANALYSIS

We used standard Cochrane Effective Practice and Organisation of Care (EPOC) group methodological procedures. Two review authors independently extracted data, assessed risk of bias and assessed the certainty of the evidence using GRADE. We planned to perform a meta-analysis using a random-effects model but included studies were not homogeneous enough to make pooling their results meaningful.

MAIN RESULTS

We included two cluster-randomised trials. Both introduced death review and audit as part of a multicomponent intervention, and compared this to current care. The QUARITE study (QUAlity of care, RIsk management, and TEchnology) concerned maternal death reviews in hospitals in West Africa, which had very high maternal and perinatal mortality rates. In contrast, the OPERA trial studied perinatal morbidity/mortality conferences (MMCs) in maternity units in France, which already had very low perinatal mortality rates at baseline. The OPERA intervention in France started with an outreach visit to brief obstetricians, midwives and anaesthetists on the national guidelines on morbidity/mortality case management, and was followed by a series of perinatal MMCs. Half of the intervention units were randomised to receive additional support from a clinical psychologist during these meetings. The OPERA intervention may make little or no difference to overall perinatal mortality (low certainty evidence), however we are uncertain about the effect of the intervention on perinatal mortality related to suboptimal care (very low certainty evidence).The intervention probably reduces perinatal morbidity related to suboptimal care (unadjusted odds ratio (OR) 0.62, 95% confidence interval (CI) 0.40 to 0.95; 165,353 births; moderate-certainty evidence). The effect of the intervention on stillbirth rate, neonatal mortality, mortality rate in children under five years of age, maternal mortality or adverse effects was not reported. The QUARITE intervention in West Africa focused on training leaders of hospital obstetric teams using the ALARM (Advances in Labour And Risk Management) course, which included one day of training about conducting maternal death reviews. The leaders returned to their hospitals, established a multidisciplinary committee and started auditing maternal deaths, with the support of external facilitators. The intervention probably reduces inpatient maternal deaths (adjusted OR 0.85, 95% CI 0.73 to 0.98; 191,167 deliveries; moderate certainty evidence) and probably also reduces inpatient neonatal mortality within 24 hours following birth (adjusted OR 0.74, 95% CI 0.61 to 0.90; moderate certainty evidence). However, QUARITE probably makes little or no difference to the inpatient stillbirth rate (moderate certainty evidence) and may make little or no difference to the inpatient neonatal mortality rate after 24 hours, although the 95% confidence interval includes both benefit and harm (low certainty evidence). The QUARITE intervention probably increases the percent of women receiving high quality of care (OR 1.87, 95% CI 1.35 - 2.57, moderate-certainty evidence). The effect of the intervention on perinatal mortality, mortality rate in children under five years of age, or adverse effects was not reported. We did not find any studies that evaluated child death audit and review or community-based death reviews or costs.

AUTHORS' CONCLUSIONS

A complex intervention including maternal death audit and review, as well as development of local leadership and training, probably reduces inpatient maternal mortality in low-income country district hospitals, and probably slightly improves quality of care. Perinatal death audit and review, as part of a complex intervention with training, probably improves quality of care, as measured by perinatal morbidity related to suboptimal care, in a high-income setting where mortality was already very low. The WHO recommends that maternal and perinatal death reviews should be conducted in all hospitals globally. However, conducting death reviews in isolation may not be sufficient to achieve the reductions in mortality observed in the QUARITE trial. This review suggests that maternal death audit and review may need to be implemented as part of an intervention package which also includes elements such as training of a leading doctor and midwife in each hospital, annual recertification, and quarterly outreach visits by external facilitators to provide supervision and mentorship. The same may also apply to perinatal and child death reviews. More operational research is needed on the most cost-effective ways of implementing maternal, perinatal and paediatric death reviews in low- and middle-income countries.

[Quality of caesarean delivery: A cross-sectional study in 12 hospitals in Benin]

BACKGROUND

Caesarean section (CS) is a major obstetric intervention, widely recognized as an effective means to reduce maternal and perinatal mortality, when appropriately performed. CS numbers and rates are regularly published but quality is rarely taken into account. This study aims to describe the quality of caesarean delivery in selected hospitals in Benin.

METHODS

A cross-sectional study was performed among women who had undergone a CS between 18 December 2013 and 8 February 2014 in one randomly selected hospital in each of the 12 administrative districts of Benin. The quality of CS was defined according to the analytical framework of Dujardin and Delvaux (1998) with its four pillars (access, diagnosis, procedure, postoperative care). Data were collected from hospital files and questionnaires from women and hospital directors. Data analysis was performed using Epi Info 3.5.1.

RESULTS

Six hundred and thirty-two women delivered by CS during the period and 579 were eligible for the study. They were aged 26.5±6.3 years, 73.2% living more than 5km from the hospital, 63.0% referred to a health facility of whom 46.0% and 21.8% were transported by motorcycle and by ambulance respectively. The median expenditure by family was FCFA 30 000, ranging from 0 to FCFA 200 000. The admission examination was complete in 12.6% of women and the partograph used in 32.6%. The average CS rate was 37.6%, the average response time, 124minutes. Emergency CS was performed in 80.7%, for absolute maternal indications in 48.0% and under spinal anesthesia in 84.2% (98.3% of which were conducted by a nurse or midwife anesthetist). Maternal mortality was 2000 maternal deaths per 100 000 deliveries, while perinatal mortality was 7.4% (88.4% due to stillbirths).

CONCLUSION

CS in Benin hospitals partially fulfilled quality criteria. However access to CS remains difficult and errors of diagnosis or excessive delay are too frequent. Quality CS is not yet a reality in Benin hospitals.

Management of abortion complications at a rural hospital in Uganda: a quality assessment by a partially completed criterion-based audit

Background

Complications of unsafe abortion are a major contributor to maternal deaths in developing countries. This study aimed to evaluate the clinical assessment for life-threatening complications and the following management in women admitted with complications from abortions at a rural hospital in Uganda.

Methods

A partially completed criterion-based audit was conducted comparing actual to optimal care. The audit criteria cover initial clinical assessment of vital signs and management of common severe complications such as sepsis and haemorrhage. Sepsis shall be managed by immediate evacuation of the uterus and antibiotics in relation to and after surgical management. Shock by aggressive rehydration followed by evacuation.

In total 238 women admitted between January 2007 and April 2012 were included. Complications were categorized as incomplete, threatened, inevitable, missed or septic abortion and by trimester. Actual management was compared to the audit criteria and presented by descriptive statistics.

Results

Fifty six per cent of the women were in second trimester. Abortion complications were distributed as follows: 53 % incomplete abortions, 28 % threatened abortions, 12 % inevitable abortions, 4 % missed abortions and 3 % septic abortions. Only one of 238 cases met all criteria of optimal clinical assessment and management. Thus, vital signs were measured in 3 %, antibiotic criteria was met in 59 % of the cases, intravenous fluid resuscitation was administered to 35 % of women with hypotension and pain was managed in 87 % of the cases. Sharp curettage was used in 69 % of those surgically evacuated and manual vacuum aspiration in 14 %. In total 3 % of the abortions were categorized as unsafe. Two of eight women with septic abortion had evacuation performed during admission-day, one woman died due to septic abortion and one from severe haemorrhage.

Conclusions

Guidelines were not followed and suboptimal assessment or management was observed in all but one case. This was especially due to missing documentation of vital signs necessary to diagnose life-threatening complications, poor fluid resuscitation at signs of shock, and delayed evacuation of septic abortion.

Criteria-based audit of quality of care to women with severe pre-eclampsia and eclampsia in a referral hospital in Accra, Ghana

OBJECTIVES

Severe pre-eclampsia and eclampsia are one of the major causes of maternal mortality globally. Reducing maternal morbidity and mortality demands optimizing quality of care. Criteria-based audits are a tool to define, assess and improve quality of care. The aim of this study was to determine applicability of a criteria-based audit to assess quality of care delivered to women with severe hypertensive disorders in pregnancy, and to assess adherence to protocols and quality of care provided at a regional hospital in Accra, Ghana.

METHODS

Checklists for management of severe preeclampsia, hypertensive emergency and eclampsia were developed in an audit cycle based on nine existing key clinical care protocols. Fifty cases were audited to assess quality of care, defined as adherence to protocols. Analysis was stratified for complicated cases, defined as (imminent) eclampsia, perinatal mortality and/or one or more WHO maternal near miss C-criteria.

RESULTS

Mean adherence to the nine protocols ranged from 15-85%. Protocols for 'plan for delivery' and 'magnesium sulphate administration' were best adhered to (85%), followed by adherence to protocols for 'eclampsia' (64%), 'severe pre-eclampsia at admission' (60%), 'severe pre-eclampsia ward follow-up' (53%) and 'hypertensive emergency' (53%). Protocols for monitoring were least adhered to (15%). No difference was observed for severe disease. Increased awareness, protocol-based training of staff, and clear task assignment were identified as contributors to better adherence.

CONCLUSION

A criteria-based audit is an effective tool to determine quality of care, identify gaps in standard of care, and allow for monitoring and evaluation in a health facility, ultimately resulting in improved quality of care provided and reduced maternal morbidity and mortality. In our audit, good adherence was observed for plan for delivery and treatment with magnesium sulphate. Substandard adherence to a number of protocols was identified, and points towards opportunities for targeted improvement strategies.

Training and nutritional components of PMTCT programmes associated with improved intrapartum quality of care in Mali and Senegal

OBJECTIVE

Scale-up of prevention of mother-to-child transmission (PMTCT) of HIV programmes in sub-Saharan Africa has stimulated interest to assess whether these programmes can indirectly affect other health priorities. This study assesses whether PMTCT programmes, or components of these programmes, are associated with better obstetrical quality of care and how PMTCT may reinforce existing maternal health programmes.

DESIGN

Cross-sectional analysis of data from a cluster-randomized trial called QUARITE.

SETTING

Mali and Senegal, West Africa.

PARTICIPANTS

Thirty-one referral hospitals and 612 obstetrical patients.

INTERVENTION

The exposure of interest was PMTCT measured with a scale containing 10 components describing different prongs of a hospital PMTCT programme. Other variables of interest included: presence of a quality of care improvement programme, hospital resources and patient demographic characteristics.

MAIN OUTCOME MEASURE

Obstetrical quality of care measured through a validated chart abstraction tool.

RESULTS

Of 45 points, the mean hospital PMTCT score was 26.1 (SD: 6.7). Total PMTCT score was not significantly associated with quality of care, but programme component scores were. After adjustment for known predictors of quality of care, staff training in PMTCT (P = 0.03) and complementary nutritional services (P = 0.03) were significantly associated with better quality obstetrical care. A point increase in scores for either of these components was associated with 40% greater odds of good obstetrical care.

CONCLUSIONS

PMTCT training and nutritional components are significantly associated with better quality intrapartum care. Health professionals' training in maternal healthcare and PMTCT could be combined to improve the quality of obstetric care in the region.

Assessment of maternal near-miss and quality of care in a hospital-based study in Accra, Ghana

OBJECTIVE

To assess the baseline incidence of maternal near-miss, process indicators related to facility access, and quality of care at a tertiary care facility in urban Ghana.

METHODS

A prospective observational study of all women delivering at the facility, including those with pregnancy-related complications, was conducted between October 2010 and March 2011. Quality of maternal health care was assessed via a newly developed WHO instrument based on near-miss criteria and criterion-based clinical audit methodology.

RESULTS

Among 3438 women, 516 had potentially life-threatening conditions and 131 had severe maternal outcomes (94 near-miss cases and 37 maternal deaths). More than half (64.4%) of the women had been referred to the facility. The incidence of maternal near-miss was 28.6 cases per 1000 live births. Anemia contributed to most cases with a severe maternal outcome. More than half of all women with severe maternal outcomes developed organ dysfunction or died within the first 12 hours of hospital admission. Although preventive measures were prevalent, treatment-related indicators showed mixed results.

CONCLUSION

The WHO near-miss approach was found to represent a feasible strategy in low-resource countries. Improving referral systems, effective use of critical care, and evidence-based interventions can potentially reduce severe maternal outcomes.

Effect of a facility-based multifaceted intervention on the quality of obstetrical care: a cluster randomized controlled trial in Mali and Senegal

Background

Maternal mortality in referral hospitals in Mali and Senegal surpasses 1% of obstetrical admissions. Poor quality obstetrical care contributes to high maternal mortality; however, poor care is often linked to insufficient hospital resources. One promising method to improve obstetrical care is maternal death review. With a cluster randomized trial, we assessed whether an intervention, based on maternal death review, could improve obstetrical quality of care.

Methods

The trial began with a pre-intervention year (2007), followed by two years of intervention activities and a post-intervention year. We measured obstetrical quality of care in the post-intervention year using a criterion-based clinical audit (CBCA). We collected data from 32 of the 46 trial hospitals (16 in each trial arm) and included 658 patients admitted to the maternity unit with a trial of labour. The CBCA questionnaire measured 5 dimensions of care- patient history, clinical examination, laboratory examination, delivery care and postpartum monitoring. We used adjusted mixed models to evaluate differences in CBCA scores by trial arms and examined how levels of hospital human and material resources affect quality of care differences associated with the intervention.

Results

For all women, the mean percentage of care criteria met was 66.3 (SD 13.5). There were significantly greater mean CBCA scores in women treated at intervention hospitals (68.2) compared to control hospitals (64.5). After adjustment, women treated at intervention sites had 5 points’ greater scores than those at control sites. This difference was mostly attributable to greater clinical examination and post-partum monitoring scores. The association between the intervention and quality of care was the same, irrespective of the level of resources available to a hospital; however, as resources increased, so did quality of care scores in both arms of the trial.

Trial registration

The QUARITE trial is registered on the Current Controlled Trials website under ISRCTN46950658