Evaluation of a national surveillance system for mortality alerts: a mixed-methods study

Evaluation of Timeliness, Simplicity, Acceptability, and Flexibility in Child Mortality Surveillance System for Children Aged 1-59 Months in Iran

Background:

Child mortality surveillance system (CMSS) for children aged 1–59 months is a critical issue in the prevention of mortality. This surveillance system like other health programs needs to be evaluated. Therefore, this study aims to evaluate CMSS in Iran.

Methods:

This evaluation was performed from March 2015 to March 2016 based on selected criteria for assessing the public health surveillance system proposed by the Centers for Disease Control and Prevention. Selected criteria examined in this study included timeliness, simplicity, acceptability, and flexibility. These criteria were evaluated in two ways. First, it included the use of a researcher-made questionnaire. The questionnaires were completed by 100 experts on CMSS. Second, to perform a more exact evaluation of these criteria, 24 of these experts were selected for the focus group.

Results:

In this study, the response rate was 91% (42% hospital-based and 49% primary care-based). In the timeliness section, 49% of the experts believed that approvals of the child mortality committees have not been sent within the designated time frame; hardware, software, and questionnaires were reported as effective factors in this respect. The structural and administrative problems were effective in simplicity domain and the experts of mortality registration and mood of relatives were effective in acceptability domain. The flexibility of the system was high and appropriate.

Conclusions:

The findings of the present study reveal that CMSS has some limitations and problems in the timeliness, simplicity, and acceptability criteria, which can be resolved. But this program has an appropriate situation in terms of flexibility.

What is the relationship between mortality alerts and other indicators of quality of care? A national cross-sectional study

Objectives

To assess whether mortality alerts, triggered by sustained higher than expected hospital mortality, are associated with other potential indicators of hospital quality relating to factors of hospital structure, clinical process and patient outcomes.

Methods

Cross-sectional study of National Health Service hospital trusts in England (2011–2013) using publicly available hospital measures reflecting organizational structure (mean acute bed occupancy, nurse/bed ratio, training satisfaction and proportion of trusts with low National Health Service Litigation Authority risk assessment or in financial deficit); process (mean proportion of eligible patients who receive percutaneous coronary intervention within 90 minutes) and outcomes (mean patient satisfaction scores, summary measures of hospital mortality and proportion of patients harmed). Mortality alerts were based on hospital administrative data.

Results

Mortality alerts were associated with structural indicators and outcome indicators of quality. There was insufficient data to detect an association between mortality alerts and the process indicator.

Conclusions

Mortality alerts appear to reflect aspects of quality within an English hospital setting, suggesting that there may be value in a mortality alerting system in highlighting poor hospital quality.

National hospital mortality surveillance system: a descriptive analysis

OBJECTIVE

To provide a description of the Imperial College Mortality Surveillance System and subsequent investigations by the Care Quality Commission (CQC) in National Health Service (NHS) hospitals receiving mortality alerts.

BACKGROUND

The mortality surveillance system has generated monthly mortality alerts since 2007, on 122 individual diagnosis and surgical procedure groups, using routinely collected hospital administrative data for all English acute NHS hospital trusts. The CQC, the English national regulator, is notified of each alert. This study describes the findings of CQC investigations of alerting trusts.

METHODS

We carried out (1) a descriptive analysis of alerts (2007-2016) and (2) an audit of CQC investigations in a subset of alerts (2011-2013).

RESULTS

Between April 2007 and October 2016, 860 alerts were generated and 76% (654 alerts) were sent to trusts. Alert volumes varied over time (range: 40-101). Septicaemia (except in labour) was the most commonly alerting group (11.5% alerts sent). We reviewed CQC communications in a subset of 204 alerts from 96 trusts. The CQC investigated 75% (154/204) of alerts. In 90% of these pursued alerts, trusts returned evidence of local case note reviews (140/154). These reviews found areas of care that could be improved in 69% (106/154) of alerts. In 25% (38/154) trusts considered that identified failings in care could have impacted on patient outcomes. The CQC investigations resulted in full trust action plans in 77% (118/154) of all pursued alerts.

CONCLUSION

The mortality surveillance system has generated a large number of alerts since 2007. Quality of care problems were found in 69% of alerts with CQC investigations, and one in four trusts reported that failings in care may have an impact on patient outcomes. Identifying whether mortality alerts are the most efficient means to highlight areas of substandard care will require further investigation.

Investigating the association of alerts from a national mortality surveillance system with subsequent hospital mortality in England: an interrupted time series analysis

Objective

To investigate the association between alerts from a national hospital mortality surveillance system and subsequent trends in relative risk of mortality.

Background

There is increasing interest in performance monitoring in the NHS. Since 2007, Imperial College London has generated monthly mortality alerts, based on statistical process control charts and using routinely collected hospital administrative data, for all English acute NHS hospital trusts. The impact of this system has not yet been studied.

Methods

We investigated alerts sent to Acute National Health Service hospital trusts in England in 2011–2013. We examined risk-adjusted mortality (relative risk) for all monitored diagnosis and procedure groups at a hospital trust level for 12 months prior to an alert and 23 months post alert. We used an interrupted time series design with a 9-month lag to estimate a trend prior to a mortality alert and the change in trend after, using generalised estimating equations.

Results

On average there was a 5% monthly increase in relative risk of mortality during the 12 months prior to an alert (95% CI 4% to 5%). Mortality risk fell, on average by 61% (95% CI 56% to 65%), during the 9-month period immediately following an alert, then levelled to a slow decline, reaching on average the level of expected mortality within 18 months of the alert.

Conclusions

Our results suggest an association between an alert notification and a reduction in the risk of mortality, although with less lag time than expected. It is difficult to determine any causal association. A proportion of alerts may be triggered by random variation alone and subsequent falls could simply reflect regression to the mean. Findings could also indicate that some hospitals are monitoring their own mortality statistics or other performance information, taking action prior to alert notification.