Large-scale empirical optimisation of statistical control charts to detect clinically relevant increases in surgical site infection rates

Mass mortality event in South American sea lions (Otaria flavescens) correlated to highly pathogenic avian influenza (HPAI) H5N1 outbreak in Chile

In Chile, since January 2023, a sudden and pronounced increase in strandings and mortality has been observed among South American (SA) sea lions (Otaria flavescens), prompting significant concern. Simultaneously, an outbreak of highly pathogenic avian influenza H5N1 (HPAIV H5N1) in avian species has emerged since December 2022. To investigate the cause of this unexpected mortality, we conducted a comprehensive epidemiological and pathologic study. One hundred sixty-nine SA sea lions were sampled to ascertain their HPAIV H5N1 status, and long-term stranding trends from 2009 to 2023 were analyzed. In addition, two animals were necropsied. Remarkably, a significant surge in SA sea lion strandings was observed initiating in January 2023 and peaking in June 2023, with a count of 4,545 stranded and deceased animals. Notably, this surge in mortality correlates geographically with HPAIV outbreaks affecting wild birds. Among 168 sampled SA sea lions, 34 (20%) tested positive for Influenza A virus, and 21 confirmed for HPAIV H5N1 2.3.4.4b clade in tracheal/rectal swab pools. Clinical and pathological evaluations of the two necropsied stranded sea lions revealed prevalent neurological and respiratory signs, including disorientation, tremors, ataxia, and paralysis, as well as acute dyspnea, tachypnea, profuse nasal secretion, and abdominal breathing. The lesions identified in necropsied animals aligned with observed clinical signs. Detection of the virus via immunohistochemistry (IHC) and real-time PCR in the brain and lungs affirmed the findings. The findings provide evidence between the mass mortality occurrences in SA sea lions and HPAIV, strongly indicating a causal relationship. Further studies are needed to better understand the pathogenesis and transmission.

Use of Statistical Process Control Methods for Early Detection of Healthcare Facility-Associated Nontuberculous Mycobacteria Outbreaks: A Single-Center Pilot Study

BACKGROUND

Nontuberculous mycobacteria (NTM) are emerging pathogens increasingly implicated in healthcare facility-associated (HCFA) infections and outbreaks. We analyzed the performance of statistical process control (SPC) methods in detecting HCFA NTM outbreaks.

METHODS

We retrospectively analyzed 3 NTM outbreaks that occurred from 2013 to 2016 at a tertiary care hospital. The outbreaks consisted of pulmonary Mycobacterium abscessus complex (MABC) acquisition, cardiac surgery-associated extrapulmonary MABC infection, and a bronchoscopy-associated pseudo-outbreak of Mycobacterium avium complex (MAC). We analyzed monthly case rates of unique patients who had positive respiratory cultures for MABC, non-respiratory cultures for MABC, and bronchoalveolar lavage cultures for MAC, respectively. For each outbreak, we used these rates to construct a pilot moving average (MA) SPC chart with a rolling baseline window. We also explored the performance of numerous alternative control charts, including exponentially weighted MA, Shewhart, and cumulative sum charts.

RESULTS

The pilot MA chart detected each outbreak within 2 months of outbreak onset, preceding actual outbreak detection by an average of 6 months. Over a combined 117 months of pre-outbreak and post-outbreak surveillance, no false-positive SPC signals occurred (specificity, 100%). Prospective use of this chart for NTM surveillance could have prevented an estimated 108 cases of NTM. Six high-performing alternative charts detected all outbreaks during the month of onset, with specificities ranging from 85.7% to 94.9%.

CONCLUSIONS

SPC methods have potential to substantially improve HCFA NTM surveillance, promoting early outbreak detection and prevention of NTM infections. Additional study is needed to determine the best application of SPC for prospective HCFA NTM surveillance in other settings.

Grand rounds in methodology: four critical decision points in statistical process control evaluations of quality improvement initiatives

Quality improvement (QI) projects often employ statistical process control (SPC) charts to monitor process or outcome measures as part of ongoing feedback, to inform successive Plan-Do-Study-Act cycles and refine the intervention (formative evaluation). SPC charts can also be used to draw inferences on effectiveness and generalisability of improvement efforts (summative evaluation), but only if appropriately designed and meeting specific methodological requirements for generalisability. Inadequate design decreases the validity of results, which not only reduces the chance of publication but could also result in patient harm and wasted resources if incorrect conclusions are drawn. This paper aims to bring together much of what has been written in various tutorials, to suggest a process for using SPC in QI projects. We highlight four critical decision points that are often missed, how these are inter-related and how they affect the inferences that can be drawn regarding effectiveness of the intervention: (1) the need for a stable baseline to enable drawing inferences on effectiveness; (2) choice of outcome measures to assess effectiveness, safety and intervention fidelity; (3) design features to improve the quality of QI projects; (4) choice of SPC analysis aligned with the type of outcome, and reporting on the potential influence of other interventions or secular trends.These decision points should be explicitly reported for readers to interpret and judge the results, and can be seen as supplementing the Standards for Quality Improvement Reporting Excellence guidelines. Thinking in advance about both formative and summative evaluation will inform more deliberate choices and strengthen the evidence produced by QI projects.

Early recognition and response to increases in surgical site infections using optimised statistical process control charts-The early 2RIS trial: A multicentre stepped wedge cluster randomised controlled trial

BACKGROUND

Traditional approaches for surgical site infection (SSI) surveillance have deficiencies that delay detection of SSI outbreaks and other clinically important increases in SSI rates. We investigated whether use of optimised statistical process control (SPC) methods and feedback for SSI surveillance would decrease rates of SSI in a network of US community hospitals.

METHODS

We conducted a stepped wedge cluster randomised trial of patients who underwent any of 13 types of common surgical procedures across 29 community hospitals in the Southeastern United States. We divided the 13 procedures into six clusters; a cluster of procedures at a single hospital was the unit of randomisation and analysis. In total, 105 clusters were randomised to 12 groups of 8-10 clusters. All participating clusters began the trial in a 12-month baseline period of control or "traditional" SSI surveillance, including prospective analysis of SSI rates and consultative support for SSI outbreaks and investigations. Thereafter, a group of clusters transitioned from control to intervention surveillance every three months until all clusters received the intervention. Electronic randomisation by the study statistician determined the sequence by which clusters crossed over from control to intervention surveillance. The intervention was the addition of weekly application of optimised SPC methods and feedback to existing traditional SSI surveillance methods. Epidemiologists were blinded to hospital identity and randomisation status while adjudicating SPC signals of increased SSI rates, but blinding was not possible during SSI investigations. The primary outcome was the overall SSI prevalence rate (PR=SSIs/100 procedures), evaluated via generalised estimating equations with a Poisson regression model. Secondary outcomes compared traditional and optimised SPC signals that identified SSI rate increases, including the number of formal SSI investigations generated and deficiencies identified in best practices for SSI prevention. This trial was registered at ClinicalTrials.gov, NCT03075813.

FINDINGS

Between Mar 1, 2016, and Feb 29, 2020, 204,233 unique patients underwent 237,704 surgical procedures. 148,365 procedures received traditional SSI surveillance and feedback alone, and 89,339 procedures additionally received the intervention of optimised SPC surveillance. The primary outcome of SSI was assessed for all procedures performed within participating clusters. SSIs occurred after 1171 procedures assigned control surveillance (prevalence rate [PR] 0.79 per 100 procedures), compared to 781 procedures that received the intervention (PR 0·87 per 100 procedures; model-based PR ratio 1.10, 95% CI 0.94-1.30, p=0.25). Traditional surveillance generated 24 formal SSI investigations that identified 120 SSIs with deficiencies in two or more perioperative best practices for SSI prevention. In comparison, optimised SPC surveillance generated 74 formal investigations that identified 458 SSIs with multiple best practice deficiencies.

INTERPRETATION

The addition of optimised SPC methods and feedback to traditional methods for SSI surveillance led to greater detection of important SSI rate increases and best practice deficiencies but did not decrease SSI rates. Additional research is needed to determine how to best utilise SPC methods and feedback to improve adherence to SSI quality measures and prevent SSIs.

FUNDING

Agency for Healthcare Research and Quality.

A Pragmatic Method for Measuring Inpatient Complications and Complication-Specific Mortality

OBJECTIVES

The primary objective of this study was to develop hospital-level metrics of major complications associated with mortality that allows for the identification of opportunities for improvement. The secondary objective is to improve upon current metrics for failure to rescue (i.e., death from serious but treatable complications.).

METHODS

Agency for Healthcare Research and Quality metrics served as the basis for identifying specific complications related to major organ system morbidity associated with death. Complication-specific occurrence rates, observed mortality, and risk-adjusted mortality indices were calculated for the study institution and 182 peer organizations using component International Classification of Disease, Tenth Revision codes. Data were included for adults over a 4-year period, with exclusion of hospice patients and complications present on admission. Temporal visualizations of each metric were used to compare past and recent performance at the study hospital and in comparison to peers.

RESULTS

The complication-specific method showed statistically significant differences in the study hospital occurrence rates and associated mortality rates compared with peer institutions. The monthly control-chart presentation of these metrics provides assessment of hospital-level interventions to prevent complications and/or reduce failure to rescue deaths.

CONCLUSIONS

The method described supplements existing metrics of serious complications that occur during the course of acute hospitalization allowing for enhanced visualization of opportunities to improve care delivery systems. This method leverages existing measure components to minimize reporting burden. Monthly time-series data allow interventions to prevent and/or rescue patients to be rapidly assessed for impact.

A Multidisciplinary Quality Improvement Initiative to Facilitate Penicillin Allergy Delabeling Among Hospitalized Pediatric Patients

BACKGROUND

Penicillin allergy is reported in up to 10% of the general population; however, >90% of patients reporting an allergy are tolerant. Patients labeled as penicillin allergic have longer hospital stays, increased exposure to suboptimal antibiotics, and an increased risk of methicillin-resistant Staphylococcus aureus and Clostridioides difficile. The primary aim with our quality improvement initiative was to increase penicillin allergy delabeling to at least 10% among all hospitalized pediatric patients reporting a penicillin allergy with efforts directed toward patients determined to be low risk for true allergic reaction.

METHODS

Our quality improvement project included several interventions: the development of a multidisciplinary clinical care pathway to identify eligible patients, workflow optimization to support delabeling, an educational intervention, and participation in our institution's quality improvement incentive program. Our interventions were targeted to facilitate appropriate delabeling by the primary hospital medicine team. Statistical process control charts were used to assess the impact of this intervention pre- and postpathway implementation.

RESULTS

After implementation of the clinical pathway, the percentage of patients admitted to hospital medicine delabeled of their penicillin allergy by discharge increased to 11.7%. More than one-half of those delabeled (51.2%) received a penicillin-based antimicrobial at time of discharge. There have been no adverse events or allergic reactions requiring emergency medication administration since pathway implementation.

CONCLUSIONS

Our quality improvement initiative successfully increased the rate of penicillin allergy delabeling among low-risk hospitalized pediatric patients, allowing for increased use of optimal antibiotics.

Early recognition and response to increases in surgical site infections using optimized statistical process control charts-the Early 2RIS Trial: a multicenter cluster randomized controlled trial with stepped wedge design

Background

Surgical site infections (SSIs) cause significant patient suffering. Surveillance and feedback of SSI rates is an evidence-based strategy to reduce SSIs, but traditional surveillance methods are slow and prone to bias. The objective of this cluster randomized controlled trial (RCT) is to determine if using optimized statistical process control (SPC) charts for SSI surveillance and feedback lead to a reduction in SSI rates compared to traditional surveillance.

Methods

The Early 2RIS Trial is a prospective, multicenter cluster RCT using a stepped wedge design. The trial will be performed in 29 hospitals in the Duke Infection Control Outreach Network (DICON) and 105 clusters over 4 years, from March 2016 through February 2020; year one represents a baseline period; thereafter, 8–9 clusters will be randomized to intervention every 3 months over a 3-year period using a stepped wedge randomization design. All patients who undergo one of 13 targeted procedures at study hospitals will be included in the analysis; these procedures will be included in one of six clusters: cardiac, orthopedic, gastrointestinal, OB-GYN, vascular, and spinal. All clusters will undergo traditional surveillance for SSIs; once randomized to intervention, clusters will also undergo surveillance and feedback using optimized SPC charts. Feedback on surveillance data will be provided to all clusters, regardless of allocation or type of surveillance. The primary endpoint is the difference in rates of SSI between the SPC intervention compared to traditional surveillance and feedback alone.

Discussion

The traditional approach for SSI surveillance and feedback has several major deficiencies because SSIs are rare events. First, traditional statistical methods require aggregation of measurements over time, which delays analysis until enough data accumulate. Second, traditional statistical tests and resulting p values are difficult to interpret. Third, analyses based on average SSI rates during predefined time periods have limited ability to rapidly identify important, real-time trends. Thus, standard analytic methods that compare average SSI rates between arbitrarily designated time intervals may not identify an important SSI rate increase on time unless the “signal” is very strong. Therefore, novel strategies for early identification and investigation of SSI rate increases are needed to decrease SSI rates. While SPC charts are used throughout industry and healthcare to improve and optimize processes, including other types of healthcare-associated infections, they have not been evaluated as a tool for SSI surveillance and feedback in a randomized trial.

Trial registration

ClinicalTrials.govNCT03075813, Registered March 9, 2017.