Comparison of control charts for monitoring clinical performance using binary data

Evaluating the Impact of Uncertainty Visualization on Model Reliance

Machine learning models have gained traction as decision support tools for tasks that require processing copious amounts of data. However, to achieve the primary benefits of automating this part of decision-making, people must be able to trust the machine learning model's outputs. In order to enhance people's trust and promote appropriate reliance on the model, visualization techniques such as interactive model steering, performance analysis, model comparison, and uncertainty visualization have been proposed. In this study, we tested the effects of two uncertainty visualization techniques in a college admissions forecasting task, under two task difficulty levels, using Amazon's Mechanical Turk platform. Results show that (1) people's reliance on the model depends on the task difficulty and level of machine uncertainty and (2) ordinal forms of expressing model uncertainty are more likely to calibrate model usage behavior. These outcomes emphasize that reliance on decision support tools can depend on the cognitive accessibility of the visualization technique and perceptions of model performance and task difficulty.

Risk adjusted EWMA control chart based on support vector machine with application to cardiac surgery data

In the current study, we demonstrate the use of a quality framework to review the process for improving the quality and safety of the patient in the health care department. The researchers paid attention to assessing the performance of the health care service, where the data is usually heterogeneous to patient's health conditions. In our study, the support vector machine (SVM) regression model is used to handle the challenge of adjusting the risk factors attached to the patients. Further, the design of exponentially weighted moving average (EWMA) control charts is proposed based on the residuals obtained through SVM regression model. Analyzing real cardiac surgery patient data, we employed the SVM method to gauge patient condition. The resulting SVM-EWMA chart, fashioned via SVM modeling, revealed superior shift detection capabilities and demonstrated enhanced efficacy compared to the risk-adjusted EWMA control chart.

Strengthening global health security through health early warning systems: A literature review and case study

Disease transmission is dependent on a variety of factors, including the characteristics of an event, such as crowding and shared accommodations, the potential of participants having prolonged exposure and close contact with infectious individuals, the type of activities, and the characteristics of the participants, such as their age and immunity to infectious agents [1-3]. Effective control of outbreaks of infectious diseases requires rapid diagnosis and intervention in high-risk settings. As a result, syndromic and event-based surveillance may be used to enhance the responsiveness of the surveillance system [1]. In public health, surveillance is collecting, analyzing, and interpreting data across time to inform decision-making and aid policy implementation [1]. In this review article we aimed to provide an overview of the principles, types, uses, advantages, and limitations of surveillance systems and to highlight the importance of early warning systems in response to the information received by disease surveillance. The study conducted a comprehensive literature search using several databases, selecting, and reviewing 78 articles that covered different types of surveillance systems, their applications, and their impact on controlling infectious diseases. The article also presents a case study from the Hajj gathering, which highlighted the development, evaluation, and impact of early warning systems on response to the information received by disease surveillance. The study concludes that ongoing disease surveillance should be accompanied by well-designed early warning and response systems, and continuous efforts should be invested in evaluating and validating these systems to minimize the risk of reporting delays and reducing the risk of outbreaks.

Challenging Cases in Statistical Process Control for Quality Improvement in Neonatal Intensive Care

Effective quality improvement (QI) depends on rigorous analysis of time-series data through methods such as statistical process control (SPC). As use of SPC has become more prevalent in health care, QI practitioners must also be aware of situations that warrant special attention and potential modifications to common SPC charts, which include skewed continuous data, autocorrelation, small persistent changes in performance, confounders, and workload or productivity measures. This article reviews these situations and provides examples of SPC approaches for each.

Multivariate semiparametric control charts for mixed-type data

A useful tool that has gained popularity in the Quality Control area is the control chart which monitors a process over time, identifies potential changes, understands variations, and eventually improves the quality and performance of the process. This article introduces a new class of multivariate semiparametric control charts for monitoring multivariate mixed-type data, which comprise both continuous and discrete random variables (rvs). Our methodology leverages ideas from clustering and Statistical Process Control to develop control charts for MIxed-type data. We propose four control chart schemes based on modified versions of the KAy-means for MIxed LArge KAMILA data clustering algorithm, where we assume that the two existing clusters represent the reference and the test sample. The charts are semiparametric, the continuous rvs follow a distribution that belongs in the class of elliptical distributions. Categorical scale rvs follow a multinomial distribution. We present the algorithmic procedures and study the characteristics of the new control charts. The performance of the proposed schemes is evaluated on the basis of the False Alarm Rate and in-control Average Run Length. Finally, we demonstrate the effectiveness and applicability of our proposed methods utilizing real-world data.

Grand rounds in methodology: four critical decision points in statistical process control evaluations of quality improvement initiatives

Quality improvement (QI) projects often employ statistical process control (SPC) charts to monitor process or outcome measures as part of ongoing feedback, to inform successive Plan-Do-Study-Act cycles and refine the intervention (formative evaluation). SPC charts can also be used to draw inferences on effectiveness and generalisability of improvement efforts (summative evaluation), but only if appropriately designed and meeting specific methodological requirements for generalisability. Inadequate design decreases the validity of results, which not only reduces the chance of publication but could also result in patient harm and wasted resources if incorrect conclusions are drawn. This paper aims to bring together much of what has been written in various tutorials, to suggest a process for using SPC in QI projects. We highlight four critical decision points that are often missed, how these are inter-related and how they affect the inferences that can be drawn regarding effectiveness of the intervention: (1) the need for a stable baseline to enable drawing inferences on effectiveness; (2) choice of outcome measures to assess effectiveness, safety and intervention fidelity; (3) design features to improve the quality of QI projects; (4) choice of SPC analysis aligned with the type of outcome, and reporting on the potential influence of other interventions or secular trends.These decision points should be explicitly reported for readers to interpret and judge the results, and can be seen as supplementing the Standards for Quality Improvement Reporting Excellence guidelines. Thinking in advance about both formative and summative evaluation will inform more deliberate choices and strengthen the evidence produced by QI projects.

Learning curve of transoral robotic thyroidectomy

BACKGROUND

Transoral thyroidectomy has superior cosmesis and better postoperative voice outcomes than conventional thyroidectomy. However, it usually requires a steep learning curve and longer operative time. The transoral robotic thyroidectomy (TORT) learning curve has not been well investigated. This study aimed to evaluate the TORT learning curve and factors affecting operative time.

METHODS

We retrospectively studied 173 consecutive patients who underwent TORT with or without central neck dissection from July 2017 to August 2021. We assessed the TORT learning curve using operative time, complication rate, and surgical success (procedure conversion) rate. The operative time and surgical success rate learning curves were calculated using the cumulative summation (CUSUM) method. Additionally, we analyzed factors affecting operative time in TORT.

RESULTS

Total thyroidectomy operative time was significantly longer than those of lobectomy and isthmusectomy (p < 0.001). In correlation analysis, a significantly positive correlation was observed between body mass index (BMI) and operative time (R² = 0.04, p = 0.025). The TORT learning curve was 52 cases in the CUSUM operative time analysis. In the CUSUM surgical success rate chart, the turning point was the 55th case. Complication and procedure conversion rates were significantly decreased after the learning curve.

CONCLUSIONS

The CUSUM learning curve of TORT was about 52-55 cases, and the operative time, total complication rate, and procedure conversion decreased significantly after the learning curve. The operative time was associated with the extent of thyroidectomy and BMI.

Mortality as an indicator of quality of neurosurgical care in England: a retrospective cohort study

OBJECTIVES

Postoperative mortality is a widely used quality indicator, but it may be unreliable when procedure numbers and/or mortality rates are low, due to insufficient statistical power. The objective was to investigate the statistical validity of postoperative 30-day mortality as a quality metric for neurosurgical practice across healthcare providers.

DESIGN

Retrospective cohort study.

SETTING

Hospital Episode Statistics data from all neurosurgical units in England.

PARTICIPANTS

Patients who underwent neurosurgical procedures between April 2013 and March 2018. Procedures were grouped using the National Neurosurgical Audit Programme classification.

OUTCOMES MEASURED

National 30-day postoperative mortality rates were calculated for elective and non-elective neurosurgical procedural groups. The study estimated the proportion of neurosurgeons and NHS trusts in England that performed sufficient procedures in 3-year and 5-year periods to detect unusual performance (defined as double the national rate of mortality). The actual difference in mortality rates that could be reliably detected based on procedure volumes of neurosurgeons and units over a 5-year period was modelled.

RESULTS

The 30-day mortality rates for all elective and non-elective procedures were 0.4% and 6.1%, respectively. Only one neurosurgeon in England achieved the minimum sample size (n=2402) of elective cases in 5 years needed to detect if their mortality rate was double the national average. All neurosurgical units achieved the minimum sample sizes for both elective (n=2402) and non-elective (n=149) procedures. In several neurosurgical subspecialties, approximately 80% of units (or more) achieved the minimum sample sizes needed to detect if their mortality rate was double the national rate, including elective neuro-oncology (baseline mortality rate=2.3%), non-elective neuro-oncology (rate=5.7%), neurovascular (rate=6.7%) and trauma (rate=11%).

CONCLUSION

Postoperative mortality lacks statistical power as a measure of individual neurosurgeon performance. Neurosurgical units in England performed sufficient procedure numbers overall and in several subspecialty areas to support the use of mortality as a quality indicator.

Completion of postoperative polysomnography for children with severe obstructive sleep apnea: A quality improvement project

Objective

Pediatric patients with severe obstructive sleep apnea (OSA) are at risk for residual OSA following tonsillectomy with/without adenoidectomy (T ± A). We initiated a quality improvement (QI) project to increase the percentage of postoperative (postop) polysomnography (PSG) completion to identify residual OSA.

Methods

This is a prospective QI project carried out at a tertiary pediatric academic hospital. Children ≤18 years of age who underwent T ± A for severe OSA were included. Our Specific, Measurable, Attainable, Relevant, and Time-based (SMART) aim was to increase the percentage of completed postop PSGs in this cohort from a baseline of 70% to95% by May 31, 2021. We focused on patient education and leveraged both clinical decision support and reporting functionalities of the electronic medical record for project implementation.

Results

During the pre-intervention period between January 1, 2019 to June 30, 2020, 472 patients met the inclusion criteria with an average age of 8.6 years (SD 4.6). The rate of postop PSG completion was 69.7% (SD 11.4%) with an average time of 99 days (SD 66) between surgery and the postop PSG. A shift was observed starting in September 2020, and the PSG completion rate improved to 94.9% by September 30, 2021. Post-intervention, there were 178 patients with an average age of 9.3 years (SD 4.9). The average time between surgery and the postop PSG was significantly reduced to 57 days (SD 16; p < .001).

Conclusions

Through a multidisciplinary approach, we successfully completed our SMART aim. With the establishment of QI infrastructure, our goal is to deliver better care in a sustainable fashion using QI methodology.

The problem with dichotomizing quality improvement measures

The Anesthesia Quality Institute (AQI) promotes improvements in clinical care outcomes by managing data entered in the National Anesthesia Clinical Outcomes Registry (NACOR). Each case included in NACOR is classified as “performance met” or “performance not met” and expressed as a percentage for a length of time. The clarity associated with this binary classification is associated with limitations on data analysis and presentations that may not be optimal guides to evaluate the quality of care. High compliance benchmarks present another obstacle for evaluating quality. Traditional approaches for interpreting statistical process control (SPC) charts depend on data points above and below a center line, which may not provide adequate characterizations of a QI process with a low failure rate, or few possible data points below the center line. This article demonstrates the limitations associated with the use of binary datasets to evaluate the quality of care at an individual organization with QI measures, describes a method for characterizing binary data with continuous variables and presents a solution to analyze rare QI events using g charts.

Using the Laney p’ Control Chart for Monitoring COVID-19 Cases in Jordan

In this research, we examine the use of the Laney p' control chart and the application of test rules to assess governmental interventions throughout the COVID-19 pandemic and understand how certain activities and events that took place affected the infection rate. Data for the infection rate (IR) were collected between October 31, 2020, and March 19, 2022. The IR was calculated by dividing the number of confirmed cases by the number of PCR (polymerase chain reaction) tests performed. The IR data were subsequently plotted on the Laney p' control charts using the Minitab software. The charts thereby allowed us to study the effects on infection rates of the government's moves to restrict the movements and activities of the population, as well as the results of easing these restrictions. The restrictive measures proved to be effective in decreasing the infection rate, whereas relaxing these measures had the opposite effect. Typically, test signals are considered as an indication of a change in a process, although in some situations we have observed that slight changes are not accompanied by a signal. Regardless, the analysis shows cases where using test rules rapidly detected patterns and changes in IR, and allowing remedial action to be taken without delay. In this study, we use the Laney p' control chart to monitor the COVID-19 IR and compare its performance with that of the EWMA control chart. In addition, we analyze the performance of various test rules in detecting IR changes. Comparing the Laney p' control chart with the EWMA control chart, the data showed that in most cases, the Laney p' control chart was able to identify the change of IR faster. Comparing the performance of different tests in detecting changes in the IR, one can see that no particular test outperformed the others in all cases. We also recommend analyzing the data points in both single-stage and multistage analyses in accordance with this new perspective rather than the traditional one used in process improvement projects. Accordingly, the single-stage analysis gives a complete picture of how the infection rate is changing overall, whereas the multistage analysis is more sensitive to small changes.

Exponentially weighed moving average charts for monitoring zero-inflated proportions with applications in health care

In the context of public health surveillance, the aim is to monitor the occurrence of health-related events. Among them, statistical process monitoring focuses very often on the monitoring of rates and proportions (i.e. values in (0,1)) such as the proportion of patients with a specific disease. A popular control chart that is able to detect quickly small to moderate shifts in process parameters is the exponentially weighed moving average control chart. There are various models that are used to describe values in (0,1). However, especially in the case of rare health events, zero values occur very frequently which, for example, denote the absence of the disease. In this paper, we study the performance and the statistical design of exponentially weighed moving average control charts for monitoring proportions that arise in a health-related framework. The proposed chart is based on the zero-inflated Beta distribution, a mixed (discrete-continuous) distribution, suitable for modelling data in [0,1). We use a Markov chain method to study the run length distribution of the exponentially weighed moving average chart. Also, we investigate the statistical design as well as the performance of the proposed charts. Comparisons with a Shewhart-type chart are also given. Finally, we provide an example for the practical implementation of the proposed charts.

A control chart based on Pearson residuals for a negative binomial regression: application to infant mortality data

Purpose

The study aims to design a control chart based on an exponentially weighted moving average (EWMA) chart of Pearson's residuals of a model of negative binomial regression in order to detect possible anomalies in mortality data.

Design/methodology/approach

In order to evaluate the performance of the proposed chart, the authors have considered official historical records of death of children of Ecuador. A negative binomial regression model was fitted to the data, and a chart of the Pearson residuals was designed. The parameters of the chart were obtained by simulation, as well as the performances of the charts related to changes in the mean of death.

Findings

When the chart was plotted, outliers were detected in the deaths of children in the years 1990–1995, 2001–2006, 2013–2015, which could show that there are underreporting or an excessive growth in mortality. In the analysis of performances, the value of λ = 0.05 presented the fastest detection of changes in the mean death.

Originality/value

The proposed charts present better performances in relation to EWMA charts for deviance residuals, with a remarkable advantage of the Pearson residuals, which are much easier to interpret and calculate. Finally, the authors would like to point out that although this paper only applies control charts to Ecuadorian infant mortality, the methodology can be used to calculate mortality in any geographical area or to detect outbreaks of infectious diseases.

Quality Standards for Surgery of Colorectal Peritoneal Metastasis After Cytoreductive Surgery and Hyperthermic Intraperitoneal Chemotherapy

BACKGROUND

The standardization of surgical outcomes throughout surgical procedures is mandatory. Cytoreductive surgery with hyperthermic intraperitoneal chemotherapy (CRS + HIPEC) should provide proficient oncological and surgical outcomes.

STUDY DESIGN

The aim of this study was to identify clinically relevant quality indicators and their quality standard, and to determine their acceptable quality limit. A systematic review on cytoreductive results from 2000 to 2018 was performed focusing on clinical guidelines, consensus conferences, and publications. After the selection of quality indicators, a systematic review of indexed references was performed in order to calculate the quality standard for each indicator.

STUDY SELECTION

Unicentric/multicentric series, comparative studies, and clinical trials. Studies were to include outcomes after cytoreduction of colorectal origin and series with more than 50 patients. Quality indicators with at least 10 series were mandatory and objective measurements were also mandatory for inclusion.

MAIN OUTCOME MEASUREMENTS

Quality indicators selected were 1- to 5-year survival, overall disease-free survival, 1- to 5-year disease-free survival, complete surgical resection, duration of surgery, length of stay, overall morbimortality, major morbidity, re-intervention, postoperative hemorrhage, intestinal fistula, anastomotic leakage, wound infection, postoperative medical complications, overall recurrence, and failure to rescue.

RESULTS

The most relevant quality indicators and critical quality limits were overall disease-free survival and 5-year overall disease-free survival (14 months and <10 months, and 14% and <4%, respectively), completeness of surgical resection (89% and <80%, respectively), overall mortality (3% and >8%, respectively), overall morbidity (47% and >63%, respectively), failure to rescue (12% and <30%, respectively), reintervention (13 and <22%, respectively), anastomotic leakage (6% and <13%, respectively), and overall recurrence (60% and <74%, respectively).

CONCLUSION

This is the first study to assess quality standards in CRS + HIPEC for colorectal peritoneal metastases. The current data are of particular relevance for future studies to control the variability of this surgery.

Comparative Effectiveness of Risk-adjusted Cumulative Sum and Periodic Evaluation for Monitoring Hospital Perioperative Mortality

BACKGROUND

National surgical quality improvement (QI) programs use periodic, risk-adjusted evaluation to identify hospitals with higher than expected perioperative mortality. Rapid, accurate identification of poorly performing hospitals is critical for avoiding potentially preventable mortality and represents an opportunity to enhance QI efforts.

METHODS

Hospital-level analysis using Veterans Affairs (VA) Surgical Quality Improvement Program data (2011-2016) to compare identification of hospitals with excess, risk-adjusted 30-day mortality using observed-to-expected (O-E) ratios (ie, current gold standard) and cumulative sum (CUSUM) with V-mask. Various V-mask slopes and radii were evaluated-slope of 2.5 and radius of 1.0 was used as the base case.

RESULTS

Hospitals identified by CUSUM and quarterly O-E were identified midway into a quarter [median 47 days; interquartile range (IQR): 24-61 days before quarter end] translating to a median of 129 (IQR: 60-187) surgical cases and 368 (IQR: 145-681) postoperative inpatient days occurring after a CUSUM signal, but before the quarter end. At hospitals identified by CUSUM but not O-E, a median of 2 deaths within a median of 5 days triggered a signal. In some cases, these clusters extended beyond CUSUM identification date with as many as 8 deaths undetected using O-E. Sensitivity and negative predictive values for CUSUM relative to O-E were 71.9% (95% confidence interval: 66.2%-77.1%) and 95.5% (94.4%-96.4%), respectively.

CONCLUSIONS

CUSUM evaluation identifies hospitals with clusters of mortality in excess of expected more rapidly than periodic analysis. CUSUM represents an analytic tool national QI programs could utilize to provide participating hospitals with data that could facilitate more proactive implementation of local interventions to help reduce potentially avoidable perioperative mortality.

Monitoring operating room performance with control charts: findings from a Greek public hospital

OBJECTIVE

To describe the development and demonstrate the use of a statistical framework based on statistical quality control (SQC) in order to monitor the performance of operating rooms (ORs).

DESIGN

Data related to scheduled surgical operations have been collected from the information system of an existing Greek hospital. The data that contain the anesthesia and operation start and completion times of the operations carried out in the 14 ORs of the hospital are analyzed using control p-charts and hypotheses testing. The results obtained provide crucial information to health-care managers.

SETTING

A large Greek public hospital.

PARTICIPANTS

Real-world data captured on daily basis from January 2015 to November 2017.

INTERVENTION

The proportion of the idle time of an OR over its total available time is proposed as an OR key performance index. We present two directions of data monitoring and analysis: one that uses control p-charts and a second based on hypotheses testing. The improved Laney's p΄-chart and the Laney's approach for cross-sectional data are employed in order to overcome overdispersion that affects OR idle time data.

RESULTS

The proposed methodology allows hospital management (i) to monitor the percentage of the idle time of an operating room through time and (ii) to identify the ORs that demonstrate exceptionally high or low percentage of idle time at a given period of time.

CONCLUSION

SQC charts are simple, yet powerful tools that may support the hospital management in monitoring OR performance and decision-making. The development of a dedicated management information system that automatically captures the required data and constructs the corresponding control charts would support effectively managerial decision-making.

Episode detection based on personalized intensity of care thresholds: a schizophrenia case study

BACKGROUND

Schizophrenia Spectrum Disorder (SSD) is characterized by its chronic, episodic nature. The clear definition of such episodes is essential for various clinical and research purposes. Most current definitions of episodes in SSD are based on either hospitalizations or on symptom scales. Both have drawbacks; symptom scales are measured infrequently, while hospitalization rates are often affected by policy. This study presents an approach for defining episodes in healthcare data that does not suffer such drawbacks.

METHODS

Healthcare use of 13,155 SSD patients in the Northern Netherlands with up to 12 years of follow-up was available. Patient-level structural changes in the trend of healthcare use costs were determined using Exponentially Weighted Moving Average (EWMA) control charts. Control charts restart with updated parameters after a detected structural change. Episodes were defined using these structural changes. The resulting episodes were validated by investigating their association with the Global Assessment of Functioning (GAF) scale.

RESULTS

The mean number of episodes was 0.61 (sd: 0.60) per patient per year. For the sub-group without hospitalizations this was 0.51 (sd: 0.71). Average episode duration of the sub-group (147 days, sd: 309.4) was similar to that of the full sample (150 days, sd: 305.5). A significant inverse association was identified between GAF scores and the episode-state indicator.

CONCLUSIONS

The repeated application of EWMA control charts based on healthcare-intensity is a feasible and promising tool for quantifying patient-level healthcare episodes. The validation using GAF scores indicates that our episode indicator is associated with lower levels of global functioning. Results for individuals without hospitalizations indicate that the method is robust with regard to changes in healthcare policy.

Monitoring Hospital Performance with Statistical Process Control After Total Hip and Knee Arthroplasty: A Study to Determine How Much Earlier Worsening Performance Can Be Detected

BACKGROUND

Given the low early revision rate after total hip arthroplasty (THA) and total knee arthroplasty (TKA), hospital performance is typically compared using 3 years of data. The purpose of this study was to assess how much earlier worsening hospital performance in 1-year revision rates after THA and TKA can be detected.

METHODS

All 86,468 THA and 73,077 TKA procedures performed from 2014 to 2016 and recorded in the Dutch Arthroplasty Register were included. Negative outlier hospitals were identified by significantly higher O/E (observed divided by expected) 1-year revision rates in a funnel plot. Monthly Shewhart p-charts (with 2 and 3-sigma control limits) and cumulative sum (CUSUM) charts (with 3.5 and 5 control limits) were constructed to detect a doubling of revisions (odds ratio of 2), generating a signal when the control limit was reached. The median number of months until generation of a first signal for negative outliers and the number of false signals for non-negative outliers were calculated. Sensitivity, specificity, and accuracy were calculated for all charts and control limit settings using outlier status in the funnel plot as the gold standard.

RESULTS

The funnel plot showed that 13 of 97 hospitals had significantly higher O/E 1-year revision rates and were negative outliers for THA and 7 of 98 hospitals had significantly higher O/E 1-year revision rates and were negative outliers for TKA. The Shewhart p-chart with the 3-sigma control limit generated 68 signals (34 false-positive) for THA and 85 signals (63 false-positive) for TKA. The sensitivity for THA and TKA was 92% and 100%, respectively; the specificity was 69% and 51%, respectively; and the accuracy was 72% and 54%, respectively. The CUSUM chart with a 5 control limit generated 18 signals (1 false-positive) for THA and 7 (1 false-positive) for TKA. The sensitivity was 85% and 71% for THA and TKA, respectively; the specificity was 99% for both; and the accuracy was 97% for both. The Shewhart p-chart with a 3-sigma control limit generated the first signal for negative outliers after a median of 10 months (interquartile range [IQR] = 2 to 18) for THA and 13 months (IQR = 5 to 18) for TKA. The CUSUM chart with a 5 control limit generated the first signal after a median of 18 months (IQR = 7 to 22) for THA and 21 months (IQR = 9 to 25) for TKA.

CONCLUSIONS

Monthly monitoring using CUSUM charts with a 5 control limit enables earlier detection of worsening 1-year revision rates with accuracy so that initiatives to improve care can start earlier.

Continuous Monitoring of Coronary Care Mortality Using the Global Registry for Acute Coronary Events (GRACE) Score

BACKGROUND

The aim was to assess the utility of cumulative sum techniques for continuous monitoring of coronary care outcomes, applied to patients with acute coronary syndrome stratified by the Global Registry for Acute Coronary Events (GRACE) risk score.

METHODS

A prospective longitudinal study to assess GRACE score for real-time monitoring of coronary care mortality in a community hospital was conducted between January 2012 and December 2017. An expected-to-observed probability of death chart for individual risk and a variable life-adjusted display were used to monitor the results.

RESULTS

A total of 1,255 patients undergoing acute coronary syndrome were included in the analysis. GRACE-based variable life-adjusted plots monitoring in-hospital mortality showed that observed death rates remained in general within the expected 95% confidence limit over time, and these behaviors were similar for ST-segment elevation and non-ST-segment elevation myocardial infarction. In-hospital all-cause mortality was 2.6% for the overall population, and 56% of these cases corresponded to unexpected deaths; conversely, unexpected survival occurred in 5.2% of survivors.

CONCLUSIONS

Continuous monitoring of coronary care mortality based on cumulative sum charts and the GRACE score demonstrated the occurrence of series of favorable and unfavorable outcomes on a real-time basis. Additionally, plotting the expected-to-observed probability of death for individual cases was useful to individualize unexpected deaths in low-risk patients. Although overall coronary care performance was adequate according to the GRACE score, we found that there is still some room for improvement, since over half of the deaths occurring in low-risk patients were potentially preventable.

Including non-concurrent control patients in the analysis of platform trials: is it worth it?

Background

Platform trials allow adding new experimental treatments to an on-going trial. This feature is attractive to practitioners due to improved efficiency. Nevertheless, the operating characteristics of a trial that adds arms have not been well-studied. One controversy is whether just the concurrent control data (i.e. of patients who are recruited after a new arm is added) should be used in the analysis of the newly added treatment(s), or all control data (i.e. non-concurrent and concurrent).

Methods

We investigate the benefits and drawbacks of using non-concurrent control data within a two-stage setting. We perform simulation studies to explore the impact of a linear and a step trend on the inference of the trial. We compare several analysis approaches when one includes all the control data or only concurrent control data in the analysis of the newly added treatment.

Results

When there is a positive trend and all the control data are used, the marginal power of rejecting the corresponding hypothesis and the type one error rate can be higher than the nominal value. A model-based approach adjusting for a stage effect is equivalent to using concurrent control data; an adjustment with a linear term may not guarantee valid inference when there is a non-linear trend.

Conclusions

If strict error rate control is required then non-concurrent control data should not be used; otherwise it may be beneficial if the trend is sufficiently small. On the other hand, the root mean squared error of the estimated treatment effect can be improved through using non-concurrent control data.

Outcome quality standards for surgery of colorectal liver metastasis

PURPOSE

Liver metastases are the most common malignant solid liver lesions, approximately 40% of which stem from colorectal tumors. Liver resection is currently the only curative treatment for colorectal cancer liver metastases (CRLM). However, there is a lack of consensus criteria to assess the results of this treatment. In order to evaluate the quality of surgical outcomes, it is necessary to identify quality indicators (QIs) and their corresponding quality standards (QS). We propose a simple method to determine QI and QS in CRLM surgery (CRLMS) and establish acceptable quality limits (AQL) for each QI.

MATERIAL AND METHODS

A systematic review of CRLMS results published from 2006 to 2016. Clinical guidelines, consensus conferences, and publications related to the CRLMS were reviewed to identify and select QIs. Once selected, a new review of the papers including the results of at least one of the QIs was performed. Statistical process control (SPC) method was applied to calculate the QS and AQL of each QI. The limits of variability were established from mean and confidence intervals at 95% and 99.8%.

RESULTS

The most relevant QIs and its AQLs were postoperative mortality (2%, < 4.5%), overall postoperative morbidity (33%, < 41%), liver failure (5%, < 8%), postoperative hemorrhage (1%, < 3%), biliary fistula (6%, < 10%), reoperation (3%, < 6%), R1 resection margins (18%, < 25%), and overall survival at 12 and 60 months (84%, > 77%; and 34%, > 25%, respectively).

CONCLUSIONS

Despite its limitations, the present study constitutes the most extensive scientific evidence to date on QI and AQL in CRLMS and may constitute a reference in future studies.

A Fast Online Monitoring Approach for Surgical Risks

Risk monitoring has been widely used in health care, further, control charts are often used as monitoring methods for surgical outcomes. Most of the methods can only detect step shifts of position parameters, but cannot take measures on scale parameters. In this paper, we proposed four methods based on EWMA control charts, namely SESOP, STSSO, SESOP-MFIR and STSSO-MFIR, to improve the existing monitoring methods. Specifically, SESOP standardizes variable on the basis of an EWMA charting method; STSSO replaces the statistics of the original EWMA charting method with the score test statistics; for SESOP-MFIR and STSSO-MFIR, we upgrade their control limits from asymptotic to time-varying based on SESOP and STSSO, which enhance the timeliness of the earlier shifts monitoring. In order to verify the improvement of surgical outcomes monitoring, we respectively carry out simulation experiment and a practical application on ESOP and our four methods. SESOP can raise the overall efficiency of detecting shifts; STSSO led to a significant increase in the monitoring stability, especially for small volatilities; the optimization brought by SESOP-MFIR and STSSO-MFIR are more obvious, that the speed of detecting earlier shifts can even be reduced to half of the existing methods. Then, we apply these methods to the SOMIP program of Hong Kong, SESOP-MFIR and STSSO-MFIR have the best performance and can detect early shifts in time. According to the results, the methods we proposed can monitor both early shifts and scale parameters and improve the performance of surgical outcome monitoring in different degrees compared to those existing methods.

Monitoring surgical quality: the cumulative sum (CUSUM) approach

Monitoring the quality of new or ongoing surgical activities is a necessity. Several Statistical Process Control (SPC) tools are available to professionals. Among them, Shewhart charts and cumulative sum charts (CUSUM charts) are useful methods to provide visual feedback before significant quality issues arise. In this paper, we discuss both methods based on our current approach. On Shewhart charts, one variable value is plotted on a time-series line. This method provides information about every single determination. Random variations of the values appear and by adjusting the adequate control limits it is possible to know whether those variations are random or out-of-control. Although large variations are easily detected, small but relevant changes are not. On the contrary, CUSUM charts have the capability of detecting small changes quickly. CUSUM is defined as a statistical tool that graphically represents the sequential monitoring of cumulative performance of any dichotomized or continuous variable under assessment. It emphasizes failures penalizing them against the correct performance when individual risk is adjusted. This makes CUSUM especially sensitive to negative changes. CUSUM can be created without the need of a specific sample size and grow with every new case included. Besides the variable under control (with specific definitions of acceptable and unacceptable outcomes), the type I and II errors for the defined parameter and the individual risk of acceptable or unacceptable outcomes must be included in the chart. Graphical representation of these three parameters is easy and intuitive to read making CUSUM graphs a reliable tool to understand the trending of the parameter under control. If performance is considered inadequate: analysis, discussion and implementation of agreed measures should be taken. Despite its limitations, CUSUM analysis is considered the best tool for quality control in health care domain.

Large-scale empirical optimisation of statistical control charts to detect clinically relevant increases in surgical site infection rates

OBJECTIVE

Surgical site infections (SSIs) are common costly hospital-acquired conditions. While statistical process control (SPC) use in healthcare has increased, limited rigorous empirical research compares and optimises these methods for SSI surveillance. We sought to determine which SPC chart types and design parameters maximise the detection of clinically relevant SSI rate increases while minimising false alarms.

DESIGN

Systematic retrospective data analysis and empirical optimisation.

METHODS

We analysed 12 years of data on 13 surgical procedures from a network of 58 community hospitals. Statistically significant SSI rate increases (signals) at individual hospitals initially were identified using 50 different SPC chart variations (Shewhart or exponentially weighted moving average, 5 baseline periods, 5 baseline types). Blinded epidemiologists evaluated the clinical significance of 2709 representative signals of potential outbreaks (out of 5536 generated), rating them as requiring 'action' or 'no action'. These ratings were used to identify which SPC approaches maximised sensitivity and specificity within a broader set of 3600 individual chart variations (additional baseline variations and chart types, including moving average (MA), and five control limit widths) and over 32 million dual-chart combinations based on different baseline periods, reference data (network-wide vs local hospital SSI rates), control limit widths and other calculation considerations. Results were validated with an additional year of data from the same hospital cohort.

RESULTS

The optimal SPC approach to detect clinically important SSI rate increases used two simultaneous MA charts calculated using lagged rolling baseline windows and 1 SD limits. The first chart used 12-month MAs with 18-month baselines and best identified small sustained increases above network-wide SSI rates. The second chart used 6-month MAs with 3-month baselines and best detected large short-term increases above individual hospital SSI rates. This combination outperformed more commonly used charts, with high sensitivity (0.90; positive predictive value=0.56) and practical specificity (0.67; negative predictive value=0.94).

CONCLUSIONS

An optimised combination of two MA charts had the best performance for identifying clinically relevant small but sustained above-network SSI rates and large short-term individual hospital increases.

Robotic-Assisted Roux-en-Y Gastric Bypass: Learning Curve Assessment Using Cumulative Sum and Literature Review

Purpose: Robotic-assisted Roux-en-Y gastric bypass (RARYGB) is a procedure that is used with increasing frequency in the United States. Among other bariatric procedures, RARYGB is a good model for the robotic platform because it allows hand-sewn suturing and energy devices application. The aim of this study was to conduct a literature review of robotic approach in RARYGB, its learning curve using the cumulative sum (CUSUM) method, and our experience as Center of Excellence recognized by the American Society for Metabolic and Bariatric Surgery (ASMBS). Methods: A total of 67 patients were included. Results revealed that the learning curve was achieved after case 11. Eighteen studies were included in the pooled analysis. Results: An increase in the operative time was noted at case 46, in which a second phase was identified. A significant difference between these two phases was found only related to previous bariatric surgery. The outcomes of this series were comparable with the ones available in the literature. Conclusions: The robotic platform is increasing its role in complex procedures such as RARYGB. The hand-sewn technique may offer important advantages in terms of shorter learning curve, reduced conversion rate, and lower leakage rate.

FORWARD (Feeding via the Oral Route With Acknowledged Risk of Deterioration): evaluation of a novel tool to support patients eating and drinking at risk of aspiration

BACKGROUND

care of patients with a permanently unsafe swallow who are inappropriate for tube feeding is challenging. Eating and drinking with acknowledged risk (EDAR) may be an appropriate strategy but without clear decision making and communication patients may spend unnecessarily long 'nil by mouth' (NBM), they or their family may experience significant anxieties and advance care plans may not be made.

METHODS

the FORWARD (Feeding via the Oral Route With Acknowledged Risk of Deterioration) care bundle was sequentially co-designed and embedded across different in-patient clinical services using 'plan-do-study-act' methodology to systematise best practice. Care before and after FORWARD's implementation was evaluated using a time-series analysis of 305 'EDAR patients' (19 in 6 months pre-FORWARD; 42 in a 12-month 'pilot'; 244 patients in the subsequent 27 months).

RESULTS

median (IQR) days patients were NBM without an alternative feeding route reduced significantly from 2 (1-4) pre-FORWARD to 0 (0-2) in the 'pilot' and 0 (0) post-'pilot' (P < 0.05). G-chart analysis demonstrated sustained performance across time and different clinical settings. Implementation of FORWARD significantly improved documentation of capacity assessment (42%→98%), discussions with next of kin (47%→98%) and onward communication of feeding plans (67%→81%). In wards where FORWARD was introduced, rate of aspiration pneumonia (a 'balancing measure' sensitive to harm associated with EDAR) increased at half the rate of dysphagia (0.8%/year versus 1.6%/year).

CONCLUSION

the FORWARD care bundle supported complex decision-making around EDAR in patients with persistent dysphagia. The benefits of FORWARD were shown to be sustained over time and in a wide in-patient context.

Development of a single-center quality bundle to prevent sudden unexpected postnatal collapse

BACKGROUND

Recent reports suggest a rising awareness of sudden unexpected postnatal collapse (SUPC).

LOCAL PROBLEM

Five SUPC events during a 17-month period.

METHODS

A multidisciplinary team used a quality-improvement approach to develop the intervention. The smart aim was to develop a bundled intervention to eliminate SUPC from occurring in the delivery room during skin-to-skin care.

INTERVENTION

A bundled intervention included a standardized assessment tool and measurement of oxygen saturation levels, with prescribed responses to abnormal values, during skin-to-skin care in the delivery room.

RESULTS

Pre-intervention, there were five SUPC events/9143 live births (incidence 0.54/1000 live births) compared with 0 SUPC events/13,964 live births post intervention, p = 0.011. Special cause variation was achieved after implementation when the number of deliveries between SUPC events exceeded 3-sigma.

CONCLUSION

A bundled approach to monitoring during skin-to-skin care, including measurement of oxygen saturation, was associated with no additional cases of SUPC.

Flickering in Information Spreading Precedes Critical Transitions in Financial Markets

As many complex dynamical systems, financial markets exhibit sudden changes or tipping points that can turn into systemic risk. This paper aims at building and validating a new class of early warning signals of critical transitions. We base our analysis on information spreading patterns in dynamic temporal networks, where nodes are connected by short-term causality. Before a tipping point occurs, we observe flickering in information spreading, as measured by clustering coefficients. Nodes rapidly switch between "being in" and "being out" the information diffusion process. Concurrently, stock markets start to desynchronize. To capture these features, we build two early warning indicators based on the number of regime switches, and on the time between two switches. We divide our data into two sub-samples. Over the first one, using receiver operating curve, we show that we are able to detect a tipping point about one year before it occurs. For instance, our empirical model perfectly predicts the Global Financial Crisis. Over the second sub-sample, used as a robustness check, our two statistical metrics also capture, to a large extent, the 2016 financial turmoil. Our results suggest that our indicators have informational content about a future tipping point, and have therefore strong policy implications.

Implementing Strategies to Identify and Mitigate Adverse Safety Events: A Case Study with Unplanned Extubations

BACKGROUND

Patient safety events result from failures in complex health care delivery processes. To ensure safety, teams must implement ways to identify events that occur in a nonrandom fashion and respond in a timely manner. To illustrate this, one children's hospital's experience with an outbreak of unplanned extubations (UEs) in the neonatal ICU (NICU) is described.

METHODS

The quality improvement team measured UEs using three complementary data streams. Interventions to decrease the rate of UE were tested with success. Three statistical process control (SPC) charts (u-chart, g-chart, and an exponentially weighted moving average [EWMA] chart) were used for real-time monitoring.

RESULTS

From July 2015 to May 2016, the UE rate was stable at 1.1 UE/100 ventilator days. In early June 2016, a cluster of UEs, including four events within one week, was observed. Two of three SPC charts showed special cause variation, although at different time points. The EWMA chart alerted the team more than two weeks earlier than the u-chart. Within days of discovering the outbreak, the team identified that the hospital had replaced the tape used to secure endotracheal tubes with a nearly identical product. After multiple tape products were tested over the next month, the team selected one that returned the system to a state of stability.

CONCLUSION

Ongoing monitoring using SPC charts allowed early detection and rapid mitigation of an outbreak of UEs in the NICU. This highlights the importance of continuous monitoring using tools such as SPC charts that can alert teams to both improvement and worsening of processes.

Quantitative Assessment of the Learning Curve for Robotic Thyroid Surgery

With the increased utilization of robot thyroidectomy in recent years, surgical proficiency is the paramount consideration. However, there is no single perfect or ideal method for measuring surgical proficiency. In this study, we evaluated the learning curve of robotic thyroidectomy using various parameters. A total of 172 robotic total thyroidectomies were performed by a single surgeon between March 2014 and February 2018. Cumulative summation analysis revealed that it took 50 cases for the surgeon to significantly improve the operation time. Mean operation time was significantly shorter in the group that included the 51st to the 172nd case, than in the group that included only the first 50 cases (132.8 ± 27.7 min vs. 166.9 ± 29.5 min; p < 0.001). On the other hand, the surgeon was competent after the 75th case when postoperative transient hypoparathyroidism was used as the outcome measure. The incidence of hypoparathyroidism gradually decreased from 52.0%, for the first 75 cases, to 40.2% after the 76th case. These results indicated that the criteria used to assess proficiency greatly influenced the interpretation of the learning curve. Incorporation of the operation time, complications, and oncologic outcomes should be considered in learning curve assessment.

EWMA Control Chart Using Repetitive Sampling for Monitoring Blood Glucose Levels in Type-II Diabetes Patients

In this paper, we discuss the application of the exponentially weighted moving average (EWMA) control chart for the monitoring blood glucose in type-II diabetes patients. We present tables for the practical use in healthcare. From the simulation results and a real example, the efficiency of the proposed chart in detecting a shift in diabetic level is compared with the existing chart. It is found that the proposed chart provides a strict method to monitor the diabetic levels in diabetes patients. From the simulation results and a real example, it is concluded that the use of the proposed chart in health care issues may reduce the risk of heart disease by monitoring diabetic levels in an effective way.

Defining the learning curve for transanal total mesorectal excision for rectal adenocarcinoma

BACKGROUND

Early observational data suggest that this approach is safe and feasible, but it is technically challenging and the learning curve has not yet been determined. The objective of this study was to determine the number of cases required achieve proficiency in transanal total mesorectal excision (TA-TME) for rectal adenocarcinoma.

METHODS

All TA-TME cases performed from 03/2012-01/2017 at a single high-volume tertiary care institution for rectal adenocarcinoma were included. A cumulative summation (CUSUM) analysis was performed to determine the number of cases required to reach proficiency, defined as high-quality TME (complete or near-complete mesorectal envelope, negative distal (DRM), and circumferential resection (> 1 mm; CRM) margin). The acceptable and unacceptable rates of good quality TME were defined based on the incidence of high-quality TME in laparoscopic (unacceptable rate = 81.7%) and open (acceptable rate = 86.9%) arms of the ACOSOG Z6051 trial.

RESULTS

A total of 87 consecutive cases were included with mean tumor height 4.8 cm (SD 2.7) and 80% (70/87) received neoadjuvant chemoradiation. Post-operative morbidity occurred in 44% (38/87) of cases, including 21% (18/87) readmissions. Median length of stay was 4 days [IQR 3-8]. A good quality TME was performed in 95% (83/87) of cases including 98% (85/87) negative CRM, 99% (86/87) negative DRM, and 99% (86/87) complete or near-complete mesorectal envelope. CUSUM analysis reported that the good quality TME rate reaches an acceptable rate after 51 cases overall, and 45 cases if abdominoperineal resections are excluded.

CONCLUSION

TA-TME is a complex technique that requires a minimum of 45-51 cases to reach an acceptable incidence of high-quality TME and lower operative duration.