New quality monitoring tools provided by the Scientific Registry of Transplant Recipients: CUSUM

Using risk-adjusted cumulative sum to evaluate surgeon, divisional, and institutional outcomes-a feasibility study

BACKGROUND

Few objective, real-time measurements of surgeon performance exist. The risk-adjusted cumulative sum is a novel method that can track surgeon-level outcomes on a continuous basis. The objective of this study was to demonstrate the feasibility of using risk-adjusted cumulative sum to monitor outcomes after colorectal operations and identify clinically relevant performance variations.

METHODS

The National Surgical Quality Improvement Program was queried to obtain patient-level data for 1,603 colorectal operations at a high-volume center from 2011 to 2020. For each case, expected risks of morbidity, mortality, reoperation, readmission, and prolonged length of stay were estimated using the National Surgical Quality Improvement Program risk calculator. Risk-adjusted cumulative sum curves were generated to signal observed-to-expected odds ratios of 1.5 (poor performance) and 0.5 (exceptional performance). Control limits were set based on a false positive rate of 5% (α = 0.05).

RESULTS

The cohort included data on 7 surgeons (those with more than 20 cases in the study period). Institutional observed versus expected outcomes were the following: morbidity 12.5% (vs 15.0%), mortality 2.5% (vs 2.0%), prolonged length of stay 19.7% (vs 19.1%), reoperation 11.1% (vs 11.3%), and 30-day readmission 6.1% (vs 4.8%). Risk-adjusted cumulative sum accurately demonstrated within- and between-surgeon performance variations across these metrics and proved effective when considering division-level data.

CONCLUSION

Risk-adjusted cumulative sum adjusts for patient-level risk factors to provide real-time data on surgeon-specific outcomes. This approach enables prompt identification of performance outliers and can contribute to quality assurance, root-cause analysis, and incentivization not only at the surgeon level but at divisional and institutional levels as well.

Collaborative leadership in transplantation: Blending clinical, business, and regulatory roles

Transplantation is a high-risk, high-cost treatment for end-stage diseases and is the most strictly regulated area of healthcare in the United States. Thus, achieving success for patients and the program requires skillful and collaborative leadership. Various factors, such as outcomes, volume, and financial health, may measure the success of a transplant program. Strong collaboration between clinical and administrative leaders is key to achieving and maintaining success in those three categories. Clinical leaders of adult programs, such as medical and surgical directors, bear the primary responsibility for a program's volume, outcomes, and patient safety, while administrative directors are focused on business intelligence and regulatory compliance. This paper aims to provide readers with insights into the critical role of collaborative leadership in running a successful program, with a focus on clinical, business, and regulatory perspectives.

Monitoring performance in laparoscopic gastric bypass surgery using risk-adjusted cumulative sum at 2 high-volume centers

BACKGROUND

Traditional surgical outcomes are measured retrospectively and intermittently, limiting opportunities for early intervention.

OBJECTIVES

The objective of this study was to use risk-adjusted cumulative sum (RA-CUSUM) to track perioperative surgical outcomes for laparoscopic gastric bypass. We hypothesized that RA-CUSUM could identify performance variations between surgeons.

SETTING

Two mid-Atlantic quaternary care academic centers.

METHODS

Patient-level data from the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBSAQIP) were abstracted for laparoscopic gastric bypasses performed by 3 surgeons at 2 high-volume centers from 2014 to 2021. Estimated probabilities of serious complications, reoperation, and readmission were derived from the MBSAQIP risk calculator. RA-CUSUM curves were generated to signal observed-to-expected odds ratios (ORs) of 1.5 (poor performance) and .5 (superior performance). Control limits were set based on a false positive rate of 5% (α = .05).

RESULTS

We included 1192 patients: Surgeon A = 767, Surgeon B = 188, and Surgeon C = 237. Overall rates of serious complications, 30-day reoperations, and 30-day readmissions were 3.9%, 2.5%, and 5.2% respectively, with expected rates of 4.7%, 2.2%, and 5.8%. RA-CUSUM signaled lower-than-expected (OR < .5) rates of readmission and serious complication in Surgeon A, and higher-than-expected (OR > 1.5) readmission rate in Surgeon C. Surgeon A further demonstrated an early period of higher-than-expected (OR > 1.5) reoperation rate before April 2015, followed by superior performance thereafter (OR < .5). Surgeon B's performance generally reflected expected standards throughout the study period.

CONCLUSIONS

RA-CUSUM adjusts for clinical risk factors and identifies performance outliers in real-time. This approach to analyzing surgical outcomes is applicable to quality improvement, root-cause analysis, and surgeon incentivization.

Stakeholders' Perspectives on Transplant Metrics: The 2022 Scientific Registry of Transplant Recipients' Consensus Conference

In July 2022, the Scientific Registry of Transplant Recipients (SRTR) hosted an innovative, multistakeholder consensus conference to identify information and metrics desired by stakeholders in the transplantation system, including patients, living donors, caregivers, deceased donor family members, transplant professionals, organ procurement organization professionals, payers, and regulators. Crucially, patients, caregivers, living donors, and deceased donor family members were included in all aspects of this conference, including serving on the planning committee, participating in preconference focus groups and learning sessions, speaking at the conference, moderating conference sessions and breakout groups, and shaping the conclusions. Patients constituted 24% of the meeting participants. In this report, we document the proceedings and enumerate 160 recommendations, 10 of which have been highly prioritized. SRTR will use the recommendations to develop new presentations of information and metrics requested by stakeholders to support informed decision-making.

Quality assessment of donor liver procurement surgery using an unadjusted CUSUM prediction model. A practical nationwide evaluation

BACKGROUND

The aim of this study was to analyze the value of the unadjusted CUSUM graph of liver surgical injury and discard rates in organ procurement in the Netherlands.

METHODS

Unadjusted CUSUM graphs were plotted for surgical injury (C event) and discard rate (C2 event) from procured livers accepted for transplantation for each local procurement team compared with the total national cohort. The average incidence for each outcome was used as benchmark based on procurement quality forms (Sep 2010-Oct 2018). The data from the five Dutch procuring teams were blind-coded.

RESULTS

The C and C2 event rate were 17% and 1.9%, respectively (n = 1265). A total of 12 CUSUM charts were plotted for the national cohort and the five local teams. National CUSUM charts showed an overlapping "alarm signal." This overlapping signal for both C and C2, albeit a different time period, was only found in one local team. The other CUSUM alarm signal went off for two separate local teams, but only for C events or C2 events respectively, and at different points in time. The other remaining CUSUM charts showed no alarm signaling.

CONCLUSION

The unadjusted CUSUM chart is a simple and effective monitoring tool in following performance quality of organ procurement for liver transplantation. Both national and local recorded CUSUMs are useful to see the implication of national and local effects on organ procurement injury. Both procurement injury and organ discard are equally important in this analysis and need to be separately CUSUM charted.

Textbook Outcome as a Quality Metric in Living and Deceased Donor Kidney Transplantation

BACKGROUND

Quality in kidney transplantation is measured using 1-year patient and graft survival. Because 1-year patient and graft survival exceed 95%, this metric fails to measure a spectrum of quality. Textbook outcomes (TO) are a composite quality metric offering greater depth and resolution. We studied TO after living donor (LD) and deceased donor (DD) kidney transplantation.

STUDY DESIGN

United Network for Organ Sharing data for 69,165 transplant recipients between 2013 and 2017 were analyzed. TO was defined as patient and graft survival of 1 year or greater, 1-year glomerular filtration rate of greater than 40 mL/min, absence of delayed graft function, length of stay of 5 days or less, no readmissions during the first 6 months, and no episodes of rejection during the first year after transplantation. Bivariate analysis identified characteristics associated with TO, and covariates were incorporated into multivariable models. Five-year conditional survival was measured, and center TO rates were corrected for case complexity to allow center-level comparisons.

RESULTS

The national average TO rates were 54.1% and 31.7% for LD and DD transplant recipients. The hazard ratio for death at 5 years for recipients who did not experience TO was 1.92 (95% CI 1.68 to 2.18, p ≤ 0.0001) for LD transplant recipients and 2.08 (95% CI 1.93 to 2.24, p ≤ 0.0001) for DD transplant recipients. Center-level comparisons identify 18% and 24% of centers under-performing in LD and DD transplantation. High rates of TO do not correlate with transplantation center volume.

CONCLUSION

Kidney transplant recipients who experience TO have superior long-term survival. Textbook outcomes add value to the current standards of 1-year patient and graft survival.

Toward continuous improvement of Scientific Registry of Transplant Recipients performance reporting: Advances following 2012 consensus conference and future consensus building for 2022 consensus conference

The Scientific Registry of Transplant Recipients (SRTR) held a consensus conference in 2012 that examined methods used by SRTR for constructing performance metrics and made recommendations on how to improve program-specific reports. That consensus conference provided 25 recommendations categorized as follows: statistical methods, risk adjustment, and outcomes and data. During the subsequent decade, SRTR has implemented most of these recommendations; these are described in this article along with plans for another consensus conference in 2022. With the present article, SRTR aims to create transparency in the field of transplant metrics and guide discussion in the planning of the next consensus conference in 2022. The new conference will revisit the previous topics and have a broader focus to improve the metrics and information that SRTR provides. Readers can provide feedback on topics to be discussed at the next consensus conference as early as possible, by emailing srtr@srtr.org with the subject line "Task 5 Public Comment." This article is protected by copyright. All rights reserved.

Comparative Effectiveness of Risk-adjusted Cumulative Sum and Periodic Evaluation for Monitoring Hospital Perioperative Mortality

BACKGROUND

National surgical quality improvement (QI) programs use periodic, risk-adjusted evaluation to identify hospitals with higher than expected perioperative mortality. Rapid, accurate identification of poorly performing hospitals is critical for avoiding potentially preventable mortality and represents an opportunity to enhance QI efforts.

METHODS

Hospital-level analysis using Veterans Affairs (VA) Surgical Quality Improvement Program data (2011-2016) to compare identification of hospitals with excess, risk-adjusted 30-day mortality using observed-to-expected (O-E) ratios (ie, current gold standard) and cumulative sum (CUSUM) with V-mask. Various V-mask slopes and radii were evaluated-slope of 2.5 and radius of 1.0 was used as the base case.

RESULTS

Hospitals identified by CUSUM and quarterly O-E were identified midway into a quarter [median 47 days; interquartile range (IQR): 24-61 days before quarter end] translating to a median of 129 (IQR: 60-187) surgical cases and 368 (IQR: 145-681) postoperative inpatient days occurring after a CUSUM signal, but before the quarter end. At hospitals identified by CUSUM but not O-E, a median of 2 deaths within a median of 5 days triggered a signal. In some cases, these clusters extended beyond CUSUM identification date with as many as 8 deaths undetected using O-E. Sensitivity and negative predictive values for CUSUM relative to O-E were 71.9% (95% confidence interval: 66.2%-77.1%) and 95.5% (94.4%-96.4%), respectively.

CONCLUSIONS

CUSUM evaluation identifies hospitals with clusters of mortality in excess of expected more rapidly than periodic analysis. CUSUM represents an analytic tool national QI programs could utilize to provide participating hospitals with data that could facilitate more proactive implementation of local interventions to help reduce potentially avoidable perioperative mortality.

Current status: meeting the regulatory goals of your liver transplant program

PURPOSE OF REVIEW

The regulatory framework set by the Organ Procurement and Transplantation Network (OPTN) and Center for Medicare and Medicaid Services (CMS) for practice of liver transplantation in US is periodically updated and risk adjusted. Therefore, it is prudent for transplant centers to know the rules of engagement as it pertains to their practice.

RECENT FINDINGS

OPTN besides providing the regulatory oversight for safe and continued practice of transplant centers, provides necessary tools like: advanced statistical models and technological platforms to aid, and guide transplant centers including the necessary safeguards for high-quality transplant care.CMS regulations although had different thresholds to flag underperformance, often covered common grounds similar to the OPTN, therefore considered duplicative and unnecessary. But with much deliberation and consideration CMS undertook a major overhaul to the final rule for re-approval applications, a giant leap in the positive direction for transplant innovation and growth.

SUMMARY

The duplicative regulatory framework of OPTN and CMS has although achieved the goal of improving 1-year patient outcomes, it has proven costly in terms of slowing innovation, increasing organ discard and stunting growth of transplant volume. But the new updates in effect and also in the pipeline are a long-awaited opportunity for waiting transplant patients.

Constructing Learning Curves to Benchmark Operative Performance of General Surgery Residents Against a National Cohort of Peers

OBJECTIVE

No method or data exist to allow surgical trainees or their programs to contextualize their technical progress. The objective of this study was to create peer benchmarks for Cumulative Sum (CUSUM) charts based upon operative evaluations from a national cohort of general surgery residents.

DESIGN, SETTING, PARTICIPANTS

In 2016-2018, faculty from 26 general surgery residency programs nationwide rated 328 residents' operative performance on a case-by-case basis using a validated 5-point Likert scale. An individual case was considered a "misstep" if scoring below the national median score for that procedure in that postgraduate year (PGY). We constructed 2-sided observed-expected CUSUM charts to capture each resident's cumulative performance over time relative to the national medians. Upper (failure) and lower (positive outlier) benchmarks were established based on the PGY-specific 75th percentile and median misstep rates; consistent/repeated missteps are reflected by crossing of the upper boundary. Procedures with ≤10 observations and residents who were evaluated <10 times for each PGY were excluded.

RESULTS

Around 8,161 evaluations on 76 procedure types were analyzed. The individual misstep rate was lowest among PGY-3s at 13.3% and highest among PGY-4s at 28.6%. No interns had curves that crossed the failure boundary. 8.7% of PGY-2s and 8.9% of PGY-3s finished the year past the failure boundary. PGY-2s had the most positive outliers, with 28.3% of them demonstrating an outlying success performance beyond the lower boundary for at least once. PGY-5s most frequently failed, with 16.7% ever crossing the upper boundary and 11.1% remaining above it at graduation.

CONCLUSIONS

CUSUM is a valid statistical approach for benchmarking individual residents' operative performance against national peers as they progress through the year in real-time. With further validation, CUSUM could be used to set progression and/or graduation standards and objectively identify residents who might benefit from remediation.

Posttransplant outcome assessments at listing: Long-term outcomes are more important than short-term outcomes

Posttransplant outcome assessments are publicly reported for patient and regulatory use. However, the currently reported 1-year posttransplant graft survival assessments are commonly criticized for not identifying clinically meaningful differences between programs, and not providing information about longer-term posttransplant outcomes. We investigated the association of different posttransplant outcome assessments available to patients at the time of listing with subsequent posttransplant graft survival. The posttransplant assessments were from period prevalent, rather than incident, cohorts with more timely 1-, 3-, and 5-year follow-up and 6-, 12-, 18-, 24-, and 30-month cohort windows. The association of these assessments at listing with subsequent posttransplant graft survival included candidates listed between July 12, 2011, and December 15, 2015, who subsequently underwent transplant before December 31, 2018. The assessments with 1-year follow-up had uniformly weaker associations than the assessments with 3- and 5-year follow-up. The assessments with 5-year follow-up had the strongest association in kidney and liver transplantation. For kidney, liver, and lung transplantation, assessment windows of at least 18 months typically had the strongest associations with subsequent graft survival. Posttransplant assessments with 5-year follow-up and 18-30-month cohort windows are better than the current posttransplant assessment with 1-year follow-up, particularly at the time of listing.

Variability in donor organ offer acceptance and lung transplantation survival

BACKGROUND

Lung transplantation offers a survival benefit for patients with end-stage lung disease. When suitable donors are identified, centers must accept or decline the offer for a matched candidate on their waitlist. The degree to which variability in per-center offer acceptance practices impacts candidate survival is not established. The purpose of this study was to determine the degree of variability in per-center rates of lung transplantation offer acceptance and to ascertain the associated contribution to observed differences in per-center waitlist mortality.

METHODS

We performed a retrospective cohort study of candidates waitlisted for lung transplantation in the US using registry data. Logistic regression was fit to assess the relationship of offer acceptance with donor, candidate, and geographic factors. Listing center was evaluated as a fixed effect to determine the adjusted per-center acceptance rate. Competing risks analysis employing the Fine-Gray model was undertaken to establish the relationship between adjusted per-center acceptance and waitlist mortality.

RESULTS

Of 15,847 unique organ offers, 4,735 (29.9%) were accepted for first-ranked candidates. After adjustment for important covariates, transplant centers varied markedly in acceptance rate (9%-67%). Higher cumulative incidence of 1-year waitlist mortality was associated with lower acceptance rate. For every 10% increase in adjusted center acceptance rate, the risk of waitlist mortality decreased by 36.3% (sub-distribution hazard ratio 0.637; 95% confidence interval 0.592-0.685).

CONCLUSIONS

Variability in center-level behavior represents a modifiable risk factor for waitlist mortality in lung transplantation. Further intervention is needed to standardize center-level offer acceptance practices and minimize waitlist mortality.

Incorporating New Technology Into a Surgical Technique: The Learning Curve of a Single Surgeon's Stereo-Electroencephalography Experience

BACKGROUND

Technological improvements frequently outpace the publication of randomized, controlled trials in surgical patients. This makes the application of new surgical techniques difficult as surgeons solely use clinical experience to guide changes in their practice.

OBJECTIVE

To quantitatively examine the learning curve of incorporating new technology into a surgical technique and discuss the clinical significance of incorporating this new technology into daily practice. To identify areas of improvement for operative efficiency and safety.

METHODS

A retrospective observational study examining quantitative measures of operative efficiency and safety from 2009 to 2017 in 454 consecutive patients undergoing stereo-electroencephalography depth electrode implantations.

RESULTS

The transition to a new robotic technique significantly improved operative times (196 min [95% CI 173-219] vs 115 min [95% CI 111-118], P &lt; .0001). Cumulative sum (CUSUM) analysis demonstrated that mastery of the robotic technique took much longer than the frame-based technique (operative time peak at case 75 vs case 25, plateau of 150 vs 10 cases). Although hemorrhage rates using different vascular imaging techniques did not appear to differ using traditional statistical analysis (magnetic resonance imaging, MRI 22.3%, computed tomography angiography, CTA 17.9%, angiogram 18.1%, likelihood ratio χ2 = 4.84, P = .30), CUSUM analysis suggested MRI as the vascular imaging modality leading to higher hemorrhage and symptomatic hemorrhage rates at our center.

CONCLUSION

This experience demonstrates an improvement in operative efficiency through a series of changes made using clinical experience and intuition while transitioning to a completely new paradigm. CUSUM analysis identified potential areas for improvement in both operative efficiency and safety if used in a prospective manner.

Kidney transplant registries: How to optimise their utility?

The scientific Registries (RE) in renal transplantation (Tx) are very useful since they allow us to identify risk factors in this population and facilitate contrasting the information with other national and international registries, contributing to establishing strategies which improve outcomes in terms of survival. They constitute an organised and planned system that uses observational methods and standardised systematic processes, including adjusted risk models, to essentially evaluate survival outcomes. The scientific RE are complemented with clinical trials providing scientific evidence, but inexcusably need adequate statistical analysis to generate reliable clinical data that contribute to optimising the prognosis of the transplant population. In addition, scientific RE provide valuable information on the performance of Tx programmes and help generate prognostic indexes, which could contribute to improving survival. Under these prerequisites, this review will assess the following aspects related to the scientific RE in the Tx: 1) the concept and importance of implementing RE in Tx; 2) the measures that are needed for the correct execution of the scientific RE; 3) the benefits, quality and limitations of RE; 4) the statistical tools for the adequate analysis of survival; and 5) utility of RE in the evaluation of performance, quality and surveillance of transplant programmes and the generation of comorbidity índices.

Scientific Registry of Transplant Recipients program-specific reports: where we have been and where we are going

PURPOSE OF REVIEW

Reporting provider data on quality to patients and the general public is increasingly common in healthcare. Reporting outcomes in solid organ transplantation has always been controversial and deserves careful consideration to ensure optimal results.

RECENT FINDINGS

As mandated by Federal law, the Scientific Registry of Transplant Recipients publishes program-specific reports on transplant candidates, recipients, donors, and transplant outcomes every 6 months. Recent changes designed to make the results more easily understood by patients and the general public have been well received by patients and controversial among providers. In particular, outcomes are now reported using a five-tier system that distinguishes program results better than the old three-tier system, in which almost all programs were reported "as expected." Metrics that reflect access to transplant are also reported, including transplant rate and survival on the waiting list. Possible measures of longer term outcomes and program rates of accepting organs for transplant are being explored.

SUMMARY

Providing detailed information regarding transplant program practices and outcomes in ways that patients and the general public can understand remains a major focus of the Scientific Registry of Transplant Recipients. Efforts to improve data collection and metrics reported are ongoing.

Heart and lung organ offer acceptance practices of transplant programs are associated with waitlist mortality and organ yield

Variation in heart and lung offer acceptance practices may affect numbers of transplanted organs and create variability in waitlist mortality. To investigate these issues, offer acceptance ratios, or adjusted odds ratios, for heart and lung transplant programs individually and for all programs within donation service areas (DSAs) were estimated using offers from donors recovered July 1, 2016, and June 30, 2017. Logistic regressions estimated the association of DSA-level offer acceptance ratios with donor yield and local placement of organs recovered in the DSA. Competing risk methodology estimated the association of program-level offer acceptance ratios with incidence and rate of waitlist removals due to death or becoming too sick to undergo transplant. Higher DSA-level offer acceptance was associated with higher yield (odds ratios [ORs]: lung, _1.04 1.11_1.19 ; heart, _1.09 1.21_1.35 ) and more local placement of transplanted organs (ORs: lung, _1.01 1.12_1.24 ; heart, _1.47 1.69_1.93 ). Higher program-level offer acceptance was associated with lower incidence of waitlist removal due to death or becoming too sick to undergo transplant (hazard ratios [HRs]: heart, _0.80 0.86_0.93 ; lung, _0.67 0.75_0.83 ), but not with rate of waitlist removal (HRs: heart, _0.91 0.98_1.06 ; lung, _0.89 0.99_1.10 ). Heart and lung offer acceptance practices affected numbers of transplanted organs and contributed to program-level variability in the probability of waitlist mortality.

A simple signaling rule for variable life-adjusted display derived from an equivalent risk-adjusted CUSUM chart

The variable life-adjusted display (VLAD) is the first risk-adjusted graphical procedure proposed in the literature for monitoring the performance of a surgeon. It displays the cumulative sum of expected minus observed deaths. It has since become highly popular because the statistic plotted is easy to understand. But it is also easy to misinterpret a surgeon's performance by utilizing the VLAD, potentially leading to grave consequences. The problem of misinterpretation is essentially caused by the variance of the VLAD's statistic that increases with sample size. In order for the VLAD to be truly useful, a simple signaling rule is desperately needed. Various forms of signaling rules have been developed, but they are usually quite complicated. Without signaling rules, making inferences using the VLAD alone is difficult if not misleading. In this paper, we establish an equivalence between a VLAD with V-mask and a risk-adjusted cumulative sum (RA-CUSUM) chart based on the difference between the estimated probability of death and surgical outcome. Average run length analysis based on simulation shows that this particular RA-CUSUM chart has similar performance as compared to the established RA-CUSUM chart based on the log-likelihood ratio statistic obtained by testing the odds ratio of death. We provide a simple design procedure for determining the V-mask parameters based on a resampling approach. Resampling from a real data set ensures that these parameters can be estimated appropriately. Finally, we illustrate the monitoring of a real surgeon's performance using VLAD with V-mask.

Evaluation of a national surveillance system for mortality alerts: a mixed-methods study

Background

Since 2007, Imperial College London has generated monthly mortality alerts, based on statistical process control charts and using routinely collected hospital administrative data, for all English acute NHS hospital trusts. The impact of this system has not yet been studied.

Objectives

To improve understanding of mortality alerts and evaluate their impact as an intervention to reduce mortality.

Design

Mixed methods.

Setting

English NHS acute hospital trusts.

Participants

Eleven trusts were included in the case study. The survey involved 78 alerting trusts.

Main outcome measures

Relative risk of mortality and perceived efficacy of the alerting system.

Data sources

Hospital Episodes Statistics, published indicators on quality and safety, Care Quality Commission (CQC) reports, interviews and documentary evidence from case studies, and a national evaluative survey.

Methods

Descriptive analysis of alerts; association with other measures of quality; associated change in mortality using an interrupted time series approach; in-depth qualitative case studies of institutional response to alerts; and a national cross-sectional evaluative survey administered to describe the organisational structure for mortality governance and perceptions of efficacy of alerts.

Results

A total of 690 mortality alerts generated between April 2007 and December 2014. CQC pursued 75% (154/206) of alerts sent between 2011 and 2013. Patient care was cited as a factor in 70% of all investigations and in 89% of sepsis alerts. Alerts were associated with indicators on bed occupancy, hospital mortality, staffing, financial status, and patient and trainee satisfaction. On average, the risk of death fell by 58% during the 9-month lag following an alert, levelling afterwards and reaching an expected risk within 18 months of the alert. Acute myocardial infarction (AMI) and sepsis alerts instigated institutional responses across all the case study sites, although most sites were undertaking some parallel activities at a more general level to address known problems in care in these and other areas. Responses included case note review and coding improvements, changes in patient pathways, changes in diagnosis of sepsis and AMI, staff training in case note write-up and coding, greater transparency in patient deterioration, and infrastructure changes. Survey data revealed that 86% of responding trusts had a dedicated trust-level lead for mortality reduction and 92% had a dedicated trust-level mortality group or committee in place. Trusts reported that mortality reduction was a high priority and that there was strong senior leadership support for mortality monitoring. The weakest areas reported concerned the accuracy of coding, the quality of specialty-level mortality data and understanding trends in specialty-level mortality data.

Limitations

Owing to the correlational nature of our analysis, we could not ascribe a causal link between mortality alerts and reductions in mortality. The complexity of the institutional context and behaviour hindered our capacity to attribute locally reported changes specifically to the effects of the alerts rather than to ongoing institutional strategy.

Conclusions

The mortality alert surveillance system reflects aspects of quality care and is valued by trusts. Alerts were considered a useful focus for identifying problems and implementing interventions around mortality.

Future work

A further analysis of site visits and survey material, the application of evaluative framework to other interventions, a blinded case note review and the dissemination of findings.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Influence of kidney offer acceptance behavior on metrics of allocation efficiency

We investigated associations of deceased donor kidney offer acceptance with likelihood of the kidney being discarded, cold ischemia time at transplant (CIT), and likelihood of the kidney being exported outside the donation service area (DSA). We used kidney offers from donors in the Scientific Registry of Transplant Recipients July 1, 2015-June 30, 2016, and a stratified logistic regression to estimate odds ratios of acceptance for candidates wait-listed in a DSA. We estimated associations between these ratios and likelihood of discard or export and CIT at transplant. Approximately 0.50 kidneys were discarded per donor; lower DSA-specific offer acceptance ratios were associated with more discards (R=-0.20; P=0.006). For a median donor, the DSA with the highest acceptance ratio would place 0.12 more kidneys per donor than the DSA with the lowest ratio. Low acceptance ratios were associated with higher CIT (R=-0.23; P<0.001). For the median donor, CIT was 2.9 hours shorter for the DSA with the highest versus lowest acceptance ratio. Low acceptance ratios were associated with more exports (R=-0.43; P<0.001); the probability was 15% higher for a median donor in the DSA with the lowest versus highest acceptance ratio. Improving lower-than-expected offer acceptance would likely reduce discards, CIT, and exports.

Measuring transplant center performance: The goals are not controversial but the methods and consequences can be

PURPOSE OF REVIEW

Risks of regulatory scrutiny has generated widespread concern about increasingly risk averse transplant center behaviors regarding both donor and candidate acceptance patterns. To address potential unintended consequences threatening access to care, we discuss recent changes in regulatory metrics and potential improvements in quality oversight of transplant centers.

RECENT FINDINGS

Despite many recent changes to one-year patient and graft survival regulatory criteria, the capacity to accurately identify true underperforming centers and avoiding false positive flagging remains an area of great concern. Numerous studies have demonstrated restrictions in transplant volume and access following transplant center flagging.

SUMMARY

Current regulatory criteria are limited in their capacity to accurately identify poorly performing centers and potentially encourage risk-averse behavior by transplant centers. Efforts to address these concerns should focus on (1) improving risk-adjustment models with better data which captures the acuity of candidate and donor risk, (2) reconsidering primary outcomes measured to assess comprehensive transplant center performance, (3) improving education to address rational or perceived disincentives, and (4) using data more effectively to share best practices.

Quality of pancreatic transplant program assessment using a risk-adjusted cumulative sum chart: experience from a single, small center

BACKGROUND

The high rate of complication after pancreas transplantation not only had an impact on recipient quality of life and survival but also had significant financial implications. Thus, monitoring transplant center performance was crucial to indentifying changes in clinical practice that result in quality deterioration.

OBJECTIVE

To evaluate retrospectively the quality of the single, small pancreatic transplant program and to establish prospective monitoring of the center using risk-adjusted cumulative sum (CUSUM).

METHODOLOGY

From 1988 to 2014, 119 simultaneous pancreas and the kidney transplantations (SPKTx) were performed. The program was divided into 3 eras, based on surgical technique and immunosuppression. Analyses of the 15 fatal outcomes due to complication from pancreatic graft were performed. The risk model was developed using multivariable logistic regression analysis based on retrospective data of 112 SPKTx recipients. The risk-adjusted 1-sided CUSUM chart was plotted for retrospective and prospective events. The upper control limit was set to 2.

RESULTS

There were 2 main causes of death: multiorgan failure (73.3%; 11/15) and septic hemorrhage (26.7%; 4/15). Quality analysis using the CUSUM chart revealed that the process was not homogeneous; however, no significant signal of program deterioration was obtained and the performance of the whole program was within the settled control limit.

CONCLUSION

For a single pancreatic transplant center. The risk-adjusted CUSUM chart was a useful tool for quality program assessment. It could support decision making during traditional surgical morbidity and mortality conferences. For small transplant centers, increasing the sensitivity of the CUSUM method by lowering the upper control limit should be considered. However, an individual assessment approach of the for particular centers is recommended.