1
|
Blackburn KW, Cooper LE, Bafford AC, Hu Y, Brown RF. Using risk-adjusted cumulative sum to evaluate surgeon, divisional, and institutional outcomes-a feasibility study. Surgery 2024; 175:1554-1561. [PMID: 38523020 DOI: 10.1016/j.surg.2024.01.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 12/07/2023] [Accepted: 01/24/2024] [Indexed: 03/26/2024]
Abstract
BACKGROUND Few objective, real-time measurements of surgeon performance exist. The risk-adjusted cumulative sum is a novel method that can track surgeon-level outcomes on a continuous basis. The objective of this study was to demonstrate the feasibility of using risk-adjusted cumulative sum to monitor outcomes after colorectal operations and identify clinically relevant performance variations. METHODS The National Surgical Quality Improvement Program was queried to obtain patient-level data for 1,603 colorectal operations at a high-volume center from 2011 to 2020. For each case, expected risks of morbidity, mortality, reoperation, readmission, and prolonged length of stay were estimated using the National Surgical Quality Improvement Program risk calculator. Risk-adjusted cumulative sum curves were generated to signal observed-to-expected odds ratios of 1.5 (poor performance) and 0.5 (exceptional performance). Control limits were set based on a false positive rate of 5% (α = 0.05). RESULTS The cohort included data on 7 surgeons (those with more than 20 cases in the study period). Institutional observed versus expected outcomes were the following: morbidity 12.5% (vs 15.0%), mortality 2.5% (vs 2.0%), prolonged length of stay 19.7% (vs 19.1%), reoperation 11.1% (vs 11.3%), and 30-day readmission 6.1% (vs 4.8%). Risk-adjusted cumulative sum accurately demonstrated within- and between-surgeon performance variations across these metrics and proved effective when considering division-level data. CONCLUSION Risk-adjusted cumulative sum adjusts for patient-level risk factors to provide real-time data on surgeon-specific outcomes. This approach enables prompt identification of performance outliers and can contribute to quality assurance, root-cause analysis, and incentivization not only at the surgeon level but at divisional and institutional levels as well.
Collapse
Affiliation(s)
- Kyle W Blackburn
- School of Medicine, Baylor College of Medicine, Waco, TX. https://twitter.com/KyleWBlackburn
| | - Laura E Cooper
- Department of Surgery, University of Maryland School of Medicine, Baltimore, MD
| | | | - Yinin Hu
- Department of Surgery, University of Maryland School of Medicine, Baltimore, MD
| | - Rebecca F Brown
- Department of Surgery, University of Maryland School of Medicine, Baltimore, MD.
| |
Collapse
|
2
|
Crespo MM, Samra M, Korsun A, Butler L, Byford H, Tietjen A, Stillion L, Ohler L, Mehta S. Collaborative leadership in transplantation: Blending clinical, business, and regulatory roles. Clin Transplant 2023; 37:e15126. [PMID: 37747969 DOI: 10.1111/ctr.15126] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2023] [Revised: 08/16/2023] [Accepted: 09/05/2023] [Indexed: 09/27/2023]
Abstract
Transplantation is a high-risk, high-cost treatment for end-stage diseases and is the most strictly regulated area of healthcare in the United States. Thus, achieving success for patients and the program requires skillful and collaborative leadership. Various factors, such as outcomes, volume, and financial health, may measure the success of a transplant program. Strong collaboration between clinical and administrative leaders is key to achieving and maintaining success in those three categories. Clinical leaders of adult programs, such as medical and surgical directors, bear the primary responsibility for a program's volume, outcomes, and patient safety, while administrative directors are focused on business intelligence and regulatory compliance. This paper aims to provide readers with insights into the critical role of collaborative leadership in running a successful program, with a focus on clinical, business, and regulatory perspectives.
Collapse
Affiliation(s)
- Maria M Crespo
- Division of Pulmonary, Allergy, and Critical Care, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Manpreet Samra
- Kidney Transplant Program, Edward Hines Jr VA Hospital, Hines, Illinois, USA
| | | | | | | | - Andrea Tietjen
- Cooperman Barnabas Medical Center, Livingston, New Jersey, USA
| | | | - Linda Ohler
- George Washington University, Washington, District of Columbia, USA
| | - Shikha Mehta
- Kidney Transplant Program, University of Alabama, Birmingham, Alabama, USA
| |
Collapse
|
3
|
Blackburn KW, Turrentine FE, Schirmer BD, Hallowell PT, Kubicki NS, Hu Y, Kligman MD. Monitoring performance in laparoscopic gastric bypass surgery using risk-adjusted cumulative sum at 2 high-volume centers. Surg Obes Relat Dis 2023; 19:1049-1057. [PMID: 36931965 DOI: 10.1016/j.soard.2023.02.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2022] [Revised: 01/09/2023] [Accepted: 02/04/2023] [Indexed: 02/17/2023]
Abstract
BACKGROUND Traditional surgical outcomes are measured retrospectively and intermittently, limiting opportunities for early intervention. OBJECTIVES The objective of this study was to use risk-adjusted cumulative sum (RA-CUSUM) to track perioperative surgical outcomes for laparoscopic gastric bypass. We hypothesized that RA-CUSUM could identify performance variations between surgeons. SETTING Two mid-Atlantic quaternary care academic centers. METHODS Patient-level data from the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBSAQIP) were abstracted for laparoscopic gastric bypasses performed by 3 surgeons at 2 high-volume centers from 2014 to 2021. Estimated probabilities of serious complications, reoperation, and readmission were derived from the MBSAQIP risk calculator. RA-CUSUM curves were generated to signal observed-to-expected odds ratios (ORs) of 1.5 (poor performance) and .5 (superior performance). Control limits were set based on a false positive rate of 5% (α = .05). RESULTS We included 1192 patients: Surgeon A = 767, Surgeon B = 188, and Surgeon C = 237. Overall rates of serious complications, 30-day reoperations, and 30-day readmissions were 3.9%, 2.5%, and 5.2% respectively, with expected rates of 4.7%, 2.2%, and 5.8%. RA-CUSUM signaled lower-than-expected (OR < .5) rates of readmission and serious complication in Surgeon A, and higher-than-expected (OR > 1.5) readmission rate in Surgeon C. Surgeon A further demonstrated an early period of higher-than-expected (OR > 1.5) reoperation rate before April 2015, followed by superior performance thereafter (OR < .5). Surgeon B's performance generally reflected expected standards throughout the study period. CONCLUSIONS RA-CUSUM adjusts for clinical risk factors and identifies performance outliers in real-time. This approach to analyzing surgical outcomes is applicable to quality improvement, root-cause analysis, and surgeon incentivization.
Collapse
Affiliation(s)
- Kyle W Blackburn
- Division of General and Oncologic Surgery, Department of Surgery, University of Maryland School of Medicine, Baltimore, Maryland
| | - Florence E Turrentine
- Division of General Surgery, Department of Surgery, University of Virginia School of Medicine, Charlottesville, Virginia
| | - Bruce D Schirmer
- Division of General Surgery, Department of Surgery, University of Virginia School of Medicine, Charlottesville, Virginia
| | - Peter T Hallowell
- Division of General Surgery, Department of Surgery, University of Virginia School of Medicine, Charlottesville, Virginia
| | - Natalia S Kubicki
- Division of General and Oncologic Surgery, Department of Surgery, University of Maryland School of Medicine, Baltimore, Maryland
| | - Yinin Hu
- Division of General and Oncologic Surgery, Department of Surgery, University of Maryland School of Medicine, Baltimore, Maryland
| | - Mark D Kligman
- Division of General and Oncologic Surgery, Department of Surgery, University of Maryland School of Medicine, Baltimore, Maryland.
| |
Collapse
|
4
|
Snyder JJ, Schaffhausen CR, Hart A, Axelrod DA, Dils D, Formica RN, Gaber AO, Hunt HF, Jones J, Mohan S, Patzer RE, Pinney SP, Ratner LE, Slaker D, Stewart D, Stewart ZA, Van Slyck S, Kasiske BL, Hirose R, Israni AK. Stakeholders' Perspectives on Transplant Metrics: The 2022 Scientific Registry of Transplant Recipients' Consensus Conference. Am J Transplant 2023:S1600-6135(23)00355-6. [PMID: 36958628 DOI: 10.1016/j.ajt.2023.03.012] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2022] [Revised: 03/10/2023] [Accepted: 03/14/2023] [Indexed: 03/25/2023]
Abstract
In July 2022, the Scientific Registry of Transplant Recipients (SRTR) hosted an innovative, multistakeholder consensus conference to identify information and metrics desired by stakeholders in the transplantation system, including patients, living donors, caregivers, deceased donor family members, transplant professionals, organ procurement organization professionals, payers, and regulators. Crucially, patients, caregivers, living donors, and deceased donor family members were included in all aspects of this conference, including serving on the planning committee, participating in preconference focus groups and learning sessions, speaking at the conference, moderating conference sessions and breakout groups, and shaping the conclusions. Patients constituted 24% of the meeting participants. In this report, we document the proceedings and enumerate 160 recommendations, 10 of which have been highly prioritized. SRTR will use the recommendations to develop new presentations of information and metrics requested by stakeholders to support informed decision-making.
Collapse
Affiliation(s)
- Jon J Snyder
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA; Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA; Department of Epidemiology and Community Health, School of Public Health, University of Minnesota, Minneapolis, Minnesota, USA.
| | - Cory R Schaffhausen
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA; Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA
| | - Allyson Hart
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA; Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA
| | - David A Axelrod
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA; University of Iowa Transplant Institute, Iowa City, Iowa, USA
| | - Dorrie Dils
- Gift of Life Michigan, Ann Arbor, Michigan, USA
| | | | - A Osama Gaber
- Department of Surgery, Houston Methodist Hospital, J.C. Walter Jr. Transplant Center, Houston, Texas, USA
| | | | - Jennifer Jones
- American Association of Kidney Patients, Kidney Transplant Recipient
| | - Sumit Mohan
- Department of Medicine, Vagelos College of Physicians and Surgeons, Columbia University, New York, New York, USA; Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, New York, USA
| | - Rachel E Patzer
- Department of Surgery, Division of Transplantation, Emory University School of Medicine, Atlanta, Georgia, USA; Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, Georgia, USA; Department of Medicine, Emory University School of Medicine, Atlanta, Georgia, USA
| | - Sean P Pinney
- Department of Medicine, The University of Chicago, Chicago, IL, USA
| | - Lloyd E Ratner
- Department of Surgery, Columbia University, New York, New York
| | - Dirk Slaker
- Optum Health Solutions, Eden Prairie, Minnesota, USA
| | - Darren Stewart
- Department of Surgery, NYU Langone Health, New York, New York, USA
| | - Zoe A Stewart
- Department of Surgery, NYU Langone Health, New York, New York, USA
| | | | - Bertram L Kasiske
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA; Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA
| | - Ryutaro Hirose
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA; Department of Surgery, University of California San Francisco, San Francisco, California, USA
| | - Ajay K Israni
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA; Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA; Department of Epidemiology and Community Health, School of Public Health, University of Minnesota, Minneapolis, Minnesota, USA
| |
Collapse
|
5
|
Lam HD, Schaapherder AF, Alwayn IP, Nijboer WN, Tushuizen ME, Hemke AC, Baranski A, Pas SLVD. Quality assessment of donor liver procurement surgery using an unadjusted CUSUM prediction model. A practical nationwide evaluation. Clin Transplant 2023; 37:e14940. [PMID: 36796105 DOI: 10.1111/ctr.14940] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 01/13/2023] [Accepted: 02/09/2023] [Indexed: 02/18/2023]
Abstract
BACKGROUND The aim of this study was to analyze the value of the unadjusted CUSUM graph of liver surgical injury and discard rates in organ procurement in the Netherlands. METHODS Unadjusted CUSUM graphs were plotted for surgical injury (C event) and discard rate (C2 event) from procured livers accepted for transplantation for each local procurement team compared with the total national cohort. The average incidence for each outcome was used as benchmark based on procurement quality forms (Sep 2010-Oct 2018). The data from the five Dutch procuring teams were blind-coded. RESULTS The C and C2 event rate were 17% and 1.9%, respectively (n = 1265). A total of 12 CUSUM charts were plotted for the national cohort and the five local teams. National CUSUM charts showed an overlapping "alarm signal." This overlapping signal for both C and C2, albeit a different time period, was only found in one local team. The other CUSUM alarm signal went off for two separate local teams, but only for C events or C2 events respectively, and at different points in time. The other remaining CUSUM charts showed no alarm signaling. CONCLUSION The unadjusted CUSUM chart is a simple and effective monitoring tool in following performance quality of organ procurement for liver transplantation. Both national and local recorded CUSUMs are useful to see the implication of national and local effects on organ procurement injury. Both procurement injury and organ discard are equally important in this analysis and need to be separately CUSUM charted.
Collapse
Affiliation(s)
- Hwai-Ding Lam
- Department of Transplantation Surgery, LUMC Transplant Center, Leiden University Medical Center, Leiden, The Netherlands
| | - Alexander F Schaapherder
- Department of Transplantation Surgery, LUMC Transplant Center, Leiden University Medical Center, Leiden, The Netherlands
| | - Ian Pj Alwayn
- Department of Transplantation Surgery, LUMC Transplant Center, Leiden University Medical Center, Leiden, The Netherlands
| | - Willemijn N Nijboer
- Department of Transplantation Surgery, LUMC Transplant Center, Leiden University Medical Center, Leiden, The Netherlands.,Landelijk Overleg Regionale Uitname teams, Leiden, The Netherlands
| | - Maarten E Tushuizen
- Department of Gastroenterology & Hepatology, LUMC Transplant Center, Leiden University Medical Center, Leiden, The Netherlands
| | - Aline C Hemke
- Dutch Transplantation Foundation, Leiden, The Netherlands
| | - Andrzej Baranski
- Department of Transplantation Surgery, LUMC Transplant Center, Leiden University Medical Center, Leiden, The Netherlands
| | - Stéphanie L Van der Pas
- Department of Epidemiology and Data Science, Amsterdam UMC, Amsterdam, The Netherlands.,Amsterdam Public Health, Methodology, Amsterdam, The Netherlands
| |
Collapse
|
6
|
Abstract
BACKGROUND Quality in kidney transplantation is measured using 1-year patient and graft survival. Because 1-year patient and graft survival exceed 95%, this metric fails to measure a spectrum of quality. Textbook outcomes (TO) are a composite quality metric offering greater depth and resolution. We studied TO after living donor (LD) and deceased donor (DD) kidney transplantation. STUDY DESIGN United Network for Organ Sharing data for 69,165 transplant recipients between 2013 and 2017 were analyzed. TO was defined as patient and graft survival of 1 year or greater, 1-year glomerular filtration rate of greater than 40 mL/min, absence of delayed graft function, length of stay of 5 days or less, no readmissions during the first 6 months, and no episodes of rejection during the first year after transplantation. Bivariate analysis identified characteristics associated with TO, and covariates were incorporated into multivariable models. Five-year conditional survival was measured, and center TO rates were corrected for case complexity to allow center-level comparisons. RESULTS The national average TO rates were 54.1% and 31.7% for LD and DD transplant recipients. The hazard ratio for death at 5 years for recipients who did not experience TO was 1.92 (95% CI 1.68 to 2.18, p ≤ 0.0001) for LD transplant recipients and 2.08 (95% CI 1.93 to 2.24, p ≤ 0.0001) for DD transplant recipients. Center-level comparisons identify 18% and 24% of centers under-performing in LD and DD transplantation. High rates of TO do not correlate with transplantation center volume. CONCLUSION Kidney transplant recipients who experience TO have superior long-term survival. Textbook outcomes add value to the current standards of 1-year patient and graft survival.
Collapse
|
7
|
Permission to Reach: In Search of New Quality Metrics in Transplantation. J Am Coll Surg 2022; 235:641-642. [PMID: 35816727 DOI: 10.1097/xcs.0000000000000294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
8
|
Israni AK, Hirose R, Segev DL, Hart A, Schaffhausen CR, Axelrod DA, Kasiske BL, Snyder JJ. Toward continuous improvement of Scientific Registry of Transplant Recipients performance reporting: Advances following 2012 consensus conference and future consensus building for 2022 consensus conference. Clin Transplant 2022; 36:e14716. [PMID: 35598080 DOI: 10.1111/ctr.14716] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Revised: 04/27/2022] [Accepted: 05/16/2022] [Indexed: 11/29/2022]
Abstract
The Scientific Registry of Transplant Recipients (SRTR) held a consensus conference in 2012 that examined methods used by SRTR for constructing performance metrics and made recommendations on how to improve program-specific reports. That consensus conference provided 25 recommendations categorized as follows: statistical methods, risk adjustment, and outcomes and data. During the subsequent decade, SRTR has implemented most of these recommendations; these are described in this article along with plans for another consensus conference in 2022. With the present article, SRTR aims to create transparency in the field of transplant metrics and guide discussion in the planning of the next consensus conference in 2022. The new conference will revisit the previous topics and have a broader focus to improve the metrics and information that SRTR provides. Readers can provide feedback on topics to be discussed at the next consensus conference as early as possible, by emailing srtr@srtr.org with the subject line "Task 5 Public Comment." This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Ajay K Israni
- Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA.,Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA.,Department of Epidemiology and Community Health, University of Minnesota, Minneapolis, Minnesota, USA
| | - Ryutaro Hirose
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA.,Department of Surgery, University of California San Francisco, San Francisco, California, USA
| | - Dorry L Segev
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA.,Department of Surgery, Johns Hopkins University, Baltimore, Maryland, USA
| | - Allyson Hart
- Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA.,Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA
| | - Cory R Schaffhausen
- Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA.,Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA
| | - David A Axelrod
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA.,Department of Surgery, University of Iowa, Iowa City, Iowa, USA
| | - Bertram L Kasiske
- Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA.,Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA
| | - Jon J Snyder
- Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA.,Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA.,Department of Epidemiology and Community Health, University of Minnesota, Minneapolis, Minnesota, USA
| |
Collapse
|
9
|
Massarweh NN, Chen VW, Rosen T, Dong Y, Richardson PA, Axelrod DA, Harris AHS, Wilson MA, Petersen LA. Comparative Effectiveness of Risk-adjusted Cumulative Sum and Periodic Evaluation for Monitoring Hospital Perioperative Mortality. Med Care 2021; 59:639-645. [PMID: 33900272 DOI: 10.1097/mlr.0000000000001559] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
BACKGROUND National surgical quality improvement (QI) programs use periodic, risk-adjusted evaluation to identify hospitals with higher than expected perioperative mortality. Rapid, accurate identification of poorly performing hospitals is critical for avoiding potentially preventable mortality and represents an opportunity to enhance QI efforts. METHODS Hospital-level analysis using Veterans Affairs (VA) Surgical Quality Improvement Program data (2011-2016) to compare identification of hospitals with excess, risk-adjusted 30-day mortality using observed-to-expected (O-E) ratios (ie, current gold standard) and cumulative sum (CUSUM) with V-mask. Various V-mask slopes and radii were evaluated-slope of 2.5 and radius of 1.0 was used as the base case. RESULTS Hospitals identified by CUSUM and quarterly O-E were identified midway into a quarter [median 47 days; interquartile range (IQR): 24-61 days before quarter end] translating to a median of 129 (IQR: 60-187) surgical cases and 368 (IQR: 145-681) postoperative inpatient days occurring after a CUSUM signal, but before the quarter end. At hospitals identified by CUSUM but not O-E, a median of 2 deaths within a median of 5 days triggered a signal. In some cases, these clusters extended beyond CUSUM identification date with as many as 8 deaths undetected using O-E. Sensitivity and negative predictive values for CUSUM relative to O-E were 71.9% (95% confidence interval: 66.2%-77.1%) and 95.5% (94.4%-96.4%), respectively. CONCLUSIONS CUSUM evaluation identifies hospitals with clusters of mortality in excess of expected more rapidly than periodic analysis. CUSUM represents an analytic tool national QI programs could utilize to provide participating hospitals with data that could facilitate more proactive implementation of local interventions to help reduce potentially avoidable perioperative mortality.
Collapse
Affiliation(s)
- Nader N Massarweh
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey VA Medical Center
- Michael E DeBakey Department of Surgery, Baylor College of Medicine
- Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, TX
| | - Vivi W Chen
- Michael E DeBakey Department of Surgery, Baylor College of Medicine
| | - Tracey Rosen
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey VA Medical Center
| | - Yongquan Dong
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey VA Medical Center
| | - Peter A Richardson
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey VA Medical Center
- Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, TX
| | | | - Alex H S Harris
- Veterans Affairs Health Services Research and Development Center for Innovation to Implementation, Palo Alto Veterans Affairs Health Care System, Menlo Park, CA
- Department of Surgery, Stanford University, Stanford, CA
| | - Mark A Wilson
- National Surgery Office, Veterans Health Administration, Washington, DC
- Department of Surgery, University of Pittsburgh, Pittsburgh, PA
| | - Laura A Petersen
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey VA Medical Center
- Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, TX
| |
Collapse
|
10
|
Batra RK, Mulligan DC. Current status: meeting the regulatory goals of your liver transplant program. Curr Opin Organ Transplant 2021; 26:146-151. [PMID: 33650996 DOI: 10.1097/mot.0000000000000869] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
PURPOSE OF REVIEW The regulatory framework set by the Organ Procurement and Transplantation Network (OPTN) and Center for Medicare and Medicaid Services (CMS) for practice of liver transplantation in US is periodically updated and risk adjusted. Therefore, it is prudent for transplant centers to know the rules of engagement as it pertains to their practice. RECENT FINDINGS OPTN besides providing the regulatory oversight for safe and continued practice of transplant centers, provides necessary tools like: advanced statistical models and technological platforms to aid, and guide transplant centers including the necessary safeguards for high-quality transplant care.CMS regulations although had different thresholds to flag underperformance, often covered common grounds similar to the OPTN, therefore considered duplicative and unnecessary. But with much deliberation and consideration CMS undertook a major overhaul to the final rule for re-approval applications, a giant leap in the positive direction for transplant innovation and growth. SUMMARY The duplicative regulatory framework of OPTN and CMS has although achieved the goal of improving 1-year patient outcomes, it has proven costly in terms of slowing innovation, increasing organ discard and stunting growth of transplant volume. But the new updates in effect and also in the pipeline are a long-awaited opportunity for waiting transplant patients.
Collapse
Affiliation(s)
- Ramesh K Batra
- Yale University School of Medicine, Director, Liver Transplant Program
| | - David C Mulligan
- Yale University School of Medicine, Chair, Division of Transplantation and Immunology, Yale New Haven Hospital, New Haven, Connecticut, USA
| |
Collapse
|
11
|
Nicolas JD, Huang R, Teitelbaum EN, Bilimoria KY, Hu YY. Constructing Learning Curves to Benchmark Operative Performance of General Surgery Residents Against a National Cohort of Peers. JOURNAL OF SURGICAL EDUCATION 2020; 77:e94-e102. [PMID: 33109492 DOI: 10.1016/j.jsurg.2020.10.001] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/17/2020] [Revised: 08/27/2020] [Accepted: 10/02/2020] [Indexed: 06/11/2023]
Abstract
OBJECTIVE No method or data exist to allow surgical trainees or their programs to contextualize their technical progress. The objective of this study was to create peer benchmarks for Cumulative Sum (CUSUM) charts based upon operative evaluations from a national cohort of general surgery residents. DESIGN, SETTING, PARTICIPANTS In 2016-2018, faculty from 26 general surgery residency programs nationwide rated 328 residents' operative performance on a case-by-case basis using a validated 5-point Likert scale. An individual case was considered a "misstep" if scoring below the national median score for that procedure in that postgraduate year (PGY). We constructed 2-sided observed-expected CUSUM charts to capture each resident's cumulative performance over time relative to the national medians. Upper (failure) and lower (positive outlier) benchmarks were established based on the PGY-specific 75th percentile and median misstep rates; consistent/repeated missteps are reflected by crossing of the upper boundary. Procedures with ≤10 observations and residents who were evaluated <10 times for each PGY were excluded. RESULTS Around 8,161 evaluations on 76 procedure types were analyzed. The individual misstep rate was lowest among PGY-3s at 13.3% and highest among PGY-4s at 28.6%. No interns had curves that crossed the failure boundary. 8.7% of PGY-2s and 8.9% of PGY-3s finished the year past the failure boundary. PGY-2s had the most positive outliers, with 28.3% of them demonstrating an outlying success performance beyond the lower boundary for at least once. PGY-5s most frequently failed, with 16.7% ever crossing the upper boundary and 11.1% remaining above it at graduation. CONCLUSIONS CUSUM is a valid statistical approach for benchmarking individual residents' operative performance against national peers as they progress through the year in real-time. With further validation, CUSUM could be used to set progression and/or graduation standards and objectively identify residents who might benefit from remediation.
Collapse
Affiliation(s)
- Joseph D Nicolas
- Surgical Outcomes and Quality Improvement Center (SOQIC), Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Reiping Huang
- Surgical Outcomes and Quality Improvement Center (SOQIC), Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Ezra N Teitelbaum
- Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Karl Y Bilimoria
- Surgical Outcomes and Quality Improvement Center (SOQIC), Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois; Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Yue-Yung Hu
- Surgical Outcomes and Quality Improvement Center (SOQIC), Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois; Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois; Division of Pediatric Surgery, Ann and Robert H. Lurie Children's Hospital, Chicago, Illinois.
| |
Collapse
|
12
|
Wey A, Hart A, Salkowski N, Skeans M, Kasiske BL, Israni AK, Snyder JJ. Posttransplant outcome assessments at listing: Long-term outcomes are more important than short-term outcomes. Am J Transplant 2020; 20:2813-2821. [PMID: 32282985 DOI: 10.1111/ajt.15911] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2019] [Revised: 03/05/2020] [Accepted: 03/29/2020] [Indexed: 01/25/2023]
Abstract
Posttransplant outcome assessments are publicly reported for patient and regulatory use. However, the currently reported 1-year posttransplant graft survival assessments are commonly criticized for not identifying clinically meaningful differences between programs, and not providing information about longer-term posttransplant outcomes. We investigated the association of different posttransplant outcome assessments available to patients at the time of listing with subsequent posttransplant graft survival. The posttransplant assessments were from period prevalent, rather than incident, cohorts with more timely 1-, 3-, and 5-year follow-up and 6-, 12-, 18-, 24-, and 30-month cohort windows. The association of these assessments at listing with subsequent posttransplant graft survival included candidates listed between July 12, 2011, and December 15, 2015, who subsequently underwent transplant before December 31, 2018. The assessments with 1-year follow-up had uniformly weaker associations than the assessments with 3- and 5-year follow-up. The assessments with 5-year follow-up had the strongest association in kidney and liver transplantation. For kidney, liver, and lung transplantation, assessment windows of at least 18 months typically had the strongest associations with subsequent graft survival. Posttransplant assessments with 5-year follow-up and 18-30-month cohort windows are better than the current posttransplant assessment with 1-year follow-up, particularly at the time of listing.
Collapse
Affiliation(s)
- Andrew Wey
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA
| | - Allyson Hart
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA.,Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA
| | - Nicholas Salkowski
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA
| | - Melissa Skeans
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA
| | - Bertram L Kasiske
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA.,Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA
| | - Ajay K Israni
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA.,Department of Medicine, Hennepin Healthcare, University of Minnesota, Minneapolis, Minnesota, USA.,Department of Epidemiology and Community Health, University of Minnesota, Minneapolis, Minnesota, USA
| | - Jon J Snyder
- Scientific Registry of Transplant Recipients, Hennepin Healthcare Research Institute, Minneapolis, Minnesota, USA.,Department of Epidemiology and Community Health, University of Minnesota, Minneapolis, Minnesota, USA
| |
Collapse
|
13
|
Mulvihill MS, Lee HJ, Weber J, Choi AY, Cox ML, Yerokun BA, Bishawi MA, Klapper J, Kuchibhatla M, Hartwig MG. Variability in donor organ offer acceptance and lung transplantation survival. J Heart Lung Transplant 2020; 39:353-362. [PMID: 32029400 DOI: 10.1016/j.healun.2019.12.010] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2019] [Revised: 12/14/2019] [Accepted: 12/29/2019] [Indexed: 10/25/2022] Open
Abstract
BACKGROUND Lung transplantation offers a survival benefit for patients with end-stage lung disease. When suitable donors are identified, centers must accept or decline the offer for a matched candidate on their waitlist. The degree to which variability in per-center offer acceptance practices impacts candidate survival is not established. The purpose of this study was to determine the degree of variability in per-center rates of lung transplantation offer acceptance and to ascertain the associated contribution to observed differences in per-center waitlist mortality. METHODS We performed a retrospective cohort study of candidates waitlisted for lung transplantation in the US using registry data. Logistic regression was fit to assess the relationship of offer acceptance with donor, candidate, and geographic factors. Listing center was evaluated as a fixed effect to determine the adjusted per-center acceptance rate. Competing risks analysis employing the Fine-Gray model was undertaken to establish the relationship between adjusted per-center acceptance and waitlist mortality. RESULTS Of 15,847 unique organ offers, 4,735 (29.9%) were accepted for first-ranked candidates. After adjustment for important covariates, transplant centers varied markedly in acceptance rate (9%-67%). Higher cumulative incidence of 1-year waitlist mortality was associated with lower acceptance rate. For every 10% increase in adjusted center acceptance rate, the risk of waitlist mortality decreased by 36.3% (sub-distribution hazard ratio 0.637; 95% confidence interval 0.592-0.685). CONCLUSIONS Variability in center-level behavior represents a modifiable risk factor for waitlist mortality in lung transplantation. Further intervention is needed to standardize center-level offer acceptance practices and minimize waitlist mortality.
Collapse
Affiliation(s)
- Michael S Mulvihill
- Division of Cardiovascular and Thoracic Surgery, Duke University, Durham, North Carolina.
| | - Hui J Lee
- Surgical Center for Outcomes Research, Durham, North Carolina; Department of Biostatistics and Bioinformatics, Duke University Medical Center, Durham, North Carolina
| | - Jeremy Weber
- Duke Clinical Research Institute, Durham, North Carolina
| | - Ashley Y Choi
- Duke University School of Medicine, Durham, North Carolina
| | - Morgan L Cox
- Division of Cardiovascular and Thoracic Surgery, Duke University, Durham, North Carolina; Duke Clinical Research Institute, Durham, North Carolina
| | - Babatunde A Yerokun
- Division of Cardiovascular and Thoracic Surgery, Duke University, Durham, North Carolina; Duke Clinical Research Institute, Durham, North Carolina
| | - Muath A Bishawi
- Division of Cardiovascular and Thoracic Surgery, Duke University, Durham, North Carolina
| | - Jacob Klapper
- Division of Cardiovascular and Thoracic Surgery, Duke University, Durham, North Carolina
| | - Maragatha Kuchibhatla
- Surgical Center for Outcomes Research, Durham, North Carolina; Department of Biostatistics and Bioinformatics, Duke University Medical Center, Durham, North Carolina
| | - Matthew G Hartwig
- Division of Cardiovascular and Thoracic Surgery, Duke University, Durham, North Carolina
| |
Collapse
|
14
|
Incorporating New Technology Into a Surgical Technique: The Learning Curve of a Single Surgeon's Stereo-Electroencephalography Experience. Neurosurgery 2019; 86:E281-E289. [DOI: 10.1093/neuros/nyz498] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2018] [Accepted: 09/04/2019] [Indexed: 11/15/2022] Open
Abstract
Abstract
BACKGROUND
Technological improvements frequently outpace the publication of randomized, controlled trials in surgical patients. This makes the application of new surgical techniques difficult as surgeons solely use clinical experience to guide changes in their practice.
OBJECTIVE
To quantitatively examine the learning curve of incorporating new technology into a surgical technique and discuss the clinical significance of incorporating this new technology into daily practice. To identify areas of improvement for operative efficiency and safety.
METHODS
A retrospective observational study examining quantitative measures of operative efficiency and safety from 2009 to 2017 in 454 consecutive patients undergoing stereo-electroencephalography depth electrode implantations.
RESULTS
The transition to a new robotic technique significantly improved operative times (196 min [95% CI 173-219] vs 115 min [95% CI 111-118], P < .0001). Cumulative sum (CUSUM) analysis demonstrated that mastery of the robotic technique took much longer than the frame-based technique (operative time peak at case 75 vs case 25, plateau of 150 vs 10 cases). Although hemorrhage rates using different vascular imaging techniques did not appear to differ using traditional statistical analysis (magnetic resonance imaging, MRI 22.3%, computed tomography angiography, CTA 17.9%, angiogram 18.1%, likelihood ratio χ2 = 4.84, P = .30), CUSUM analysis suggested MRI as the vascular imaging modality leading to higher hemorrhage and symptomatic hemorrhage rates at our center.
CONCLUSION
This experience demonstrates an improvement in operative efficiency through a series of changes made using clinical experience and intuition while transitioning to a completely new paradigm. CUSUM analysis identified potential areas for improvement in both operative efficiency and safety if used in a prospective manner.
Collapse
|
15
|
Continuous Monitoring of Transplant Center Performance: Different Options for Different Goals. Transplantation 2019; 103:1935-1944. [DOI: 10.1097/tp.0000000000002591] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
16
|
Developing CUSUM Charts for Monitoring Transplant Outcomes: Varied Goals and Many Possible Paths to Success. Transplantation 2019; 103:1760-1762. [DOI: 10.1097/tp.0000000000002590] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
17
|
Hernández D, Sánchez E, Armas-Padrón AM. Kidney transplant registries: How to optimise their utility? Nefrologia 2019; 39:581-591. [PMID: 30850219 DOI: 10.1016/j.nefro.2018.11.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2018] [Revised: 11/20/2018] [Accepted: 11/28/2018] [Indexed: 11/30/2022] Open
Abstract
The scientific Registries (RE) in renal transplantation (Tx) are very useful since they allow us to identify risk factors in this population and facilitate contrasting the information with other national and international registries, contributing to establishing strategies which improve outcomes in terms of survival. They constitute an organised and planned system that uses observational methods and standardised systematic processes, including adjusted risk models, to essentially evaluate survival outcomes. The scientific RE are complemented with clinical trials providing scientific evidence, but inexcusably need adequate statistical analysis to generate reliable clinical data that contribute to optimising the prognosis of the transplant population. In addition, scientific RE provide valuable information on the performance of Tx programmes and help generate prognostic indexes, which could contribute to improving survival. Under these prerequisites, this review will assess the following aspects related to the scientific RE in the Tx: 1) the concept and importance of implementing RE in Tx; 2) the measures that are needed for the correct execution of the scientific RE; 3) the benefits, quality and limitations of RE; 4) the statistical tools for the adequate analysis of survival; and 5) utility of RE in the evaluation of performance, quality and surveillance of transplant programmes and the generation of comorbidity índices.
Collapse
Affiliation(s)
- Domingo Hernández
- Unidad de Gestión Clínica de Nefrología, Hospital Regional Universitario Carlos Haya, Instituto Biomédico de Investigación de Málaga (IBIMA), Universidad de Málaga, REDinREN (RD16/0009/0006), Málaga, España.
| | - Emilio Sánchez
- Área de Gestión Clínica de Nefrología. Hospital Universitario Central de Asturias, REDinREN (RD16/0009/0021), Oviedo, Asturias, España
| | | |
Collapse
|
18
|
Scientific Registry of Transplant Recipients program-specific reports: where we have been and where we are going. Curr Opin Organ Transplant 2018; 24:58-63. [PMID: 30575617 DOI: 10.1097/mot.0000000000000597] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
PURPOSE OF REVIEW Reporting provider data on quality to patients and the general public is increasingly common in healthcare. Reporting outcomes in solid organ transplantation has always been controversial and deserves careful consideration to ensure optimal results. RECENT FINDINGS As mandated by Federal law, the Scientific Registry of Transplant Recipients publishes program-specific reports on transplant candidates, recipients, donors, and transplant outcomes every 6 months. Recent changes designed to make the results more easily understood by patients and the general public have been well received by patients and controversial among providers. In particular, outcomes are now reported using a five-tier system that distinguishes program results better than the old three-tier system, in which almost all programs were reported "as expected." Metrics that reflect access to transplant are also reported, including transplant rate and survival on the waiting list. Possible measures of longer term outcomes and program rates of accepting organs for transplant are being explored. SUMMARY Providing detailed information regarding transplant program practices and outcomes in ways that patients and the general public can understand remains a major focus of the Scientific Registry of Transplant Recipients. Efforts to improve data collection and metrics reported are ongoing.
Collapse
|
19
|
Wey A, Valapour M, Skeans MA, Salkowski N, Colvin M, Kasiske BL, Israni AK, Snyder JJ. Heart and lung organ offer acceptance practices of transplant programs are associated with waitlist mortality and organ yield. Am J Transplant 2018; 18:2061-2067. [PMID: 29673099 PMCID: PMC6836691 DOI: 10.1111/ajt.14885] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2017] [Revised: 04/06/2018] [Accepted: 04/08/2018] [Indexed: 01/25/2023]
Abstract
Variation in heart and lung offer acceptance practices may affect numbers of transplanted organs and create variability in waitlist mortality. To investigate these issues, offer acceptance ratios, or adjusted odds ratios, for heart and lung transplant programs individually and for all programs within donation service areas (DSAs) were estimated using offers from donors recovered July 1, 2016, and June 30, 2017. Logistic regressions estimated the association of DSA-level offer acceptance ratios with donor yield and local placement of organs recovered in the DSA. Competing risk methodology estimated the association of program-level offer acceptance ratios with incidence and rate of waitlist removals due to death or becoming too sick to undergo transplant. Higher DSA-level offer acceptance was associated with higher yield (odds ratios [ORs]: lung, 1.04 1.111.19 ; heart, 1.09 1.211.35 ) and more local placement of transplanted organs (ORs: lung, 1.01 1.121.24 ; heart, 1.47 1.691.93 ). Higher program-level offer acceptance was associated with lower incidence of waitlist removal due to death or becoming too sick to undergo transplant (hazard ratios [HRs]: heart, 0.80 0.860.93 ; lung, 0.67 0.750.83 ), but not with rate of waitlist removal (HRs: heart, 0.91 0.981.06 ; lung, 0.89 0.991.10 ). Heart and lung offer acceptance practices affected numbers of transplanted organs and contributed to program-level variability in the probability of waitlist mortality.
Collapse
Affiliation(s)
- Andrew Wey
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota
| | - Maryam Valapour
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota,Department of Pulmonary Medicine, Respiratory Institute, Cleveland Clinic, Cleveland, Ohio
| | - Melissa A. Skeans
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota
| | - Nicholas Salkowski
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota
| | - Monica Colvin
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota,Department of Cardiology, University of Michigan, Ann Arbor, MI
| | - Bertram L. Kasiske
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota,Department of Medicine, Hennepin County Medical Center, University of Minnesota, Minneapolis, Minnesota
| | - Ajay K. Israni
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota,Department of Medicine, Hennepin County Medical Center, University of Minnesota, Minneapolis, Minnesota,Department of Epidemiology and Community Health, University of Minnesota, Minneapolis, Minnesota
| | - Jon J. Snyder
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota,Department of Epidemiology and Community Health, University of Minnesota, Minneapolis, Minnesota
| |
Collapse
|
20
|
Wittenberg P, Gan FF, Knoth S. A simple signaling rule for variable life-adjusted display derived from an equivalent risk-adjusted CUSUM chart. Stat Med 2018; 37:2455-2473. [PMID: 29667215 DOI: 10.1002/sim.7647] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2017] [Revised: 11/10/2017] [Accepted: 01/25/2018] [Indexed: 11/06/2022]
Abstract
The variable life-adjusted display (VLAD) is the first risk-adjusted graphical procedure proposed in the literature for monitoring the performance of a surgeon. It displays the cumulative sum of expected minus observed deaths. It has since become highly popular because the statistic plotted is easy to understand. But it is also easy to misinterpret a surgeon's performance by utilizing the VLAD, potentially leading to grave consequences. The problem of misinterpretation is essentially caused by the variance of the VLAD's statistic that increases with sample size. In order for the VLAD to be truly useful, a simple signaling rule is desperately needed. Various forms of signaling rules have been developed, but they are usually quite complicated. Without signaling rules, making inferences using the VLAD alone is difficult if not misleading. In this paper, we establish an equivalence between a VLAD with V-mask and a risk-adjusted cumulative sum (RA-CUSUM) chart based on the difference between the estimated probability of death and surgical outcome. Average run length analysis based on simulation shows that this particular RA-CUSUM chart has similar performance as compared to the established RA-CUSUM chart based on the log-likelihood ratio statistic obtained by testing the odds ratio of death. We provide a simple design procedure for determining the V-mask parameters based on a resampling approach. Resampling from a real data set ensures that these parameters can be estimated appropriately. Finally, we illustrate the monitoring of a real surgeon's performance using VLAD with V-mask.
Collapse
Affiliation(s)
- Philipp Wittenberg
- Department of Mathematics and Statistics, Helmut Schmidt University, Hamburg, Germany
| | - Fah Fatt Gan
- Department of Statistics and Applied Probability, National University of Singapore, Singapore
| | - Sven Knoth
- Department of Mathematics and Statistics, Helmut Schmidt University, Hamburg, Germany
| |
Collapse
|
21
|
Gurien LA, Tepas JJ, Lind DS, Chesire DJ, Sabra MJ, Dalton BGA, Awad ZT. How Safe Is the Safety Net? Comparison of Ivor-Lewis Esophagectomy at a Safety-Net Hospital Using the NSQIP Database. J Am Coll Surg 2018; 226:680-684. [DOI: 10.1016/j.jamcollsurg.2017.12.036] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2017] [Accepted: 12/11/2017] [Indexed: 11/28/2022]
|
22
|
Aylin P, Bottle A, Burnett S, Cecil E, Charles KL, Dawson P, D’Lima D, Esmail A, Vincent C, Wilkinson S, Benn J. Evaluation of a national surveillance system for mortality alerts: a mixed-methods study. HEALTH SERVICES AND DELIVERY RESEARCH 2018. [DOI: 10.3310/hsdr06070] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
BackgroundSince 2007, Imperial College London has generated monthly mortality alerts, based on statistical process control charts and using routinely collected hospital administrative data, for all English acute NHS hospital trusts. The impact of this system has not yet been studied.ObjectivesTo improve understanding of mortality alerts and evaluate their impact as an intervention to reduce mortality.DesignMixed methods.SettingEnglish NHS acute hospital trusts.ParticipantsEleven trusts were included in the case study. The survey involved 78 alerting trusts.Main outcome measuresRelative risk of mortality and perceived efficacy of the alerting system.Data sourcesHospital Episodes Statistics, published indicators on quality and safety, Care Quality Commission (CQC) reports, interviews and documentary evidence from case studies, and a national evaluative survey.MethodsDescriptive analysis of alerts; association with other measures of quality; associated change in mortality using an interrupted time series approach; in-depth qualitative case studies of institutional response to alerts; and a national cross-sectional evaluative survey administered to describe the organisational structure for mortality governance and perceptions of efficacy of alerts.ResultsA total of 690 mortality alerts generated between April 2007 and December 2014. CQC pursued 75% (154/206) of alerts sent between 2011 and 2013. Patient care was cited as a factor in 70% of all investigations and in 89% of sepsis alerts. Alerts were associated with indicators on bed occupancy, hospital mortality, staffing, financial status, and patient and trainee satisfaction. On average, the risk of death fell by 58% during the 9-month lag following an alert, levelling afterwards and reaching an expected risk within 18 months of the alert. Acute myocardial infarction (AMI) and sepsis alerts instigated institutional responses across all the case study sites, although most sites were undertaking some parallel activities at a more general level to address known problems in care in these and other areas. Responses included case note review and coding improvements, changes in patient pathways, changes in diagnosis of sepsis and AMI, staff training in case note write-up and coding, greater transparency in patient deterioration, and infrastructure changes. Survey data revealed that 86% of responding trusts had a dedicated trust-level lead for mortality reduction and 92% had a dedicated trust-level mortality group or committee in place. Trusts reported that mortality reduction was a high priority and that there was strong senior leadership support for mortality monitoring. The weakest areas reported concerned the accuracy of coding, the quality of specialty-level mortality data and understanding trends in specialty-level mortality data.LimitationsOwing to the correlational nature of our analysis, we could not ascribe a causal link between mortality alerts and reductions in mortality. The complexity of the institutional context and behaviour hindered our capacity to attribute locally reported changes specifically to the effects of the alerts rather than to ongoing institutional strategy.ConclusionsThe mortality alert surveillance system reflects aspects of quality care and is valued by trusts. Alerts were considered a useful focus for identifying problems and implementing interventions around mortality.Future workA further analysis of site visits and survey material, the application of evaluative framework to other interventions, a blinded case note review and the dissemination of findings.FundingThe National Institute for Health Research Health Services and Delivery Research programme.
Collapse
Affiliation(s)
- Paul Aylin
- Dr Foster Unit at Imperial, Department of Primary Care and Public Health, Imperial College London, London, UK
| | - Alex Bottle
- Dr Foster Unit at Imperial, Department of Primary Care and Public Health, Imperial College London, London, UK
| | - Susan Burnett
- Faculty of Medicine, Department of Surgery & Cancer, Imperial College London, London, UK
| | - Elizabeth Cecil
- Dr Foster Unit at Imperial, Department of Primary Care and Public Health, Imperial College London, London, UK
| | - Kathryn L Charles
- Faculty of Medicine, Department of Surgery & Cancer, Imperial College London, London, UK
| | - Paul Dawson
- Faculty of Medicine, Department of Surgery & Cancer, Imperial College London, London, UK
| | - Danielle D’Lima
- Faculty of Medicine, Department of Surgery & Cancer, Imperial College London, London, UK
| | - Aneez Esmail
- Division of Population Health, Health Services Research & Primary Care, University of Manchester, Manchester, UK
| | | | - Samantha Wilkinson
- Dr Foster Unit at Imperial, Department of Primary Care and Public Health, Imperial College London, London, UK
| | - Jonathan Benn
- Faculty of Medicine, Department of Surgery & Cancer, Imperial College London, London, UK
| |
Collapse
|
23
|
Wey A, Salkowski N, Kasiske BL, Israni AK, Snyder JJ. Influence of kidney offer acceptance behavior on metrics of allocation efficiency. Clin Transplant 2017; 31:10.1111/ctr.13057. [PMID: 28712148 PMCID: PMC5689462 DOI: 10.1111/ctr.13057] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/11/2017] [Indexed: 11/26/2022]
Abstract
We investigated associations of deceased donor kidney offer acceptance with likelihood of the kidney being discarded, cold ischemia time at transplant (CIT), and likelihood of the kidney being exported outside the donation service area (DSA). We used kidney offers from donors in the Scientific Registry of Transplant Recipients July 1, 2015-June 30, 2016, and a stratified logistic regression to estimate odds ratios of acceptance for candidates wait-listed in a DSA. We estimated associations between these ratios and likelihood of discard or export and CIT at transplant. Approximately 0.50 kidneys were discarded per donor; lower DSA-specific offer acceptance ratios were associated with more discards (R=-0.20; P=0.006). For a median donor, the DSA with the highest acceptance ratio would place 0.12 more kidneys per donor than the DSA with the lowest ratio. Low acceptance ratios were associated with higher CIT (R=-0.23; P<0.001). For the median donor, CIT was 2.9 hours shorter for the DSA with the highest versus lowest acceptance ratio. Low acceptance ratios were associated with more exports (R=-0.43; P<0.001); the probability was 15% higher for a median donor in the DSA with the lowest versus highest acceptance ratio. Improving lower-than-expected offer acceptance would likely reduce discards, CIT, and exports.
Collapse
Affiliation(s)
- Andrew Wey
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota
| | - Nicholas Salkowski
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota
| | - Bertram L. Kasiske
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota
- Department of Medicine, Hennepin County Medical Center, Minneapolis, Minnesota
| | - Ajay K. Israni
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota
- Department of Medicine, Hennepin County Medical Center, Minneapolis, Minnesota
- Division of Epidemiology and Community Health, School of Public Health, University of Minnesota, Minneapolis, Minnesota
| | - Jon J. Snyder
- Scientific Registry of Transplant Recipients, Minneapolis Medical Research Foundation, Minneapolis, Minnesota
- Division of Epidemiology and Community Health, School of Public Health, University of Minnesota, Minneapolis, Minnesota
| |
Collapse
|
24
|
Jay C, Schold JD. Measuring transplant center performance: The goals are not controversial but the methods and consequences can be. CURRENT TRANSPLANTATION REPORTS 2017; 4:52-58. [PMID: 28966901 DOI: 10.1007/s40472-017-0138-9] [Citation(s) in RCA: 49] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
PURPOSE OF REVIEW Risks of regulatory scrutiny has generated widespread concern about increasingly risk averse transplant center behaviors regarding both donor and candidate acceptance patterns. To address potential unintended consequences threatening access to care, we discuss recent changes in regulatory metrics and potential improvements in quality oversight of transplant centers. RECENT FINDINGS Despite many recent changes to one-year patient and graft survival regulatory criteria, the capacity to accurately identify true underperforming centers and avoiding false positive flagging remains an area of great concern. Numerous studies have demonstrated restrictions in transplant volume and access following transplant center flagging. SUMMARY Current regulatory criteria are limited in their capacity to accurately identify poorly performing centers and potentially encourage risk-averse behavior by transplant centers. Efforts to address these concerns should focus on (1) improving risk-adjustment models with better data which captures the acuity of candidate and donor risk, (2) reconsidering primary outcomes measured to assess comprehensive transplant center performance, (3) improving education to address rational or perceived disincentives, and (4) using data more effectively to share best practices.
Collapse
Affiliation(s)
- Colleen Jay
- University Transplant Center, University of Texas Health Science Center, San Antonio
| | - Jesse D Schold
- Department of Quantitative Health Sciences, Cleveland Clinic, Cleveland, Ohio
- Cleveland Clinic Lerner College of Medicine, Case Western Reserve University, Cleveland, Ohio
| |
Collapse
|
25
|
Irwin FD, Wu C, Bannister WM, Bonagura AF, Laihinen B, Axelrod DA, Schnitzler MA. A commercial transplant network's perspective of value in solid organ transplantation: Strategizing for value in transplant care. Transplant Rev (Orlando) 2016; 30:71-6. [DOI: 10.1016/j.trre.2015.11.002] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2015] [Accepted: 11/10/2015] [Indexed: 10/22/2022]
|
26
|
|
27
|
Grochowiecki T, Jakimowicz T, Grabowska-Derlatka L, Szmidt J. Quality of pancreatic transplant program assessment using a risk-adjusted cumulative sum chart: experience from a single, small center. Transplant Proc 2014; 46:2810-4. [PMID: 25380924 DOI: 10.1016/j.transproceed.2014.08.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
BACKGROUND The high rate of complication after pancreas transplantation not only had an impact on recipient quality of life and survival but also had significant financial implications. Thus, monitoring transplant center performance was crucial to indentifying changes in clinical practice that result in quality deterioration. OBJECTIVE To evaluate retrospectively the quality of the single, small pancreatic transplant program and to establish prospective monitoring of the center using risk-adjusted cumulative sum (CUSUM). METHODOLOGY From 1988 to 2014, 119 simultaneous pancreas and the kidney transplantations (SPKTx) were performed. The program was divided into 3 eras, based on surgical technique and immunosuppression. Analyses of the 15 fatal outcomes due to complication from pancreatic graft were performed. The risk model was developed using multivariable logistic regression analysis based on retrospective data of 112 SPKTx recipients. The risk-adjusted 1-sided CUSUM chart was plotted for retrospective and prospective events. The upper control limit was set to 2. RESULTS There were 2 main causes of death: multiorgan failure (73.3%; 11/15) and septic hemorrhage (26.7%; 4/15). Quality analysis using the CUSUM chart revealed that the process was not homogeneous; however, no significant signal of program deterioration was obtained and the performance of the whole program was within the settled control limit. CONCLUSION For a single pancreatic transplant center. The risk-adjusted CUSUM chart was a useful tool for quality program assessment. It could support decision making during traditional surgical morbidity and mortality conferences. For small transplant centers, increasing the sensitivity of the CUSUM method by lowering the upper control limit should be considered. However, an individual assessment approach of the for particular centers is recommended.
Collapse
Affiliation(s)
- T Grochowiecki
- Department of General, Vascular and Transplant Surgery, Medical University of Warsaw, Poland.
| | - T Jakimowicz
- Department of General, Vascular and Transplant Surgery, Medical University of Warsaw, Poland
| | | | - J Szmidt
- Department of General, Vascular and Transplant Surgery, Medical University of Warsaw, Poland
| |
Collapse
|