1
|
Blackburn KW, Cooper LE, Bafford AC, Hu Y, Brown RF. Using risk-adjusted cumulative sum to evaluate surgeon, divisional, and institutional outcomes-a feasibility study. Surgery 2024; 175:1554-1561. [PMID: 38523020 DOI: 10.1016/j.surg.2024.01.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 12/07/2023] [Accepted: 01/24/2024] [Indexed: 03/26/2024]
Abstract
BACKGROUND Few objective, real-time measurements of surgeon performance exist. The risk-adjusted cumulative sum is a novel method that can track surgeon-level outcomes on a continuous basis. The objective of this study was to demonstrate the feasibility of using risk-adjusted cumulative sum to monitor outcomes after colorectal operations and identify clinically relevant performance variations. METHODS The National Surgical Quality Improvement Program was queried to obtain patient-level data for 1,603 colorectal operations at a high-volume center from 2011 to 2020. For each case, expected risks of morbidity, mortality, reoperation, readmission, and prolonged length of stay were estimated using the National Surgical Quality Improvement Program risk calculator. Risk-adjusted cumulative sum curves were generated to signal observed-to-expected odds ratios of 1.5 (poor performance) and 0.5 (exceptional performance). Control limits were set based on a false positive rate of 5% (α = 0.05). RESULTS The cohort included data on 7 surgeons (those with more than 20 cases in the study period). Institutional observed versus expected outcomes were the following: morbidity 12.5% (vs 15.0%), mortality 2.5% (vs 2.0%), prolonged length of stay 19.7% (vs 19.1%), reoperation 11.1% (vs 11.3%), and 30-day readmission 6.1% (vs 4.8%). Risk-adjusted cumulative sum accurately demonstrated within- and between-surgeon performance variations across these metrics and proved effective when considering division-level data. CONCLUSION Risk-adjusted cumulative sum adjusts for patient-level risk factors to provide real-time data on surgeon-specific outcomes. This approach enables prompt identification of performance outliers and can contribute to quality assurance, root-cause analysis, and incentivization not only at the surgeon level but at divisional and institutional levels as well.
Collapse
Affiliation(s)
- Kyle W Blackburn
- School of Medicine, Baylor College of Medicine, Waco, TX. https://twitter.com/KyleWBlackburn
| | - Laura E Cooper
- Department of Surgery, University of Maryland School of Medicine, Baltimore, MD
| | | | - Yinin Hu
- Department of Surgery, University of Maryland School of Medicine, Baltimore, MD
| | - Rebecca F Brown
- Department of Surgery, University of Maryland School of Medicine, Baltimore, MD.
| |
Collapse
|
2
|
Stratulat V, Melamed N, Barrett J, Ladhani NNN, Anabusi S, Quaglietta P, Hack K, Ronzoni S. Cervical assessment certification and its impact on performance quality in the context of universal cervical screening. Int J Gynaecol Obstet 2024; 164:951-958. [PMID: 37675927 DOI: 10.1002/ijgo.15078] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Revised: 08/08/2023] [Accepted: 08/17/2023] [Indexed: 09/08/2023]
Abstract
OBJECTIVE To assess the impact of the introduction of universal transvaginal cervical screening and certification on the quality of cervical length ultrasound images. METHODS The present study included a retrospective cohort of singleton pregnancies that underwent transvaginal cervical length measurement at the anatomical scan (180/7 and 236/7 weeks) before (period A, 2015-2017) and after (period B, 2017-2019) the introduction of universal transvaginal cervical length screening. Independent observers blindly evaluated the images obtained for cervical length using a qualitative scoring method based on five criteria, according to the Fetal Medicine Foundation. RESULTS In all, 6013 patients met the inclusion criteria, 3333 in period A and 2680 in period B. Maternal characteristics and risk factors for preterm birth were similar between the two periods. The acceptance of transvaginal cervical length measurement in period B was 95.5% in the overall cohort and 100% in the subgroup of high-risk patients. The quality score was significantly higher in period B than in period A. Among the image quality criteria, the anterior/posterior ratio, the correct magnification of the images, and the calipers' placement contributed significantly to the improved quality score in period B. Most of the sonographers performed better in period B, irrespective of the years of experience, but certificate holders obtained higher scores than non-certified sonographers, particularly those in mid-career. The identification of short cervix was significantly higher in period B than in period A. CONCLUSION The implementation of universal transvaginal cervical length screening and the certification process are associated with improved quality of cervical length images, even among expert sonographers and in the presence of anatomical pitfalls.
Collapse
Affiliation(s)
- Vasilica Stratulat
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Dan Women & Babies Program, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
- Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
| | - Nir Melamed
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Dan Women & Babies Program, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
| | - Jon Barrett
- Department of Obstetric and Gynecology, McMaster University, Hamilton, Ontario, Canada
| | - Noor N N Ladhani
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Dan Women & Babies Program, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
| | - Saja Anabusi
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Dan Women & Babies Program, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
| | - Paula Quaglietta
- Department of Genetics and Genome Biology, Institute of Medical Science, University of Toronto, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Kalesha Hack
- Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
| | - Stefania Ronzoni
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Dan Women & Babies Program, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
3
|
Blackburn KW, Turrentine FE, Schirmer BD, Hallowell PT, Kubicki NS, Hu Y, Kligman MD. Monitoring performance in laparoscopic gastric bypass surgery using risk-adjusted cumulative sum at 2 high-volume centers. Surg Obes Relat Dis 2023; 19:1049-1057. [PMID: 36931965 DOI: 10.1016/j.soard.2023.02.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2022] [Revised: 01/09/2023] [Accepted: 02/04/2023] [Indexed: 02/17/2023]
Abstract
BACKGROUND Traditional surgical outcomes are measured retrospectively and intermittently, limiting opportunities for early intervention. OBJECTIVES The objective of this study was to use risk-adjusted cumulative sum (RA-CUSUM) to track perioperative surgical outcomes for laparoscopic gastric bypass. We hypothesized that RA-CUSUM could identify performance variations between surgeons. SETTING Two mid-Atlantic quaternary care academic centers. METHODS Patient-level data from the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBSAQIP) were abstracted for laparoscopic gastric bypasses performed by 3 surgeons at 2 high-volume centers from 2014 to 2021. Estimated probabilities of serious complications, reoperation, and readmission were derived from the MBSAQIP risk calculator. RA-CUSUM curves were generated to signal observed-to-expected odds ratios (ORs) of 1.5 (poor performance) and .5 (superior performance). Control limits were set based on a false positive rate of 5% (α = .05). RESULTS We included 1192 patients: Surgeon A = 767, Surgeon B = 188, and Surgeon C = 237. Overall rates of serious complications, 30-day reoperations, and 30-day readmissions were 3.9%, 2.5%, and 5.2% respectively, with expected rates of 4.7%, 2.2%, and 5.8%. RA-CUSUM signaled lower-than-expected (OR < .5) rates of readmission and serious complication in Surgeon A, and higher-than-expected (OR > 1.5) readmission rate in Surgeon C. Surgeon A further demonstrated an early period of higher-than-expected (OR > 1.5) reoperation rate before April 2015, followed by superior performance thereafter (OR < .5). Surgeon B's performance generally reflected expected standards throughout the study period. CONCLUSIONS RA-CUSUM adjusts for clinical risk factors and identifies performance outliers in real-time. This approach to analyzing surgical outcomes is applicable to quality improvement, root-cause analysis, and surgeon incentivization.
Collapse
Affiliation(s)
- Kyle W Blackburn
- Division of General and Oncologic Surgery, Department of Surgery, University of Maryland School of Medicine, Baltimore, Maryland
| | - Florence E Turrentine
- Division of General Surgery, Department of Surgery, University of Virginia School of Medicine, Charlottesville, Virginia
| | - Bruce D Schirmer
- Division of General Surgery, Department of Surgery, University of Virginia School of Medicine, Charlottesville, Virginia
| | - Peter T Hallowell
- Division of General Surgery, Department of Surgery, University of Virginia School of Medicine, Charlottesville, Virginia
| | - Natalia S Kubicki
- Division of General and Oncologic Surgery, Department of Surgery, University of Maryland School of Medicine, Baltimore, Maryland
| | - Yinin Hu
- Division of General and Oncologic Surgery, Department of Surgery, University of Maryland School of Medicine, Baltimore, Maryland
| | - Mark D Kligman
- Division of General and Oncologic Surgery, Department of Surgery, University of Maryland School of Medicine, Baltimore, Maryland.
| |
Collapse
|
4
|
Aiartzaguena A, Del Campo A, Melchor I, Gutiérrez J, Melchor JC, Burgos J. Expected-value bias in mid-trimester preterm birth screening. ULTRASOUND IN OBSTETRICS & GYNECOLOGY : THE OFFICIAL JOURNAL OF THE INTERNATIONAL SOCIETY OF ULTRASOUND IN OBSTETRICS AND GYNECOLOGY 2022; 59:793-798. [PMID: 34542928 DOI: 10.1002/uog.24778] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Revised: 08/13/2021] [Accepted: 09/06/2021] [Indexed: 06/13/2023]
Abstract
OBJECTIVES Cervical length (CL) measurement ≤ 25 mm on mid-trimester ultrasound scan is a known risk factor for preterm birth, for which vaginal progesterone is recommended. The aims of this study were to evaluate whether CL measurement is affected by observer bias and to assess the impact on short cervix prevalence of masking CL measurement during routine mid-trimester ultrasound scan. METHODS This was a flash study designed for a 2-month period (October and November 2018) at Cruces University Hospital (Bizkaia, Spain), in which all CL measurements from routine mid-trimester scans were masked. During the study period, there was no modification of the routine screening method, and women with a short cervix were prescribed 200 mg vaginal progesterone daily as per usual. The control group included women examined in a 2-month period (April and May 2018) prior to the study, in which CL measurements were taken as usual by a non-blinded operator. The primary outcome was the prevalence of short cervix in each group. RESULTS A total of 983 CL measurements were analyzed, including 457 in the blinded group and 526 in the control group. The prevalence of short cervix was 2.7% in the non-blinded group and 5.5% in the blinded group (P = 0.024). We identified a statistically significant difference in the incidence of CL of 24-25 mm between the two groups, with a lower prevalence in the non-blinded vs blinded group (0.6% vs 2.4%; P < 0.005). Moreover, the distribution of CL values was normal in the blinded group, in contrast to the non-blinded group, which was characterized by skewed distribution of CL values. CONCLUSIONS Expected-value bias exists and should be taken into account when measuring CL in mid-trimester preterm birth screening. Blinding has demonstrated to be an effective strategy to improve the performance of CL screening in clinical practice. © 2021 International Society of Ultrasound in Obstetrics and Gynecology.
Collapse
Affiliation(s)
- A Aiartzaguena
- Hospital Universitario Cruces, Biocruces Bizkaia Health Research Institute, UPV/EHU, Bizkaia, Spain
| | - A Del Campo
- Hospital Universitario Cruces, Biocruces Bizkaia Health Research Institute, UPV/EHU, Bizkaia, Spain
| | - I Melchor
- Hospital Universitario Cruces, Biocruces Bizkaia Health Research Institute, UPV/EHU, Bizkaia, Spain
| | - J Gutiérrez
- Hospital Universitario Cruces, Biocruces Bizkaia Health Research Institute, UPV/EHU, Bizkaia, Spain
| | - J C Melchor
- Hospital Universitario Cruces, Biocruces Bizkaia Health Research Institute, UPV/EHU, Bizkaia, Spain
| | - J Burgos
- Hospital Universitario Cruces, Biocruces Bizkaia Health Research Institute, UPV/EHU, Bizkaia, Spain
| |
Collapse
|
5
|
Soto EE, Hernandez-Andrade E, Huntley ES, Blackwell SC. Implementing a Universal Cervical Length Screening Program in a Large Hospital System, it Takes Some Time to Achieve Consistent Results. Gynecol Obstet Invest 2022; 87:124-132. [PMID: 35354147 DOI: 10.1159/000524361] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Accepted: 03/28/2022] [Indexed: 11/19/2022]
Abstract
OBJECTIVE To describe changes in the acceptance of transvaginal (TV) cervical length (CL) measurements, and in the variance of CL measurements among operators, after implementation of universal TV-CL measurements at 18+0-23+6 weeks/days of gestation. STUDY DESIGN Retrospective cohort study Participants/Materials, Setting, Methods This study was performed after universal TV-CL screening was implemented at the University of Texas Health Science Center in Houston, Texas, for all women undergoing an anatomy ultrasound (US) between 18 0/6 and 23 6/7 weeks of gestation. Pregnant women carrying singletons without prior history of preterm delivery who underwent anatomy ultrasound evaluation between September 2017 and March 2020 (30 months) were included. The complete study period was divided in five epochs of six months each. Changes in patient's acceptance for the TV scan, in CL distribution, in the prevalence of short cervix defined as ≤15, ≤20, or ≤25 mm, and in the performance of ultrasound operators across the five epochs were evaluated. For this analysis, success rate was defined as the percentage of TV-CL measurements obtained in relation to the number of second trimester anatomy scans. RESULTS A total of 22,207 low risk pregnant women evaluated by 36 trained sonographers were analyzed. Overall, the acceptance for TV-CL measurement was 82.3% (18,289/22,207), increasing from 76.7% in the first epoch to 82.8% (p<0.0001) in the last epoch. The mean CL did not significantly change from 38.6 mm in the first epoch to 38.5 mm in the last epoch (p=0.7); however, the standard deviation decreased from 7.9 mm in the first epoch to 7.04 mm in the last epoch (p=<0.01). The prevalence of a short cervix ≤25 mm was 2.2% (n=399/18,289), ≤20 mm was 1.2% (224/18,289), and ≤15 mm was 0.9% (162/18,289). This prevalence varied only for CL ≤25 mm from 3.02% (88/2,907) in the first epoch to 1.77% (64/3,615) in the last epoch (p=0.0009). There was a variation in CL measurements among operators (mean 3.3 mm). Sonographers with less than one year of experience had a lower success rate for completing TV-CL examinations than more experienced sonographers (80.8% vs. 85.8%; p <0.03). In general 77% (27/35) of operators had a success rate ≥80% for completing TV-SL scans. LIMITATIONS Characteristics of individuals who accepted versus those who declined TV-CL were not compared; CL values were not correlated with clinical outcomes. CONCLUSIONS During the first six months after implementation of a universal cervical length screening program, there was greater variation in CL measurements, lower acceptance for a TV ultrasound, and a higher number of women diagnosed with a CL ≤25 mm, as compared to subsequent epochs. After the first six months, these metrics improved and remained stable. Most operators improved their performance over time; however, there were a few with a low success rate for TV-CL, and others who systematically over or under estimate CL measurements.
Collapse
Affiliation(s)
- Eleazar E Soto
- Department of Obstetrics and Gynecology and Reproductive Sciences, McGovern Medical School, University of Texas, Health Science Center at Houston (UTHealth), Houston, Texas, USA
| | - Edgar Hernandez-Andrade
- Department of Obstetrics and Gynecology and Reproductive Sciences, McGovern Medical School, University of Texas, Health Science Center at Houston (UTHealth), Houston, Texas, USA
| | - Erin S Huntley
- Department of Obstetrics and Gynecology and Reproductive Sciences, McGovern Medical School, University of Texas, Health Science Center at Houston (UTHealth), Houston, Texas, USA
| | - Sean C Blackwell
- Department of Obstetrics and Gynecology and Reproductive Sciences, McGovern Medical School, University of Texas, Health Science Center at Houston (UTHealth), Houston, Texas, USA
| |
Collapse
|
6
|
de Costa C. ANZJOG
turns 60! Aust N Z J Obstet Gynaecol 2020; 60:9-10. [DOI: 10.1111/ajo.13125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2020] [Accepted: 01/08/2020] [Indexed: 11/30/2022]
Affiliation(s)
- Caroline de Costa
- James Cook University College of Medicine Cairns Queensland Australia
| |
Collapse
|