Developing a quality assurance program for transvaginal cervical length measurement at 18-21 weeks' gestation

Using risk-adjusted cumulative sum to evaluate surgeon, divisional, and institutional outcomes-a feasibility study

BACKGROUND

Few objective, real-time measurements of surgeon performance exist. The risk-adjusted cumulative sum is a novel method that can track surgeon-level outcomes on a continuous basis. The objective of this study was to demonstrate the feasibility of using risk-adjusted cumulative sum to monitor outcomes after colorectal operations and identify clinically relevant performance variations.

METHODS

The National Surgical Quality Improvement Program was queried to obtain patient-level data for 1,603 colorectal operations at a high-volume center from 2011 to 2020. For each case, expected risks of morbidity, mortality, reoperation, readmission, and prolonged length of stay were estimated using the National Surgical Quality Improvement Program risk calculator. Risk-adjusted cumulative sum curves were generated to signal observed-to-expected odds ratios of 1.5 (poor performance) and 0.5 (exceptional performance). Control limits were set based on a false positive rate of 5% (α = 0.05).

RESULTS

The cohort included data on 7 surgeons (those with more than 20 cases in the study period). Institutional observed versus expected outcomes were the following: morbidity 12.5% (vs 15.0%), mortality 2.5% (vs 2.0%), prolonged length of stay 19.7% (vs 19.1%), reoperation 11.1% (vs 11.3%), and 30-day readmission 6.1% (vs 4.8%). Risk-adjusted cumulative sum accurately demonstrated within- and between-surgeon performance variations across these metrics and proved effective when considering division-level data.

CONCLUSION

Risk-adjusted cumulative sum adjusts for patient-level risk factors to provide real-time data on surgeon-specific outcomes. This approach enables prompt identification of performance outliers and can contribute to quality assurance, root-cause analysis, and incentivization not only at the surgeon level but at divisional and institutional levels as well.

Cervical assessment certification and its impact on performance quality in the context of universal cervical screening

OBJECTIVE

To assess the impact of the introduction of universal transvaginal cervical screening and certification on the quality of cervical length ultrasound images.

METHODS

The present study included a retrospective cohort of singleton pregnancies that underwent transvaginal cervical length measurement at the anatomical scan (180/7 and 236/7 weeks) before (period A, 2015-2017) and after (period B, 2017-2019) the introduction of universal transvaginal cervical length screening. Independent observers blindly evaluated the images obtained for cervical length using a qualitative scoring method based on five criteria, according to the Fetal Medicine Foundation.

RESULTS

In all, 6013 patients met the inclusion criteria, 3333 in period A and 2680 in period B. Maternal characteristics and risk factors for preterm birth were similar between the two periods. The acceptance of transvaginal cervical length measurement in period B was 95.5% in the overall cohort and 100% in the subgroup of high-risk patients. The quality score was significantly higher in period B than in period A. Among the image quality criteria, the anterior/posterior ratio, the correct magnification of the images, and the calipers' placement contributed significantly to the improved quality score in period B. Most of the sonographers performed better in period B, irrespective of the years of experience, but certificate holders obtained higher scores than non-certified sonographers, particularly those in mid-career. The identification of short cervix was significantly higher in period B than in period A.

CONCLUSION

The implementation of universal transvaginal cervical length screening and the certification process are associated with improved quality of cervical length images, even among expert sonographers and in the presence of anatomical pitfalls.

Monitoring performance in laparoscopic gastric bypass surgery using risk-adjusted cumulative sum at 2 high-volume centers

BACKGROUND

Traditional surgical outcomes are measured retrospectively and intermittently, limiting opportunities for early intervention.

OBJECTIVES

The objective of this study was to use risk-adjusted cumulative sum (RA-CUSUM) to track perioperative surgical outcomes for laparoscopic gastric bypass. We hypothesized that RA-CUSUM could identify performance variations between surgeons.

SETTING

Two mid-Atlantic quaternary care academic centers.

METHODS

Patient-level data from the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBSAQIP) were abstracted for laparoscopic gastric bypasses performed by 3 surgeons at 2 high-volume centers from 2014 to 2021. Estimated probabilities of serious complications, reoperation, and readmission were derived from the MBSAQIP risk calculator. RA-CUSUM curves were generated to signal observed-to-expected odds ratios (ORs) of 1.5 (poor performance) and .5 (superior performance). Control limits were set based on a false positive rate of 5% (α = .05).

RESULTS

We included 1192 patients: Surgeon A = 767, Surgeon B = 188, and Surgeon C = 237. Overall rates of serious complications, 30-day reoperations, and 30-day readmissions were 3.9%, 2.5%, and 5.2% respectively, with expected rates of 4.7%, 2.2%, and 5.8%. RA-CUSUM signaled lower-than-expected (OR < .5) rates of readmission and serious complication in Surgeon A, and higher-than-expected (OR > 1.5) readmission rate in Surgeon C. Surgeon A further demonstrated an early period of higher-than-expected (OR > 1.5) reoperation rate before April 2015, followed by superior performance thereafter (OR < .5). Surgeon B's performance generally reflected expected standards throughout the study period.

CONCLUSIONS

RA-CUSUM adjusts for clinical risk factors and identifies performance outliers in real-time. This approach to analyzing surgical outcomes is applicable to quality improvement, root-cause analysis, and surgeon incentivization.

Expected-value bias in mid-trimester preterm birth screening

OBJECTIVES

Cervical length (CL) measurement ≤ 25 mm on mid-trimester ultrasound scan is a known risk factor for preterm birth, for which vaginal progesterone is recommended. The aims of this study were to evaluate whether CL measurement is affected by observer bias and to assess the impact on short cervix prevalence of masking CL measurement during routine mid-trimester ultrasound scan.

METHODS

This was a flash study designed for a 2-month period (October and November 2018) at Cruces University Hospital (Bizkaia, Spain), in which all CL measurements from routine mid-trimester scans were masked. During the study period, there was no modification of the routine screening method, and women with a short cervix were prescribed 200 mg vaginal progesterone daily as per usual. The control group included women examined in a 2-month period (April and May 2018) prior to the study, in which CL measurements were taken as usual by a non-blinded operator. The primary outcome was the prevalence of short cervix in each group.

RESULTS

A total of 983 CL measurements were analyzed, including 457 in the blinded group and 526 in the control group. The prevalence of short cervix was 2.7% in the non-blinded group and 5.5% in the blinded group (P = 0.024). We identified a statistically significant difference in the incidence of CL of 24-25 mm between the two groups, with a lower prevalence in the non-blinded vs blinded group (0.6% vs 2.4%; P < 0.005). Moreover, the distribution of CL values was normal in the blinded group, in contrast to the non-blinded group, which was characterized by skewed distribution of CL values.

CONCLUSIONS

Expected-value bias exists and should be taken into account when measuring CL in mid-trimester preterm birth screening. Blinding has demonstrated to be an effective strategy to improve the performance of CL screening in clinical practice. © 2021 International Society of Ultrasound in Obstetrics and Gynecology.

Implementing a Universal Cervical Length Screening Program in a Large Hospital System, it Takes Some Time to Achieve Consistent Results

OBJECTIVE

To describe changes in the acceptance of transvaginal (TV) cervical length (CL) measurements, and in the variance of CL measurements among operators, after implementation of universal TV-CL measurements at 18+0-23+6 weeks/days of gestation.

STUDY DESIGN

Retrospective cohort study Participants/Materials, Setting, Methods This study was performed after universal TV-CL screening was implemented at the University of Texas Health Science Center in Houston, Texas, for all women undergoing an anatomy ultrasound (US) between 18 0/6 and 23 6/7 weeks of gestation. Pregnant women carrying singletons without prior history of preterm delivery who underwent anatomy ultrasound evaluation between September 2017 and March 2020 (30 months) were included. The complete study period was divided in five epochs of six months each. Changes in patient's acceptance for the TV scan, in CL distribution, in the prevalence of short cervix defined as ≤15, ≤20, or ≤25 mm, and in the performance of ultrasound operators across the five epochs were evaluated. For this analysis, success rate was defined as the percentage of TV-CL measurements obtained in relation to the number of second trimester anatomy scans.

RESULTS

A total of 22,207 low risk pregnant women evaluated by 36 trained sonographers were analyzed. Overall, the acceptance for TV-CL measurement was 82.3% (18,289/22,207), increasing from 76.7% in the first epoch to 82.8% (p<0.0001) in the last epoch. The mean CL did not significantly change from 38.6 mm in the first epoch to 38.5 mm in the last epoch (p=0.7); however, the standard deviation decreased from 7.9 mm in the first epoch to 7.04 mm in the last epoch (p=<0.01). The prevalence of a short cervix ≤25 mm was 2.2% (n=399/18,289), ≤20 mm was 1.2% (224/18,289), and ≤15 mm was 0.9% (162/18,289). This prevalence varied only for CL ≤25 mm from 3.02% (88/2,907) in the first epoch to 1.77% (64/3,615) in the last epoch (p=0.0009). There was a variation in CL measurements among operators (mean 3.3 mm). Sonographers with less than one year of experience had a lower success rate for completing TV-CL examinations than more experienced sonographers (80.8% vs. 85.8%; p <0.03). In general 77% (27/35) of operators had a success rate ≥80% for completing TV-SL scans.

LIMITATIONS

Characteristics of individuals who accepted versus those who declined TV-CL were not compared; CL values were not correlated with clinical outcomes.

CONCLUSIONS

During the first six months after implementation of a universal cervical length screening program, there was greater variation in CL measurements, lower acceptance for a TV ultrasound, and a higher number of women diagnosed with a CL ≤25 mm, as compared to subsequent epochs. After the first six months, these metrics improved and remained stable. Most operators improved their performance over time; however, there were a few with a low success rate for TV-CL, and others who systematically over or under estimate CL measurements.