The CUSUM test applied in prospective nuchal translucency quality review

Real-time quality control of nuchal translucency measurements using the exponentially weighted moving average chart

OBJECTIVE

The establishment of ongoing audits for first-trimester nuchal translucency (NT) measurements is of paramount importance. The exponentially weighted moving average (EWMA) chart has been published as an efficient tool for NT quality control with the advantages of being suitable for real-time long-term monitoring. This study aimed to assess the efficacy of real-time NT quality control using EWMA charts.

MATERIALS AND METHODS

This was an ongoing prospective study conducted from January 2011 to December 2017 at the Centre for Fetal Medicine Gennet in Prague. The quality of NT measurements was assessed using the NT retrospective distribution parameters and EWMA charts, and the results were presented to the sonographers during collective meetings.

RESULTS

Overall, 28,928 NT measurements obtained from six sonographers were eligible for the study. Looking at individual EWMA charts, we observed four main outcomes. First, there was a clear improvement in the performance of sonographers with initially poor performances. Second, the performance of sonographers with an initially satisfactory quality was maintained. Third, there was an observed deterioration of the performance without the audits. Last, the sonographers appreciated an unequivocal and straightforward graphical presentation of EWMA curves.

CONCLUSION

EWMA proved to be an efficient and suitable tool for real-time monitoring of NT quality and led to an overall improvement of the sonographers' performance.

Quality control of ultrasonography markers for Down's syndrome screening: a retrospective study by the laboratory

OBJECTIVES

Quality control of ultrasonography markers is necessary to ensure greater efficacy of prenatal aneuploidy screening. The aim of this study was to assess the quality of the crown-rump length (CRL) and nuchal translucence (NT) measurement accuracy by the laboratory according to quality indicators.

METHODS

Retrospective observational study on 4,908 single-foetus pregnant women who underwent prenatal aneuploidy screening in the first trimester of pregnancy. Euploid foetuses with CRL between 45 and 84 mm were included, while those with NT≥3.5 mm were excluded. CRL measurement was considered to be accurate if the median multiples of the median (MoM) for pregnancy-associated plasma protein A (PAPP-A) was between 0.90 and 1.10. Fifteen sonographers participated in the study, six of whom comprised the control group. Systematic error for a sonographer was considered when CRL measurement was greater than ±2 mm with respect to the control group. Quality for NT was assured by means of the WHIRI method and each sonographer cumulative sum control chart (CUSUM).

RESULTS

For CRL accuracy, five sonographers underestimated the measurements, while another four overestimated them, with no statistical differences. For smaller sized foetuses, all sonographers met the established specifications. Regarding NT control, three sonographers did not meet the quality criteria for the median MoM. All sonographers met the specifications for the logarithmic standard deviation of the NT MoM levels. Thirteen sonographers met the CUSUM specifications.

CONCLUSIONS

Evaluation of a quality control of ultrasonography parameters by laboratory professionals is necessary to avoid under- or overestimation tendencies for CRL and NT measurements. CUSUM is a useful tool for the immediate correction of errors in NT measurements.

Amniocentesis learning curve using a low-cost simulation model to teach maternal-fetal medicine fellows

OBJECTIVE

To describe the learning curve for amniocentesis among Maternal-Fetal Medicine (MFM) fellows using a low-cost simulation model in Mexico.

METHODS

Fourteen first- and second-year MFM fellows with no previous experience in amniocentesis participated in this single-center prospective study from March to June of 2019. The study was approved by the Institutional Review Board at the Instituto Nacional de Perinatologia and written informed consent was obtained from all participants. After an introductory course based on a standardized technique for amniocentesis, each fellow performed this procedure using a low-cost simulation model; experienced operators supervised the procedures. Learning curves were then created using cumulative sum analysis. Thresholds for acceptable and unacceptable failure rates were defined as 10% and 25%, respectively.

RESULTS

Experienced MFM specialists evaluated 3675 procedures. On average, MFM fellows performed 263 ± 53 procedures. The mean number to achieve competence was 255 ± 53. The overall failure rate among the trainees was 16%.

CONCLUSION

We describe individual learning curves for amniocentesis among MFM fellows using a low-cost simulation model. This approach allows direct assessment of proficiency in amniocentesis before clinical practice.

Sonologist's characteristics related to a higher quality in fetal nuchal translucency measured in primary antenatal care centers

OBJECTIVE

To assess the quality of nuchal translucency (NT) measurements in primary care and the sonologist's characteristics related with a higher quality.

METHODS

The median NT expressed in multiples of the median (MoM) was calculated for each sonologist of 14 participating antenatal primary care centers of the Catalan Institute of Health. A survey to the sonologists was used to establish variables related to higher-quality measurements.

RESULTS

The median NT MoM obtained in 16 448 NT measurements, performed by 102 sonologists, was 0.94 MoM. NT underestimation was observed in 46% of the sonologists. Underestimation were less frequent among professionals who performed more than 230 ultrasounds per year (26% vs 53%;p = .022), those who completed the online Fetal Medicine Foundation (FMF) course (22% vs 54%; p = .021), and those who were subject to periodic audits (24% vs 56%; p = .021). Underestimation rate decreased from 60%, to 33% and 14% with the increase of the years of experience from less than 5 years, to 6 to 15 years and more than 15 years of experience, respectively (p = .029).

CONCLUSIONS

Higher-quality measurements were demonstrated in sonologists who performed more ultrasounds per year, those with more years of scanning experience, those who completed the online FMF course, and those periodically audited.

Developing a quality assurance program for transvaginal cervical length measurement at 18-21 weeks' gestation

BACKGROUND

Preterm birth is the leading cause of death in children under the age of five years. Transvaginal cervical length (TVCL) assessment can be used to predict preterm delivery risk at the mid-trimester scan. To optimise the screening tool, developing and maintaining quality standards is important.

AIMS

To develop an Australian reference range for TVCL at 18.0-21.0 weeks' gestation, quality standards for measurement and audit mechanisms for ultrasound operators.

MATERIALS AND METHODS

A retrospective audit was performed of consecutive patients scanned at 18.0-21.0 weeks' gestation. Each TVCL measurement ultrasound image was reviewed, and exclusions were made based on a defined set of quality criteria. Fractional polynomial Bayesian methodology was used to establish a reference range. Central tendency, dispersion plots and cumulative sum charts for operators in the original reference range cohort were created. These plots were then applied to a second validation cohort of operators to establish the efficacy of this quality assurance audit tool.

RESULTS

Median TVCL from 1031 participants was 36.0 mm (interquartile range 32.7-40.0 mm), which was independent of gestational age. The quality audit tool was applied to 15 operators from the reference cohort with a mean cervix length multiple of the median of 1.01 and a mean SD log10 cervix length multiple of the median of 0.06. Of the 22 operators in the validation cohort, 20 (90.9%) demonstrated ideal or acceptable central tendency results, and 19 (86.4%) remained in the appropriate cumulative sum zone.

CONCLUSION

An Australian cervix length measurement reference range at 18.0-21.0 weeks' gestation has been developed along with a validated quality assurance audit tool for ultrasound operators.

Impact of a shift in nuchal translucency measurements on the detection rate of first-trimester Down syndrome screening: A population-based study

OBJECTIVE

To assess the distribution of nuchal translucency (NT) measurements following a national policy without credentialing and its impact on first-trimester Down syndrome screening (DSS) detection rate.

METHOD

All first-trimester DSS data recorded in France (2010-2014) were collected by the laboratories in charge via an Internet database (https://www.bionuqual.org/echo.php). There was no minimal requirement for image quality to allow sonographers to enter the screening process. A subgroup of DSS with complete DS follow-up corresponded to 1614 sonographers. Based on the distribution of maternal age, DS detection rate was calculated and split as a function of the distribution of NT multiple of the median (MoM).

RESULTS

Four thousand nine hundred forty-three sonographers performed 2,337,372 NT measurements. Median NT expressed in MoM was 0.83. Screenings with complete follow-up consisted of 197,417 screenings, in which DSS detection rates were respectively 70.4%, 70.9%, 79.4%, 87.7%, and 79.5% for the following median NT MoM ranges: <0.7, 0.70 to 0.79, 0.80 to 0.89, 0.90 to 0.99, and >0.99 (trend χ = 12.21; P = .0158).

CONCLUSION

In France, following a policy of quality assessment without standardized credentialing, the distribution of NT measurements did not fit the expected distribution. Down syndrome detection rate was 10% lower in screenings by sonographers with a median NT < 0.80 MoM.

Assessing quality standards in measurement of uterine artery pulsatility index at 11 to 13 + 6 weeks' gestation

OBJECTIVES

To assess the effect of audit and feedback on the performance of first-trimester uterine artery pulsatility index (UtA-PI) measurement, to determine whether operator experience affects performance and whether an operator's measurement profile affects the screen-positive rate for early-onset pre-eclampsia (PE).

METHODS

This was a prospective cohort study in which UtA-PI measurements were collected between 11 to 13 + 6 weeks' gestation by 12 operators and were entered into individualized calculators to convert them to multiples of a locally-derived median (MoM). Individual sonographer cumulative sum (CUSUM) and target charts were generated to assess central tendency and dispersion to identify systematic measurement errors and deviation from expected measurement performance. Six of the operators received regular feedback whilst the remaining six received no feedback. Each group consisted of four experienced operators and two relatively inexperienced operators. The average MoM for each operator was compared with their respective screen-positive rates for early-onset PE.

RESULTS

The group that received feedback performed better than that which received none, with results more closely matching the expected measurement distribution. UtA-PI measurements were comparable between the experienced and inexperienced sonographers (mean log10 lowest PI MoM, -0.0089 vs 0.0124, respectively); however the inexperienced sonographers had a higher overall screen-positive rate for early-onset PE (10.0% vs 2.7%, respectively). There was a significant positive correlation between the mean MoM for each operator and the screen-positive rate (r = 0.63).

CONCLUSIONS

CUSUM and target graphs are an effective method of audit for first-trimester UtA-PI measurement. Feedback to operators resulted in improved measurement performance, which will ultimately result in improved screening accuracy for PE.

Calibrated breast density methods for full field digital mammography: a system for serial quality control and inter-system generalization

PURPOSE

The authors are developing a system for calibrated breast density measurements using full field digital mammography (FFDM). Breast tissue equivalent (BTE) phantom images are used to establish baseline (BL) calibration curves at time zero. For a given FFDM unit, the full BL dataset is comprised of approximately 160 phantom images, acquired prior to calibrating prospective patient mammograms. BL curves are monitored serially to ensure they produce accurate calibration and require updating when calibration accuracy degrades beyond an acceptable tolerance, rather than acquiring full BL datasets repeatedly. BL updating is a special case of generalizing calibration datasets across FFDM units, referred to as cross-calibration. Serial monitoring, BL updating, and cross-calibration techniques were developed and evaluated.

METHODS

BL curves were established for three Hologic Selenia FFDM units at time zero. In addition, one set of serial phantom images, comprised of equal proportions of adipose and fibroglandular BTE materials (50/50 compositions) of a fixed height, was acquired biweekly and monitored with the cumulative sum (Cusum) technique. These 50/50 composition images were used to update the BL curves when the calibration accuracy degraded beyond a preset tolerance of ±4 standardized units. A second set of serial images, comprised of a wide-range of BTE compositions, was acquired biweekly to evaluate serial monitoring, BL updating, and cross-calibration techniques.

RESULTS

Calibration accuracy can degrade serially and is a function of acquisition technique and phantom height. The authors demonstrated that all heights could be monitored simultaneously while acquiring images of a 50/50 phantom with a fixed height for each acquisition technique biweekly, translating into approximately 16 image acquisitions biweekly per FFDM unit. The same serial images are sufficient for serial monitoring, BL updating, and cross-calibration. Serial calibration accuracy was maintained within ±4 standardized unit variation from the ideal when applying BL updating. BL updating is a special case of cross-calibration; the BL dataset of unit 1 can be converted to the BL dataset for another similar unit (i.e., unit 2) at any given time point using the 16 serial monitoring 50/50 phantom images of unit 2 (or vice versa) acquired near this time point while maintaining the ±4 standardized unit tolerance.

CONCLUSIONS

A methodology for monitoring and maintaining serial calibration accuracy for breast density measurements was evaluated. Calibration datasets for a given unit can be translated forward in time with minimal phantom imaging effort. Similarly, cross-calibration is a method for generalizing calibration datasets across similar units without additional phantom imaging. This methodology will require further evaluation with mammograms for complete validation.

Cumulative sum plots and retrospective parameters in first-trimester ductus venosus quality assurance

OBJECTIVE

This study aimed to evaluate the application of two quality assurance methods to the ductus venosus pulsatility index (DVPI), as a first-trimester aneuploidy marker, including retrospective assessment of distribution parameters and cumulative sum (CUSUM) plots.

METHODS

The DVPI was measured in 14 444 singleton fetuses at 11+0 to 13+6 weeks in two Fetal Medicine centers during a 4-year period. Sonologist-specific quality assurance distribution parameters, previously described for nuchal translucency, were assessed: the median multiples of the median (MoM), the logarithmic standard deviation of DVPI MoMs and the weekly DVPI percent decrease. Quality assurance results were compared between median MoMs and MoM-based CUSUM plots.

RESULTS

When sonologist-specific DVPI distribution parameters were retrospectively applied for quality assurance, a 1.0 median MoM, a 0.1 median logarithmic standard deviation and a 3.4 median weekly DVPI drop percentage were observed. CUSUM plots showed good agreement with 0.9-1.1 MoMs range for median MoM, in the assessment of sonologist-specific performances.

CONCLUSION

Retrospective and prospective DVPI quality assurance methods appear to be applicable to DVPI at 11+0 to 13+6 weeks. Its use should be encouraged if DVPI is to be added to first-trimester Down syndrome or cardiac defects screening.

Analysis of quality of nuchal translucency measurements: its role in prenatal diagnosis

Objective. Quantitative analysis of the quality of nuchal translucency (NT) measurements. Methods. First-trimester combined screening for Down syndrome was performed to all pregnant women attended in our Department from October 2003 to November 2009. NT was measured according to the Fetal Medicine Foundation (FMF) criteria by 20 trained obstetricians. The performance of NT measurements was retrospectively analyzed with regard to several quality control standards. Accuracy according to experience, professional profile, crown rump length (CRL) values, and FMF certification was statistically tested. Results. A total of 14978 NT measurements were assessed. (1) The mean operator-specific median NT-MoM values was 0,98. (2) Mean percentage of cases >95th and <5th centiles were 5,0% and 4,2%, respectively. (3) Logarithmic mean and SD of the NT MoM values were 0,00 and 0,13, respectively. (4) The DR for trisomy 21 at screening time was 90,7% for a FPR of 6,7% for standard screening strategy. (5) According to Cumulative SUM (CUSUM) figures, the performance was more acceptable in FMF-certified operators. Conclusion. Overall, quality standards show optimal NT measurements in our unit. Operator experience, a dedicated profile to fetal medicine, CRL over 60 mm, and FMF certification have a significant positive impact on the quality standards.