Continuous independent quality control for fetal ultrasound biometry provided by the cumulative summation technique

Simulator, machine learning, and artificial intelligence: Time has come to assist prenatal ultrasound diagnosis

In this Commentary authors investigated and extended the role of simulator in assisting obstetric sonographers in training program. The interconnection of different digitalized technologies such as digital data, artificial neuronal and convolutional networks, machine and deep learning, telemedicine, and output are discussed and contribute to the generation of artificial intelligence.

A pilot study: Application of paired ultrasound simulator training in obstetrics

OBJECTIVE

Medical trainees in obstetrics must develop competency in fetal ultrasonography. To date, no studies have utilized ultrasound simulator training for basic fetal anatomy with paired didactic coursework. We hypothesize that ultrasound simulator training with paired didactics improves medical trainee competency in fetal ultrasonography.

METHODS

A prospective observational study was performed at a tertiary care center during the 2021-2022 academic year. Medical trainees in obstetrics without prior simulator experience could participate. Participants completed ultrasound simulator training with standardized paired didactics and subsequent real-time patient scanning. All images were reviewed by the same physician for competency. Trainees completed 11-point Likert scale surveys at three time points: pre-simulator, post-simulator, and post-real-time patient scanning. Two-tailed student's t-tests with 95% confidence intervals were performed, and p-values <0.05 were considered significant.

RESULTS

Of the 26 trainees that completed the training, 96% reported that simulation positively impacted their confidence and ability to perform real-time scanning of patients. Self-reported knowledge of fetal anatomy, ultrasound techniques, and application to clinical obstetrics all significantly increased after simulator training (p < 0.01).

CONCLUSIONS

Paired ultrasound simulation with didactic instruction significantly improves medical trainees' knowledge of fetal anatomy and ability to perform fetal ultrasonography. Implementing an ultrasound simulation curriculum may become an essential tool for obstetric residency programs.

Laparoscopic sentinel lymph node mapping with indocyanine green in endometrial cancer: surgeon's learning curve (cumulative sum analysis)

OBJECTIVES

To evaluate surgeons' learning curves for laparoscopic sentinel lymph node biopsy in endometrial cancer.

METHODS

A prospective observational study was performed at the Oncogynecology Center, Lithuanian University of Health Sciences Hospital, from March 2018 to October 2022. Participating surgeons had no previous experience of laparoscopic sentinel lymph node biopsy with indocyanine green tracer. Cumulative sum analysis was used to create learning curves for the performance of eight surgeons, based on a specific result over a time period. Two different cumulative sum plots were made for each surgeon: successful bilateral sentinel lymph node mapping and removal of sentinel lymph node specimens containing actual lymphatic tissue.

RESULTS

190 patients were included. The overall rate of sentinel lymph node mapping was 89.5%: successful bilateral mapping was achieved in 134 (70.5%) patients, while in 36 (19%) patients sentinel lymph nodes were mapped unilaterally. The bilateral detection rate significantly improved in later study periods (from 59.3% in the first year to 85.0% in the last year; p=0.03). Analysis of the performance of the surgeons for bilateral sentinel lymph node mapping showed that the cumulative sum plot crossed the H₀ limit line after 13 consecutive successful bilateral sentinel lymph node biopsies, indicating an acceptable level of competence to achieve the bilateral detection rate of at least 75%. This was accomplished by only one surgeon after 30 surgeries. Analysis of the performance of the surgeons for identification and removal of specimens containing histologically confirmed lymphatic tissue showed that the cumulative sum plots crossed the H₀ limit line after six consecutive successful sentinel lymph node removals. This was accomplished by most of the surgeons (5 of 8).

CONCLUSION

At least 30 procedures of indocyanine green traced laparoscopic sentinel lymph node biopsy were needed to reach an acceptable level of competence for a bilateral sentinel lymph node detection rate of at least 75%.

TRIAL REGISTRATION NUMBER

ACTRN12619000979156.

Organ-specific learning curves of sonographers performing first-trimester anatomical screening and impact of score-based evaluation on ultrasound image quality

INTRODUCTION

First-trimester anatomical screening (FTAS) by ultrasound has been introduced in many countries as screening for aneuploidies, but also as early screening for fetal structural abnormalities. While a lot of emphasis has been put on the detection rates of FTAS, little is known about the performance of quality control programs and the sonographers' learning curve for FTAS. The aims of the study were to evaluate the performance of a score-based quality control system for the FTAS and to assess the learning curves of sonographers by evaluating the images of the anatomical planes that were part of the FTAS protocol.

METHODS

Between 2012-2015, pregnant women opting for the combined test in the North-Netherlands were also invited to participate in a prospective cohort study extending the ultrasound investigation to include a first-trimester ultrasound performed according to a protocol. All anatomical planes included in the protocol were documented by pictures stored for each examination in logbooks. The logbooks of six sonographers were independently assessed by two fetal medicine experts. For each sonographer, logbooks of examination 25-50-75 and 100 plus four additional randomly selected logbooks were scored for correct visualization of 12 organ-system planes. A plane specific score of at least 70% was considered sufficient. The intra-class correlation coefficient (ICC), was used to measure inter-assessor agreement for the cut-off scores. Organ-specific learning curves were defined by single-cumulative sum (CUSUM) analysis.

RESULTS

Sixty-four logbooks were assessed. Mean duration of the scan was 22 ± 6 minutes and mean gestational age was 12+6 weeks. In total 57% of the logbooks graded as sufficient. Most sufficient scores were obtained for the fetal skull (88%) and brain (70%), while the lowest scores were for the face (29%) and spine (38%). Five sonographers showed a learning curve for the skull and the stomach, four for the brain and limbs, three for the bladder and kidneys, two for the diaphragm and abdominal wall and one for the heart and spine and none for the face and neck.

CONCLUSION

Learning curves for FTAS differ per organ system and per sonographer. Although score-based evaluation can validly assess image quality, more dynamic approaches may better reflect clinical performance.

Competency Assessment in Focused Cardiac Ultrasound—Can the Use of Sequential Testing Help Tailor Training Requirements?

Varying numbers of scans are required by different professional bodies before focused cardiac ultrasound (FCU) competence is assumed. It has been suggested that innovation in the assessment of FCU competence is needed and that competency assessment needs to be more individualized. We report our experience of how the use of sequential testing may help personalize the assessment of FCU competence.

The impact of individual sonographer variation on the detection of small for gestational age fetuses using a third trimester growth scan

OBJECTIVES

Detection of small for gestational age (SGA) fetuses in a third trimester ultrasound could be affected by variation in sonographer performance.

METHODS

Retrospective analysis of all singleton, non-anomalous ultrasound examinations between 35⁺⁰ -36⁺⁶ weeks gestation, in a single institution where a universal 36-week scan is offered. Screen positive was defined as estimated fetal weight (EFW) <10th centile; SGA was birthweight <10th centile. Individual sonographers' distributions of head circumference (HC), abdominal circumference (AC) and femur length (FL) were used to assess sonographers' screen positive rate (SPR), detection rate (DR) and true positive rate (TPR). Univariate and multivariate regression analysis was performed to assess the association between the sonographers' mean and SD (SD) for HC, AC, FL and their SPR, DR and TPR.

RESULTS

There were 27 sonographers performing more than 50 examinations per year, a total of 5691 scans. The mean incidence of SGA was 10.0%. For an overall SPR of 9.4%, the overall DR was 43.8% (95% CI: 39.6% - 48.1%) and the overall TPR was 46.5% (95% CI: 42.9% - 50.2%). Higher AC scatter (SD difference up to 11.6 mm) was associated with higher SPR (P = 0.001). Lower mean FL (difference up to 3.6 mm) was associated with higher SPR (P = 0.003) and higher DR (P = 0.002). As a result, DR varied amongst different sonographers between 14.3% and 85.7% and TPR varied between 8.3% and 100.0%.

CONCLUSIONS

Monitoring of individual AC and FL distributions is a simple and effective tool for institutional quality assurance.

Quality assessment of ultrasonic foetal biometry during the IUGR Risk Selection (IRIS) trial: A cross sectional study

OBJECTIVE

Intrauterine growth restriction is a major risk factor for perinatal morbidity and mortality. Ultrasonic foetal biometry is an important tool to monitor foetal growth. Therefore, the quality of these biometry scans is vital to achieve good diagnostic accuracy. We assessed the quality of foetal biometry during a nationwide trial and explored its association with sonographer's characteristics.

METHODS

Four scans from every sonographer (n = 154), performed at 29 and 35 weeks gestational age were collected. Two assessors scored these scans according to a national audit system. A quality score ≥ 65% was considered 'adequate'. We compared the quality scores per scoring criterion (i.e. foetal head measurements, abdominal circumference and femur length with regard to magnification, correctness of the plane and calliper placement) and gestational age. We analysed the associations between characteristics of the sonographers and their scores. In a subsample of scans of 30 sonographers we determined the interrater agreement on the quality scores given by the two assessors independently.

FINDINGS

The mean score was 81.3%. Thirteen sonographers (8.4%) failed to achieve 'adequate quality'. Scores for femur length (83.8%) were significantly higher than those for head (77.9%) and abdominal circumference (78.6%) (both P < 0.05). Scores for correctness of the plane (73.4%) were lower than those for magnification (81.2%) and calliper placement (85.7%) (both P < 0.05). Gestational age did not affect the quality scores. Only the number of scans performed in the previous year was positively associated with the scores (β = 0.01; P < 0.05). The mean interrater difference in quality scoring was 11.1%, with 77.6% agreement on scans of 'adequate quality', but with no agreement on scans with 'insufficient quality'.

KEY CONCLUSIONS AND IMPLICATIONS FOR PRACTICE

Most sonographers achieved an 'adequate quality' score. Highest quality scores were attained for femur length, lowest quality scores for the correct plane. The number of scans one performs is associated with the quality scores, yet the minimum number of scans to perform for guaranteed quality still needs to be determined. Further research is needed to develop a standardized method to assess and maintain good ultrasonic foetal biometry quality.

Influence of newborn head circumference and birth weight on the delivery mode of primipara: what is more important?

Objectives Aim of the study was to analyze the impact of head circumference (HC) and birth weight (BW) on the delivery mode and delivery outcomes. Methods Study population consisted of pregnancy, delivery and newborn data from 1,762 women, who delivered between 2004 and 2016 at University Hospital of Zurich (UHZ). Odds ratio (OR) with 95% confidence intervals (CI) were calculated for mode of delivery. Newborns were sorted into four groups according HC or BW. To evaluate the association between HC and delivery outcome, a descriptive analysis was performed. In addition reference charts of newborn HC at term were constructed. Results OR for instrumental delivery (ID) was 2.37 (CI 95%, 1.63-3.46), for C-Section (CS) 3.74 (CI 95%, 1.49-9.37) when HC >36 cm. OR for ID was 1.59 (CI 95%, 1.02-2.50), for CS 3.18 (CI 95% 1.08-9.350) when BW was >4,000 g. OR for ID was 2.15 (95% CI, 1.69-2.73), for CS 1.93 (95% CI, 0.89-4.18) when HC ≥36 cm and BW <4000 g. OR for ID was 2.23 (95% CI, 1.35-3.67), for CS 4.39 (95% CI, 1.48-12.99) when HC ≥36 cm and BW ≥4,000 g. HC ≥36 cm was defined as large in our study. Mothers with higher age and body mass index delivered babies with larger HC (p<0.05). Blood loss and duration of expulsion period and BW was associated with larger HC (p<0.05). Conclusions The rate of ID and CS increased in case of a larger HC and greater BW. However, the main prognostic factor for ID was size of HC: ≥36 cm, but not macrosomia.

Comparison of Current Swiss Fetal Biometry Reference Charts with Reference Charts from 1999. Are Fetuses Getting Bigger?

PURPOSE

 To create current fetal biometry reference ranges and to compare them with references published in 1999, from the same local area in order to generate data for secular trend in fetal size.

MATERIALS AND METHODS

 Applying the same methodology as previously published, we calculated reference ranges for biparietal diameter (BPD), occipitofrontal diameter (OFD), head circumference (HC), abdominal circumference (AC) and femur length (FL) in 7863 patients examined at the obstetric clinics in a cross-sectional, prospective study in a university setting from January 2008 to December 2014. In order to compare the new reference ranges with our previously published data, we used Z-Scores and displayed the pick-up of fetal biometry data below the 5^th and above the 95^th percentile using the previously published reference charts.

RESULTS

 The comparison of the charts showed a minimal but clinically relevant increase in mean fetal body measures (BPD, HC, AC). Applying the 1999 charts to the new dataset, we would classify only 162 of 339 fetuses (47.8 %) to be correctly below the 5^th percentile for AC and only 134 of 349 (38.4 %) fetuses were correctly below the 5^th percentile for HC. On the other hand, the 1999 charts classified 426 instead of 332 fetuses to be above the 95^th percentile for AC, which means an overestimation of 28.3 %.

CONCLUSION

 Applying a similar methodology, study collective and clinical setting, our new charts showed clinically relevant differences compared to the 1999 charts. The data suggest that within one generation fetuses are getting bigger and regular updates of fetal reference charts are needed.

Using Simulation to Define the Learning Curve for the Digital Cervical Examination

INTRODUCTION

To assess how skill in the digital cervical examination is acquired in medical students.

METHODS

In a longitudinal study, medical students completed 200 simulated cervical examinations. We performed regressions of each student's cumulative accuracy using the linear, power, and sigmoidal models to determine best fit. We also constructed multilevel models to determine the influence of dilation and effacement on accuracy and to determine whether the starting level and rate of learning varied between individuals. To assess skill decay, we assessed accuracy at 1, 2, and 5 months after training. We defined the amount of sustained accuracy needed to achieve competence using cumulative summation analyses and determined the amount of practice needed to reach this level of skill.

RESULTS

Twenty-five medical students participated. The median (interquartile range) of cumulative accuracy at the end of the study was 69% (65-78) for dilation and 80% (76-91) for effacement. The sigmoidal model had the best fit. All students achieved competence during the study. The multilevel models showed that accuracy decreased with higher dilation and lower effacement and found that starting level and rate of learning varied between individuals. Maximal accuracy in both dilation and effacement was seen after 150 repetitions. Accuracy of the medical students persisted for 1 month for dilation and 2 months for effacement. The average±SD number of repetitions needed to achieve competence was 89±46 (range 35-195) for dilation and 48±38 (range 11-174) for effacement.

DISCUSSION

Based on the variability in skill between individuals and the rate of skill acquisition and decay, we feel that a competence-based rather than time-based approach is most appropriate, that trainee performance should be monitored both during and after training, and that 150 repetitions, or more, should be included in any digital cervical examination simulation regimen.

Learning curve-cumulative summation analysis of visual estimation of left ventricular function in novice practitioners: A STROBE-compliant article

The aim of the present study was to determine the value of cumulative summation (CUSUM) analysis in assessing the proficiency of novice practitioners in estimating the left ventricular ejection fraction (EF).Seven novice practitioners with no echocardiography experience were recruited in this observational study. Each practitioner assessed EF from echocardiographic video files of 100 cases, one by one, and received feedback and teaching. We obtained a CUSUM score through comparison of the gold standard values of EF and the EF values determined by the practitioners. Then, the practitioners underwent the same test 4 weeks later, except without feedback and teaching, using echocardiographic video files from 100 other cases.The mean number of visual estimation cases required to pass the learning curve (LC)-CUSUM test was 56.3 ± 9.1 (95% CI 47.8-64.7). The LC-CUSUM average of the 7 novice practitioners showed improvement in visual estimation skill, with an average acceptable level achieved after a mean experience of 55 cases. In the test performed after 4 weeks, 5 of the 7 novice practitioners showed significantly good overall agreement. All novice practitioners had a kappa coefficient greater than .8, and significant and almost perfect agreement was observed. All the participants exhibited a percentage of correct answers greater than 81%.We found that the novice practitioners could acquire an acceptable level of skill for estimating EF with short-term, self-learning-focused echocardiographic training.

Improving patient safety during introduction of novel medical devices through cumulative summation analysis

OBJECTIVE The aim of this study was to implement cumulative summation (CUSUM) analysis as an early-warning detection and quality assurance system for preclinical testing of the iSYS1 novel robotic trajectory guidance system. METHODS Anatomically accurate 3D-printed skull phantoms were created for 3 patients who underwent implantation of 21 stereoelectroencephalography electrodes by surgeons using the current standard of care (frameless technique). Implantation schema were recreated using the iSYS1 system, and paired accuracy measures were compared with the previous frameless implantations. Entry point, target point, and implantation angle accuracy were measured on postimplantation CT scans. CUSUM analysis was undertaken prospectively. RESULTS The iSYS1 trajectory guidance system significantly improved electrode entry point accuracies from 1.90 ± 0.96 mm (mean ± SD) to 0.76 ± 0.57 mm (mean ± SD) without increasing implantation risk. CUSUM analysis was successful as a continuous measure of surgical performance and acted as an early-warning detection system. The surgical learning curve, although minimal, showed improvement after insertion of the eighth electrode. CONCLUSIONS The iSYS1 trajectory guidance system did not show any increased risk during phantom preclinical testing when used by neurosurgeons who had no experience with its use. CUSUM analysis is a simple technique that can be applied to all stages of the IDEAL (idea, development, exploration, assessment) framework as an extra patient safety mechanism. Further clinical trials are required to prove the efficacy of the device.

Transvaginal sonography in the diagnosis of deep infiltrating endometriosis: A review

This review summarizes the current evidence regarding the diagnostic accuracy of sonography (US) in women with deep infiltrating endometriosis (DIE). It is well known that transvaginal ultrasound (TVUS) can detect ovarian endometriomas with a high degree of sensitivity. In recent years, US has also been used to detect DIE. In the hands of an experienced sonologist, the sensitivity and specificity of TVUS in the detection of DIE is comparable to those of MRI. TVUS can eliminate the need for an MRI in the majority of patients and reduce the need for diagnostic laparoscopy, proving to be an important tool in preoperative planning. © 2016 Wiley Periodicals, Inc. J Clin Ultrasound 45:313-318, 2017.

Longitudinal z Score Distribution in Sonographic Fetal Biometry: Influence of Examiner and Experience

OBJECTIVES

To analyze the influence of examiners and their experience on the quality of biometric measurements via the evolution of z scores in a longitudinal multicenter study.

METHODS

This retrospective study included 4607 sonographic fetal biometric examinations performed by 18 examiners at the beginning of their sonography training. To analyze the quality of biometric measurements, z scores comparing the individual measurements with the expected values from a standard population were computed. To investigate the effect of examiners and their experience, we applied a multivariable regression analysis via generalized additive mixed models.

RESULTS

Mean z scores for biparietal diameter, abdominal circumference, and femur length were statistically different from the expected value of 0 (P < .001); no significant differences were found for head circumference measurements. Regression analyses showed a significant effect of the number of examinations on the distribution of z scores for each type of measurement. This effect yielded z score values close to the expected value of 0 at 100 to 200 examinations for all biometric measurements, indicating good consistency with the distribution of values in the reference population. Near the end of the study period, an increasing tendency toward either overestimation (head circumference) or underestimation (biparietal diameter, abdominal circumference, and femur length) was observed.

CONCLUSIONS

A longitudinal analysis of the z score distribution for quality control of biometry is feasible. A prospective and automatized use of this technique could help identify potential systematic errors and therefore improve the detection rate for high-risk pregnancies.

Longitudinal Assessment of Examiner Experience and the Accuracy of Sonographic Fetal Weight Estimation at Term

OBJECTIVES

To evaluate the influence of examiner experience on the accuracy of sonographic weight estimation and to further analyze examiners' individual learning curves.

METHODS

In this multicenter study, 4613 sonographic weight estimations performed by 18 examiners at the beginning of their ultrasound training were included. To assess the effect of experience on the accuracy of weight estimation, a multivariable mixed regression model analysis was performed, with percentage error and absolute percentage error as outcome variables and the examiner, the examiner's experience (number of examinations), birth weight, gestational age, scan-to-delivery interval, and maternal body mass index as fixed effects and the perinatal center as random intercepts. To further analyze the individual learning curves of the examiners, the cumulative summation technique was used.

RESULTS

Regression analyses showed a significant influence of the number of examinations on the accuracy of sonographic weight estimation after adjustment for the above-mentioned parameters (P < .001). A typical learning curve with improving accuracy was found until approximately 200 examinations. Between 200 and 300 examinations, the diagnostic performance started to deteriorate again, with a continuous decrease until the end of the study period. Cumulative summation charts representing individual learning curves varied greatly between different examiners.

CONCLUSIONS

These findings indicate the great importance of continuous quality control systems in sonographic weight estimation.

Unconditional reference values for the amniotic fluid index measurement between 26w0d and 41w6d of gestation in low-risk pregnancies

OBJECTIVE

To establish reference values for the amniotic fluid index (AFI) measurement between 26w0d and 41w6d of gestation in a Brazilian population.

METHODS

We performed a cross-sectional study with 1984 low-risk singleton pregnant women between 26w0d and 41w6d of gestation. AFI was measured according to the technique proposed by Phelan et al. Maternal abdomen was divided into four quadrants using the umbilicus and linea nigra as landmarks. Single vertical pocket in each quadrant was measured and the AFI was generated by the sum of these four values without umbilical cord or fetal parts. All ultrasound exams were performed by only two experienced examiners. AFI was expressed as median, interquartile range, mean and ranges in each gestational age (GA) interval. Polynomial regressions were performed to obtain the best fit with adjustment by the determination coefficient (R(2)).

RESULTS

Mean of AFI ranged from 14.0 ± 4.1 cm (range, 9.7-14.0) at 26w0d to 8.3 ± 4.7 cm (range, 1.9-16.5) at 41w6d, respectively. The best polynomial regression fit curve was a first-degree: AFI = 16.29-0.125*GA (R(2) = 0.01). According the scatterplot, AFI values practically did not vary with advancing GA.

CONCLUSION

Reference values for the AFI measurement between 26w0d and 41w6d of gestation in a low-risk Brazilian population were established.

The knee arthroscopy learning curve: quantitative assessment of surgical skills

PURPOSE

To assess orthopaedic trainees performing diagnostic knee arthroscopies and evaluate procedural competence using a novel statistical method, the Cumulative Summation Test for Learning Curve (LC-CUSUM).

METHODS

Twenty orthopaedic trainees in postgraduate year (PGY) 1 through 5 performed diagnostic knee arthroscopy and were evaluated intraoperatively with a validated 10-point knee task-specific checklist (TSCL) and 50-point global rating scale (GRS). A score of 40 points or greater (of 50 points) for the GRS and 8 points or greater (of 10 points) for the knee TSCL was considered to indicate a successful procedure. For the LC-CUSUM analysis, adequate performance was defined as a 10% failure rate, inadequate performance was defined as a 30% failure rate, and an acceptable deviation from adequate performance was defined as 10%. A limit h equal to 1.6 was selected to give a true-discovery rate of 90% and a false-discovery rate of 10% over 50 procedures.

RESULTS

A total of 340 consecutive procedures were performed by 20 trainees during the 12-month study period. The cumulative number of arthroscopic procedures performed by trainees before study start increased with increasing PGY. The median number of arthroscopic procedures performed per trainee was 16.5 (interquartile range, 14 to 21.75). Competency in knee arthroscopy for the TSCL was achieved by 8 trainees (40%), after a median of 16 procedures (interquartile range, 13 to 20), and for the GRS by only a single trainee (5%), after 14 procedures. Threshold-adjusted curves stratified by PGY level enabled multiple trainees to achieve competency for both the TSCL and GRS.

CONCLUSIONS

The LC-CUSUM can be successfully applied to knee arthroscopy to provide an individualized assessment of performance and quantitatively demonstrate competency for basic arthroscopic tasks.

CLINICAL RELEVANCE

The LC-CUSUM is an effective method to evaluate procedure competence in arthroscopic training and can provide objective feedback and benchmarks in the learning phase.

Individualized embryo transfer training: timing and performance

STUDY QUESTION

How long is the individualized training and the stability of competence for the embryo transfer (ET) technique?

SUMMARY ANSWER

The embryo transfer technique is easy-to-learn, hardly unlearned, and training should be individualized by monitoring with learning curve-cumulative summation (LC-CUSUM) curves.

WHAT IS KNOWN ALREADY

Like many medical procedures, embryo transfer is an operator-dependent technique. Individualized or standardized training of these medical procedures should be monitored to determine when competence is acquired.

STUDY DESIGN, SIZE, DURATION

This prospective, monocentric study involving five embryo transfer trainees was carried out between August 2011 and November 2012.

PARTICIPANTS/MATERIALS, SETTING, METHODS

The study was carried out in a large private clinic. Five gynaecologist trainees during their first year of assisted reproduction subspecialty performed embryo transfer for patients undergoing either fresh IVF, oocyte donor IVF, or frozen embryo transfer. There were 586 embryo transfers performed in 96 sessions of 3-10 embryo transfers each. An embryo transfer was considered successful if it gave rise to a positive pregnancy test 14 days later. LC-CUSUM and cumulative summation (CUSUM) curves were used to determine when competence was acquired and whether it was maintained over time, respectively. The length of time between two consecutive sessions was assessed for an effect on consolidation of the acquired competence.

MAIN RESULTS AND THE ROLE OF CHANCE

We observed that all five trainees became proficient in embryo transfer by procedure 15 (after procedure 15, 9, 7, 13 and 9, respectively). Once competence was achieved, one of the five trainees showed a loss of proficiency. After having acquired competence, the median pregnancy rate per embryo transfer session was significantly lower when the interval between consecutive embryo transfer sessions was ≥10 days compared with <10 days (20.0 versus 46.7%; P = 0.006).

LIMITATIONS, REASONS FOR CAUTION

The patient groups included in the study were heterogeneous (IVF, oocyte donor IVF and frozen embryo transfer) and their outcomes are very variable; thus the distribution and proportion of these groups can determine the timing of competence acquisition. Our data show that low numbers of embryo transfer are needed to acquire competence, but since a relative high percentage of embryo transfers in our practice are from oocyte donor IVF, extrapolation of the findings to other clinical context should be done with caution.

WIDER IMPLICATIONS OF THE FINDINGS

Personalized embryo transfer training is feasible and useful, allowing clinics, on one hand, to offer a maximum chances of pregnancy with fully trained personnel, and the other hand, to avoid the superfluous and costly overtraining of already proficient trainees. Furthermore, it is advisable to maintain a short interval of time between consecutive embryo transfer sessions after a trainee has acquired competence, to avoid a significant drop in the resulting pregnancy rate.

STUDY FUNDING/COMPETING INTEREST(S)

This work was supported in part by funding from Fundació Privada EUGIN. There are no conflicts of interest to declare.

Learning curve analysis of robot-assisted radical hysterectomy for cervical cancer: initial experience at a single institution

OBJECTIVE

The aim of this study was to evaluate the learning curve and perioperative outcomes of robot-assisted laparoscopic procedure for cervical cancer.

METHODS

A series of 65 cases of robot-assisted laparoscopic radical hysterectomies with bilateral pelvic lymph node dissection for early stage cervical cancer were included. Demographic data and various perioperative parameters including docking time, console time, and total operative time were reviewed from the prospectively collected database. Console time was set as a surrogate marker for surgical competency, in addition to surgical outcomes. The learning curve was evaluated using cumulative summation method.

RESULTS

The mean operative time was 190 minutes (range, 117 to 350 minutes). Two unique phases of the learning curve were derived using cumulative summation analysis; phase 1 (the initial learning curve of 28 cases), and phase 2 (the improvement phase of subsequent cases in which more challenging cases were managed). Docking and console times were significantly decreased after the first 28 cases compared with the latter cases (5 minutes vs. 4 minutes for docking time, 160 minutes vs. 134 minutes for console time; p<0.001 and p<0.001, respectively). There was a significant reduction in blood loss during operation (225 mL vs. 100 mL, p<0.001) and early postoperative complication rates (28% vs. 8.1%, p=0.003) in phase 2. No conversion to laparotomy occurred.

CONCLUSION

Improvement of surgical performance in robot-assisted surgery for cervical cancer can be achieved after 28 cases. The two phases identified by cumulative summation analysis showed significant reduction in operative time, blood loss, and complication rates in the latter phase of learning curve.

Obstetric ultrasound simulation

Obstetric ultrasound is becoming an increasingly important part of the practice of maternal-fetal medicine. Thus, it is important to develop rigorous and effective training curricula for obstetrics and gynecology residents and maternal-fetal medicine fellows. Traditionally, this training has come almost entirely from exposure to ultrasound in the clinical setting. However, with the increased complexity of modern ultrasound and advent of duty-hour restrictions, a purely clinical training model is no longer viable. With the advent of high-fidelity obstetric ultrasound simulators, a significant amount of training can occur in a non-clinical setting which allows learners to obtain significant skill prior to their first patient ultrasound encounter and obtain proficiency in a shorter period of time. In this manuscript we discuss the available obstetric ultrasound simulators and ways to construct a comprehensive ultrasound training curricula to meet the increasing demands of modern maternal-fetal medicine.

Assessment of competency in clinical measurement: comparison of two forms of sequential test and sensitivity of test error rates to parameter choice

OBJECTIVE

To assess clinical measurement competency by two sequential test formulations [resetting sequential probability ratio test (R-SPRT) and learning curve cumulative summation (LC-CUSUM)].

DESIGN

Numerical simulation and retrospective observational study.

SETTING

Obstetric ultrasound department.

PARTICIPANTS

Cohorts of 10,000 simulated trainees and 62 obstetric sonographers training in nuchal translucency (NT) measurement at the 11-14-week pregnancy scan with limited case availability.

INTERVENTION

Application of LC-CUSUM and R-SPRT to clinical measurement training.

MAIN OUTCOME MEASURES

Proportions of real trainees achieving competency by LC-CUSUM and R-SPRT, proportions of simulated competent trainees not achieving competency (Type I error), proportions of simulated incompetent trainees achieving competency (Type II error), distribution of case number required to achieve competency (run length) and frequency of resets.

RESULTS

For simulated cohorts, significant differences in run-length distribution and true test error rates were found between the R-SPRT and LC-CUSUM tests with equivalent parameters. Increasing the cases available to each trainee reduced the Type I error rate but increased the Type II error rate for both sequential tests for all choices of unacceptable failure rate. Discontinuities in the proportion of trainees expected to be test competent were found at critical values of unacceptable failure rate.

CONCLUSIONS

With equivalent parameters, the R-SPRT and LC-CUSUM formulations of sequential tests produced different outcomes, demonstrating that the choice of test method, as well as the choice of parameters, is important in designing a training scheme. The R-SPRT detects incompetence as well as competence and may indicate need for further training. Simulations are valuable in estimating the proportions of trainees expected to be assessed as competent.

Use of the Learning Curve-Cumulative Summation test for quantitative and individualized assessment of competency of a surgical procedure in obstetrics and gynecology: fetoscopic laser ablation as a model

OBJECTIVE

We sought to determine the learning curve (LC) for fetoscopic laser photocoagulation (FLP) as a model for the evaluation of training in surgical procedures.

STUDY DESIGN

A retrospective review of consecutive case series of FLP from 2 centers with 3 operators (operator I [O-I], observer trained; operator II [O-II], hands-on trained; and operator III [O-III], clinical fellow) was performed and the LC-cumulative summation (CUSUM) test was plotted.

RESULTS

The acceptable and unacceptable success rates for at least 1 fetus survival after FLP were set at 82% and 70%, respectively, from a systematic review. A total of 171 consecutive cases were performed by the 3 operators (O-I, 91; O-II, 49; and O-III, 31). From LC-CUSUM test O-I needed 60 procedures, O-II needed 20 procedures, and O-III needed 20 procedures to reach an acceptable performance rate for at least 1 survivor.

CONCLUSION

The LC-CUSUM test can be used to accurately assess the LC in a surgical procedure in obstetrics and gynecology. Hands-on trained operators exhibit a shorter LC.