Obstetric and gynecologic ultrasound curriculum and competency assessment in residency training programs: consensus report

Successful Implementation of the AIUM Standardized 4-Year Residency Ultrasound Curriculum in Obstetrics and Gynecology: Lessons Learned and Way Forward

BACKGROUND

Significant disparities in sonographic education exist in Obstetrics and Gynecology programs in the United States. To address the lack of standardization in ultrasound teaching, the American Institute of Ultrasound in Medicine (AIUM) assembled a multi-society task force in 2018 that resulted in the publication of a Consensus Report outlining a standardized ultrasound curriculum and competency assessment.

OBJECTIVES

The primary objective of the study was to implement the AUIM standardized curriculum within an Obstetrics and Gynecology residency program at a tertiary medical center, and report on the early implementation experience. The secondary objective was to assess current national practices in ultrasound training in Obstetrics and Gynecology residency programs.

STUDY DESIGN

Study design was an effectiveness-implementation hybrid study of a structured ultrasound educational program based on the 2018 AIUM Consensus Report. The theoretical portion of the curriculum consisted of 26 hours of didactic lectures distributed over 4 years. For the practical component, residents received instruction from faculty and sonographers and independently acquired the required images as part of their regular clinical work. Simulator app access was also provided for additional opportunities to practice and acquire images. Deidentified images for the required competencies were uploaded to a WhatsApp group for review. Theoretical knowledge was assessed with a series of multiple-choice exams. Practical skills were assessed through grading of images submitted through WhatsApp as well as direct observations of resident scans. At the completion of 4 years, the assessment of both theoretical knowledge and practical skills was carried out using the ISUOG Basic Training Examination. National patterns in the area of ultrasound education were assessed with a four-question electronic survey of Obstetrics and Gynecology program directors in the United States. Analysis of both portions of the study was primarily descriptive.

RESULTS

Four Obstetrics and Gynecology residents completed the curriculum spanning postgraduate years 1 through 4. The number of competencies completed ranged from 7 to 19 out of 41. All of the residents passed the theoretical and practical components of the ISUOG Basic Training Examination. For the residency program director survey portion of the study, response rate was 20% (59 of 290). Among the respondents, 55.9% reported having a structured curriculum and 34.4% reported implementing the AIUM curriculum. The two most commonly cited obstacles to implementation of structured ultrasound teaching were uncertainty with how to start the process (49.2%) and lack of faculty engagement (49.2%).

CONCLUSIONS

Implementation of the AIUM Curriculum within the framework of an Obstetrics and Gynecology residency program at a tertiary medical center is feasible. In our experimental group, completion of the curriculum resulted in passing of an internationally recognized validated assessment. Obstacles to implementation in our program included a wide range of engagement and participation among residents, and limited opportunities to obtain some of the required competencies. Identification of obstacles to implementation from the perspective of residency program directors is critical for developing targeted approaches to ensure widespread implementation. Successful standardized validated sonographic training of graduating Obstetrics and Gynecology residents is crucial for this operator-dependent modality that has tremendous implications on patient safety and clinical care.

Validity evidence for simulator-based obstetric ultrasound competency assessment tool: a multi-center study

PURPOSE

To collect validity evidence for a simulator-based obstetric ultrasound competency assessment tool (OUCAT).

METHODS

89 sonographers from three centers (XY, MC, DT), including novices (n=21), experienced trainees (n=44), and experts (n=24), participated in the competency assessment. Validity evidence of OUCAT was collected according to Standards for Educational and Psychological Testing. Content validity was ensured by reviewing guidelines and reaching expert consensus. The response process was ensured via training raters. Internal structure was explored through internal consistency, inter-rater reliability, and test-retest reliability. Relations to other variables were explored by comparing OUCAT scores of sonographers with different experience. Evidence for consequences was collected by determining the pass/fail level.

RESULTS

OUCAT included 123 items, of which 117 items could effectively distinguish novices from experts (P<0.05). The internal consistency was represented by the Cronbach's α coefficient (0.978). The inter-rater reliability was high, with XY being 0.868, MC being 0.877, and DT being 0.937 (P<0.001). Test-retest reliability was 0.732 (P=0.001). The performance of experts was significantly better than experienced trainees, and the performance of experienced trainees was significantly better than novices (70.3±10.7 vs. 39.8±15.0 vs. 20.5±10.6, P<0.001). The pass/fail level determined by contrast group method was 45 points. The passing rate of novices, experienced trainees and experts was 0% (0/21), 31.8% (14/44), and 100% (24/24), respectively.

CONCLUSION

Simulator-based OUCAT exhibits good reliability and validity in assessing obstetric ultrasound skills.

Current Utilization and Perceptions of Formal Education of Point-of-care Ultrasound in Urology

OBJECTIVE

To assess the extent of formal point-of-care ultrasound (POCUS) training, current utilization of POCUS, and contemporary perceptions of POCUS amongst urologists.

METHODS

A survey including questions regarding demographics, prior ultrasound education, current ultrasound utilization in practice/training, perceived optimal POCUS utilization, and the perception of formal ultrasound training was developed. The survey was disseminated to residency program directors (PDs) via the SAU and members of AUA subsection organizations. Data were collected via Redcap.

RESULTS

A total of 40 PDs and 159 other respondents completed the survey with approximately half (51%) in an academic practice and two-thirds (68%) with more than 10years in practice. PD response rate was 28%, and general response rate was 2%. Among all respondents, 95% (186/196) and 82% (160/194) agreed/strongly agreed formal POCUS training would be worthwhile during and after residency. Among urology residency PDs, 93% (37/40) agreed/strongly agreed that formal POCUS training is worthwhile in residency. The majority of respondents used some form of ultrasound in practice (77%, 154/199). However, only 37% (72/199) of all respondents had prior formal POCUS training, and 19% (5/26) of PDs reported formal training in their programs. Of respondents without formal training, 63% (80/127) reported interest in pursuing formal training.

CONCLUSION

POCUS is widely utilized in many practices. Yet, most urologists have not participated in formal POCUS training and most programs do not have curricula. Urologists have favorable opinions of the utility, safety, and efficacy of POCUS and desire training.

Influence of maternal body mass index on interobserver variability of fetal ultrasound biometry and amniotic-fluid assessment in late pregnancy

OBJECTIVE

To determine the interobserver reproducibility of fetal ultrasound biometric and amniotic-fluid measurements in the third trimester of pregnancy, according to maternal body mass index (BMI) category.

METHODS

This was a prospective cohort study of women with a singleton gestation beyond 34 weeks, recruited into four groups according to BMI category: normal (18.0-24.9 kg/m² ), overweight (25.0-29.9 kg/m²⁾ , obese (30.0-39.9 kg/m² ) and morbidly obese (≥ 40 kg/m² ). Multiple pregnancies, women with diabetes and pregnancies with a fetal growth, structural or genetic abnormality were excluded. In each woman, fetal biometric (biparietal diameter (BPD), head circumference, abdominal circumference (AC), femur length (FL) and estimated fetal weight) and amniotic-fluid (amniotic-fluid index (AFI) and maximum vertical pocket (MVP)) measurements were obtained by two experienced sonographers or physicians, blinded to gestational age and each other's measurements. Differences in measurements between observers were expressed as gestational age-specific Z-scores. The interobserver intraclass correlation coefficient (ICC) and Cronbach's reliability coefficient (CRC) were calculated. Bland-Altman analysis was used to assess the degree of reproducibility.

RESULTS

In total, 110 women were enrolled prospectively (including 1320 measurements obtained by 17 sonographers or physicians). Twenty (18.2%) women had normal BMI, 30 (27.3%) women were overweight, 30 (27.3%) women were obese and 30 (27.3%) women were morbidly obese. Except for AFI (ICC, 0.65; CRC, 0.78) and MVP (ICC, 0.49; CRC, 0.66), all parameters had a very high level of interobserver reproducibility (ICC, 0.72-0.87; CRC, 0.84-0.93). When assessing reproducibility according to BMI category, BPD measurements had a very high level of reproducibility (ICC ≥ 0.85; CRC > 0.90) in all groups. The reproducibility of AC and FL measurements increased with increasing BMI, while the reproducibility of MVP measurements decreased. Among the biometric parameters, the difference between the BMI categories in measurement-difference Z-score was significant only for FL. Interobserver differences for biometric measurements fell within the 95% limits of agreement.

CONCLUSION

Obesity does not seem to impact negatively on the reproducibility of ultrasound measurements of fetal biometric parameters when undertaken by experienced sonographers or physicians who commonly assess overweight, obese and morbidly obese women. © 2021 International Society of Ultrasound in Obstetrics and Gynecology.

Simulation and beyond - Principles of effective obstetric training

Simulation training provides a safe, non-judgmental environment where members of the multi-professional team can practice both their technical and non-technical skills. Poor teamwork and communication are recurring contributing factors to adverse maternal and neonatal outcomes. Simulation can improve outcomes and is now a compulsory part of the national training matrix. Components of successful training include involving the multi-professional team, high fidelity models, keeping training on-site, and focussing on human factors training; a key factor in adverse patient outcomes. The future of simulation training is an exciting field, with the advent of augmented reality devices and the use of artificial intelligence.

A Systematic Review and Meta-analysis of the Use of High-Fidelity Simulation in Obstetric Ultrasound

SUMMARY STATEMENT

There is little global consensus on how to train, assess, and evaluate skills in obstetric ultrasound. The outcomes of curricula, where present, are often based on the number of clinical cases completed, rather than objective outcomes. The central question in this review is whether simulation enhances training and prepares trainees for clinical practice. A systematic review was conducted of the currently available literature in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Studies considering the use of simulators in training or assessment of sonographers were eligible for inclusion. We conclude that simulation is best used for acquisition of technical skills and image optimization. Best outcomes are observed when simulation augments traditional learning, with a strong focus on specific, objective, and measurable skills. Integrating simulation into training curricula could allow trainees to contribute to clinical service while learning. How skills learned in a simulated environment translate to the clinic is poorly addressed by the literature.

Dimensionless squared jerk: An objective differential to assess experienced and novice probe movement in obstetric ultrasound

OBJECTIVE

Widely accepted, validated and objective measures of ultrasound competency have not been established for clinical practice. Outcomes of training curricula are often based on arbitrary thresholds, such as the number of clinical cases completed. We aimed to define metrics against which competency could be measured.

METHOD

We undertook a prospective, observational study of obstetric sonographers at a UK University Teaching Hospital. Participants were either experienced in fetal ultrasound (n = 10, >200 ultrasound examinations) or novice operators (n = 10, <25 ultrasound examinations). We recorded probe motion data during the performance of biometry on a commercially available mid-trimester phantom.

RESULTS

We report that Dimensionless squared jerk, an assessment of deliberate hand movements, independent of movement duration, extent, spurious peaks and dimension differed significantly different between groups, 19.26 (SD 3.02) for experienced and 22.08 (SD 1.05, p = 0.01) for novice operators, respectively. Experienced operator performance, was associated with a shorter time to task completion of 176.46 s (SD 47.31) compared to 666.94 s (SD 490.36, p = 0.0004) for novice operators. Probe travel was also shorter for experienced operators 521.23 mm (SD 27.41) versus 2234.82 mm (SD 188.50, p = 0.007) when compared to novice operators.

CONCLUSION

Our results represent progress toward an objective assessment of technical skill in obstetric ultrasound. Repeating this methodology in a clinical environment may develop insight into the generalisability of these findings into ultrasound education.

Reaudit of transvaginal ultrasound practice in a general gynecology clinic

PURPOSE

A recent pilot audit found that the quality of transvaginal ultrasound practice in a teaching hospital did not reflect current guidelines. This was a concern given the frequency and importance of ultrasound examinations in approaching a diagnosis. Interventional measures involved ad hoc training sessions coincident with installation of updated equipment. A reaudit was performed to assess any changes in the standard of ultrasound practice.

METHODS

Data were collected by random direct observation. To minimize the Hawthorne effect, staffs were not made aware of when active data collection took place. Observations of both patients and end user were compared with current guidelines. Data collected maintained the anonymity of users throughout. Practice was graded as: compliant, partially-compliant, or noncompliant. Descriptive statistics were used to demonstrate any change post-intervention.

RESULTS

Observations (n = 48) of completed gynecology ultrasound practice were recorded, and results described as percentage frequencies (%). Image optimization noncompliant rates declined in the reaudit from 65.1% to 41.67% (P = .07) and noncompliant rates of global examination of gynecology decreased from 60.47% to 14.58% (P < .05). Substantial improvements were seen in terms of image annotations (initial audit, 41.9% vs reaudit, 66.67%), and end user examination of bladder, vagina and cervix when indicated (initial audit, 25.6% vs reaudit, 77.08%).

CONCLUSION

Regular audit, ad hoc ultrasound training sessions and updated ultrasound equipment resulted in considerably improved compliance of transvaginal ultrasound practice in gynecology.

Ultrasound feedback training increases trainee accuracy in vaginal assessment of fetal head position in labor

OBJECTIVE

To evaluate the impact of an immediate ultrasound feedback intervention on trainee accuracy in vaginal-examination-based assessment of fetal head position.

METHODS

This was a prospective cohort study conducted at a single tertiary care center. Six third-year and six fourth-year residents in an obstetrics and gynecology residency training program were the study subjects. The third-year residents underwent a training intervention in which they assessed fetal head position by transvaginal digital examination and then received immediate feedback through ultrasound demonstration of the actual position. All examinations were performed in women with a singleton gestation ≥ 35 weeks and cervical dilation ≥ 8 cm, following rupture of membranes. The comparison groups were third-year residents before, during and after training and fourth-year residents who were not exposed to the training intervention. The primary outcome was the difference in accuracy of fetal-head-position assessment on vaginal examination by third-year residents before and after ultrasound feedback training. Univariate and multivariate analyses were performed to identify factors associated with digital examination accuracy.

RESULTS

Overall, 390 examinations were performed. The accuracy of fetal-head-position assessments of third-year residents was 55% (53/96) before training, 65% (74/114) during training and 70% (63/90) after training, while that of fourth-year residents who did not undergo training was 52% (47/90) (P = 0.04). Fourth-year residents who did not undergo ultrasound training demonstrated similar baseline accuracy to that of third-year residents pretraining (52% (47/90) vs 55% (53/96), P = 0.68), but had significantly lower accuracy than had the third-year residents post-training (52% (47/90) vs 70% (63/90); P = 0.01). Multivariable analysis revealed a positive association between ultrasound feedback training and the ability to assess accurately fetal head position. After adjusting for the variables included in the final model, examinations performed by third-year residents pretraining and those performed by fourth-year residents who did not undergo training were less likely to be accurate than those performed by third-year residents post-training (adjusted odds ratio, 0.48 (95% CI, 0.26-0.91) and 0.42 (95% CI, 0.22-0.80), respectively).

CONCLUSION

Immediate ultrasound feedback training increased trainee accuracy in vaginal assessment of fetal head position in labor. Its integration into obstetric training programs should be considered. Copyright © 2019 ISUOG. Published by John Wiley & Sons Ltd.

International Society of Ultrasound in Obstetrics and Gynecology (ISUOG) - the propagation of knowledge in ultrasound for the improvement of OB/GYN care worldwide: experience of basic ultrasound training in Oman

BACKGROUND

The aim of this study is to evaluate effectiveness of a new ISUOG (International Society of Ultrasound in Obstetrics and Gynecology) Outreach Teaching and Training Program delivered in Muscat, Oman.

METHODS

Quantitative assessments to evaluate knowledge and practical skills were administered before and after an ultrasound course for sonologists attending the ISUOG Outreach Course, which took place in November, 2017, in Oman. Trainees were selected from each region of the country following a national vetting process conducted by the Oman Ministry of Health. Twenty-eight of the participants were included in the analysis. Pre- and post-training practical and theoretical scores were evaluated and compared.

RESULTS

Participants achieved statistically significant improvements, on average by 47% (p < 0.001), in both theoretical knowledge and practical skills. Specifically, the mean score in the theoretical knowledge test significantly increased from 55.6% (± 14.0%) to 81.6% (± 8.2%), while in the practical test, the mean score increased from 44.6% (± 19.5%) to 65.7% (± 23.0%) (p < 0.001). Performance was improved post-course among 27/28 participants (96.4%) in the theoretical test (range: 14 to 200%) and among 24/28 (85.7%) trainees in the practical skills test (range: 5 to 217%).

CONCLUSION

Application of the ISUOG Basic Training Curriculum and Outreach Teaching and Training Course improved the theoretical knowledge and practical skills of local health personnel. Long-term re-evaluation is, however, considered imperative to ascertain and ensure knowledge retention.