A Theory-Based Curriculum for Enhancing Surgical Skillfulness

The Role of Manual Dexterity and Cognitive Functioning in Enhancing Resident Operative Autonomy

INTRODUCTION

Autonomy, both operative and nonoperative, is one of the most critical aspects of successful surgical training. Both surgeon and resident share the responsibility of achieving this goal. We hypothesize that operative autonomy is distinct and depends, for the most part, on the resident's manual dexterity, knowledge of, and preparation for the procedure.

METHODS

Over a period of 4 academic years, between July 2014 and June 2018, a total of 958 Global Rating Scale of Operative Performance evaluations were completed by 32 general and subspecialty faculty surgeons for 35 residents. Elective procedures were evaluated, including 165 (17.2%) by postgraduate year (PGY)1 residents, 253 (26.4%) by PGY2, 199 (20.8%) by PGY3, 147 (15.3%) by PGY4, and 194 (20.3%) by PGY5. The procedures evaluated were: 261 (27.2%) hernia repairs; 178 (18.6%) cholecystectomies; 102 (10.6%) colorectal and anal procedures; 73 (7.6%) vascular procedures; 56 (5.8%) thyroid and parathyroidectomies; 39 (4.1%) foregut (esophagus and stomach) procedures; 38 (4%) skin, soft tissue, and breast; 92 (10%) hepatopancreatic; 20 (2.1%) pediatric procedures; and 99 (10.3%) other procedures including amputations, cardiothoracic, and solid organs procedures. Each resident was scored from 1 to 5 (1 lowest, 5 highest) in each of the following categories of Global Rating Scale of Operative Performance: respect for tissue (RT), time and motion (T&M), instrument handling (IH), knowledge of the instrument (KI), flow of operation (FO) and resident's preparation for the procedure (RP). Resident operative autonomy (ROA) was assessed using the Zwisch scale, a 4-point scale describing faculty supervision behaviors associated with different degrees of resident autonomy (1: Show and Tell, 2: Active Help, 3: Passive Help, and 4: Supervision Only).

RESULTS

Correlation and ordinal regression analyses were conducted to examine the relationship between ROA and manual dexterity (RT, T&M, IH, and FO), and cognitive functioning (knowledge of instruments and resident preparation). Results indicated a positive correlation between ROA and RT (r = 0.528, p < 0.001), T&M (r = 0.630, p < 0.001), IH (r = 0.597, p < 0.001), KI (r = 0.490, p < 0.001), FO (r = 0.637, p < 0.001), and RP (r = 0.525, p < 0.001). Additionally, there was a weak inverse correlation between ROA and the number of years the surgeon had been in practice (r = -0.127, p = 0.001). The significant predictors of resident autonomy found by the ordinal logistic regression include time and motion (p < 0.001), flow of operation (p < 0.001), and resident's preparation for the procedure (p < 0.001).

CONCLUSIONS

Resident operative autonomy is a product of shared responsibility between the faculty and resident. However, residents' inherent and/or acquired skills and preparation for the operative procedures play a critical role. Residents should be advised to use available resources such as simulation to augment their skills preoperatively and to enhance their autonomy in the operating room.

The making of a surgeon: 10,000 hours?

There have been many changes in the "making of a surgeon". Some of the key aspects that have altered residency/fellow training include work hour restrictions; a decrease in autonomy; and the explosion in knowledge, the change in technology, and the movement of complex cases away from General Surgery. There are a number of opportunities for enhancing current surgical training which include the following: 1) returning to reasonable work hour limits; 2) improving the efficiency of resident/fellow training by promoting early development of operative skills and starting down the path toward competency-based education; 3) increasing autonomy in the General and Pediatric Surgery residencies by developing and implementing structured processes for graded autonomy, further promoting the teaching assistant role, and even incorporating time as an attending into the period of training; and 4) developing a paradigm of uniform core surgery training followed by additional qualifications and training in both General Surgery and the surgical specialties.

Predicting neonatal intubation competency in trainees

BACKGROUND AND OBJECTIVES

Pediatric residency training programs are graduating residents who are not competent in neonatal intubation, a vital skill needed for any pediatrician involved in delivery room resuscitations. However, a precise definition of competency during training is lacking. The objective of this study was to more precisely define the trajectory toward competency in neonatal intubation for pediatric residents, as a framework for later evaluating complementary training tools.

METHODS

This is a retrospective single-center observational study of resident-performed neonatal intubations at Duke University Medical Center between 2005 and 2013. Using a Bayesian statistical model, intubation competency was defined when the resident attained a 75% likelihood of intubating their next patient successfully.

RESULTS

A total of 477 unique intubation attempts by 105 residents were analyzed. The path to proficiency was defined by a categorical or milestone learning event after which all learners move toward competency in a similar manner. In our cohort, 4 cumulative successes were needed to achieve competency. Only 24 of 105 (23%) achieved competency during the study period. Residents who failed their first 2 opportunities, compared with those successful on their first 2 opportunities, needed nearly double the intubation exposure to achieve competency.

CONCLUSIONS

Bayesian statistics may be useful to more precisely describe neonatal intubation competency in residents. Achieving competency in neonatal intubation appears to be a categorical or milestone learning event whose timing varies between residents. The current educational environment does not provide adequate procedural exposure to achieve competency for most residents.

Assessment of surgery residents' operative skills in the operating theater using a modified Objective Structured Assessment of Technical Skills (OSATS): a prospective multicenter study

BACKGROUND

With the implementation of competency-based curricula, Objective Structured Assessment of Technical Skills (OSATS) increasingly is being used for the assessment of operative skills. Although evidence for its usefulness has been demonstrated in experimental study designs, data supporting OSATS application in the operating room are limited. This study evaluates the validity and reliability of the OSATS instrument to assess the operative skills of surgery residents in the operating theater.

METHODS

Twenty-four residents were recruited from seven hospitals within a general surgical training region and classified equally into three groups according to postgraduate training year (PGY). Each resident had to perform five different types of operations. Surgical performance was measured using a modified OSATS consisting of three scales: Global Rating Scale, Overall Performance Scale, and Alphabetic Summary Scale. Validity and reliability metrics included construct validity (Kruskal-Wallis test) and internal consistency reliability (Cronbach's α coefficient). Spearman's correlation coefficients were calculated to determine correlations between the different scales.

RESULTS

Eighteen residents (PGY 1-2 [n = 7]; PGY 3-4 [n = 8]; PGY 5-6 [n = 3]) performed 249 operations. Comparisons of the performance scores revealed that evidence for construct validity depended on the difficulty level of the selected procedures. For individual operations, internal consistency reliability of the Global Rating Scale ranged from 0.93 to 0.95. Scores on the different scales correlated strongly (r = 0.62-0.76, P < .001).

CONCLUSION

Assessment of operative skills in the operating theater using this modified OSATS instrument has the potential to establish learning curves, allowing adequate monitoring of residents' progress in achieving operative competence. The Alphabetic Summary Scale seems to be of additional value. Use of the Overall Performance Scale should be reconsidered.

Validation of the National Aeronautics and Space Administration Task Load Index as a tool to evaluate the learning curve for endoscopy training

BACKGROUND

Although workplace workload assessments exist in different fields, an endoscopy-specific workload assessment tool is lacking.

OBJECTIVE

To validate such a workload tool and use it to map the progression of novice trainees in gastroenterology in performing their first endoscopies.

METHODS

The National Aeronautics and Space Administration Task Load Index (NASA-TLX) workload assessment tool was completed by eight novice trainees in gastroenterology and 10 practicing gastroenterologists⁄surgeons. An exploratory factor analysis was performed to construct a streamlined endoscopy-specific task load index, which was subsequently validated. The 'Endoscopy Task Load Index' was used to monitor progression of trainee exertion and self-assessed performance over their first 40 procedures.

RESULTS

From the factor analysis of the NASA-TLX, two principal components emerged: a measure of exertion and a measure of self-efficacy. These items became the components of the newly validated Endoscopy Task Load Index. There was a steady decline in self-perceived exertion over the training period, which was more rapid for gastroscopy than colonoscopy. The self-efficacy scores for gastroscopy rapidly increased over the first few procedures, reaching a plateau after this period of time. For colonoscopy, there was a progressive increase in reported self-efficacy over the first three quartiles of procedures, followed by a drop in self-efficacy scores over the final quartile.

DISCUSSION

The present study validated an Endoscopy Task Load Index that can be completed in <1 min. Practical implications of such a tool in endoscopy education include identifying periods of higher perceived exertion among novice endoscopists, facilitating appropriate levels of guidance from trainers.

Development and verification of a taxonomy of assessment metrics for surgical technical skills

PURPOSE

To create and empirically verify a taxonomy of metrics for assessing surgical technical skills, and to determine which types of metrics, skills, settings, learners, models, and instruments were most commonly reported in the technical skills assessment literature.

METHOD

In 2011-2012, the authors used a rational analysis of existing and emerging metrics to create the taxonomy, and used PubMed to conduct a systematic literature review (2001-2011) to test the taxonomy's comprehensiveness and verifiability. Using 202 articles identified from the review, the authors classified metrics according to the taxonomy and coded data concerning their context and use. Frequencies (counts, percentages) were calculated for all variables.

RESULTS

The taxonomy contained 12 objective and 4 subjective categories. Of 567 metrics identified in the literature, 520 (92%) were classified using the new taxonomy. Process metrics outnumbered outcome metrics by 8:1. The most frequent metrics were "time," "manual techniques" (objective and subjective), "errors," and "procedural steps." Only one new metric, "learning curve," emerged. Assessments of basic motor skills and skills germane to laparoscopic surgery dominated the literature. Novices, beginners, and intermediate learners were the most frequent subjects, and box trainers and virtual reality simulators were the most frequent models used for assessing performance.

CONCLUSIONS

Metrics convey what is valued in human performance. This taxonomy provides a common nomenclature. It may help educators and researchers in procedurally oriented disciplines to use metrics more precisely and consistently. Future assessments should focus more on bedside tasks and open surgical procedures and should include more outcome metrics.

Gaze training improves the retention and transfer of laparoscopic technical skills in novices

BACKGROUND

Gaze training is an effective way of training basic laparoscopic skills, resulting in faster acquisition periods and more robust subsequent performance under pressure. The current study is a randomized control trial which examines whether the performance benefits of gaze training stand the test of time (delayed retention) and transfer to more complex skills.

METHODS

Thirty-six medical students were trained to proficiency (50 trials) on a one-handed laparoscopic task (picking and dropping balls) in either a discovery learning (DL) or gaze training (GT) group. Both groups performed the one-handed task in baseline, retention and delayed retention (1 month) tests. They also performed baseline, retention and delayed retention tests of a two-handed task (grasping and cutting). Performance (completion time) and gaze control (target locking) were assessed throughout.

RESULTS

For the one-handed task, the GT group displayed superior performance at retention (p < .001), underpinned by more expert-like gaze control (p < .05). The GT group also displayed superior performance in the one-handed task at delayed retention (p < .005), underpinned by more expert-like gaze control strategies (p < .001). Although the DL group's performance fell to 84% of performance at retention, the GT group maintained performance at 100% of retention. There were no differences between the groups for the two-handed task at retention (p = .140); however, at delayed retention, the GT group outperformed the DL group (p < .005) and displayed more expert-like gaze control (p < .001).

CONCLUSIONS

Novices trained to adopt an expert-like gaze control strategy were able to attain higher levels of performance more quickly than novices who learned by discovery alone. Furthermore, these skills were more durable over time and were transferable to more complex skills. Gaze training is a beneficial intervention to aid the acquisition of the basic motor skills required for laparoscopy.

Innovation in internship preparation: an operative anatomy course increases senior medical students' knowledge and confidence

BACKGROUND

An operative anatomy course was developed within the construct of a surgical internship preparatory curriculum. This course provided fourth-year medical students matching into a surgical residency the opportunity to perform intern-level procedures on cadavers under the guidance of surgical faculty members.

METHODS

Senior medical students performed intern-level procedures on cadavers with the assistance of faculty surgeons. Students' confidence, anxiety, and procedural knowledge were evaluated both preoperatively and postoperatively. Preoperative and postoperative data were compared both collectively and based on individual procedures.

RESULTS

Student confidence and procedural knowledge significantly increased and anxiety significantly decreased when preoperative and postoperative data were compared (P < .05). Students reported moderate to significant improvement in their ability to perform a variety of surgical tasks.

CONCLUSIONS

The consistent improvement in confidence, knowledge, and anxiety justifies further development of an operative anatomy course, with future assessment of the impact on performance in surgical residency.

A comparison C1-C2 transarticular screw placement after self-education and mentored education of orthopaedic residents

STUDY DESIGN

Prospective randomized trial.

OBJECTIVE

This study will provide preliminary data on whether residents can be "self-taught" and to what extent a lecture, demonstration, and coaching can improve skills and knowledge.

BACKGROUND DATA

Practice-based learning is an essential competency in Accreditation Council for graduate Medical Education-accredited residencies. Little has been done to demonstrate whether residents can be self-taught or benefit from mentoring in understanding and performing difficult surgical tasks.

METHODS

A written test was given to orthopedic residents on C1-C2 transarticular screw placement. They were then provided reading on C1-C2 transarticular screw placement. Residents were then divided into a "self-directed learning" group and a "mentored learning" group. All residents then performed the technique on models, with the "mentored" group receiving a lecture and coaching from the mentor. The models were analyzed for technique errors and the previous test was administered again as a posttest. The test and screw placement were repeated 4 months later.

RESULTS

Residents without mentoring had an average improvement of 4.5 points, those with mentoring had average improvement of 8.6 points (P=0.0068). The screw placement technique error rate for the nonmentored group (n=8) was 2.55, and for the mentored group (n=9) was 1.47 (P=0.004). Sixteen residents completed the delayed test, 7 from the nonmentored groups and 8 from the mentored group. Nine residents were able to repeat the screw placement technique 4 months after the initial test and screw placement, 3 nonentored, and 6 mentored. Although there were some trends toward the mentored group having better retention, neither knowledge nor skill was statistically different.

CONCLUSIONS

This preliminary trial seems to indicate that residents provided a lecture and guided technical instruction will obtain knowledge and perform procedures better than those that do not. Conclusions based upon Post Graduate year, motivation, and interest in spine surgery could not be made.

Three components of education in burn care: surgical education, inter-professional education, and mentorship

Education in burn care can be divided into three main components: surgical education, inter-professional education, e.g. critical care education and mentorship. To date these components have been used in varying degrees in most health institutions and to even lesser extent in burn care. The aim of this paper is to highlight each component and how these have been utilized in other fields to develop teams and foster education, and how they can be translated for burn teams. These ideas are not novel; however, this paper aims to shed light on how these concepts can be implemented in burn care, thus not only improving education, but also enabling recruitment and retention of health care providers in this field.

Deliberate practice improves obstetrics and gynecology residents' hysteroscopy skills

INTRODUCTION

Development of surgical skills is an integral component of residency education in obstetrics and gynecology.

OBJECTIVE

We report data from a supervised, deliberate, dry lab practice in hysteroscopy for junior obstetrics-gynecology residents, undertaken to evaluate whether simulation training improved hysteroscopy performance to a skill level similar to that of senior residents.

METHODS

A prospective, comparative, multicenter trial compared Objective Structured Assessment Of Technical Skills (OSATS) performance of 2 groups: 19 postgraduate year (PGY)-1 and PGY-2 and 18 PGY-3 and PGY-4 Ob-Gyn residents. PGY-1 and PGY-2 participants underwent 4 sessions of brief, deliberate, focused training in hysteroscope assembly and operative hysteroscopic polypectomy using uterine models. Subsequently, all participants completed a simulated hysteroscopic polypectomy OSATS, and procedure times and structured assessment scores were compared among groups.

RESULTS

PGY-1 and PGY-2 residents who had completed OSATS training performed at or above the level of untrained PGY-3 and PGY-4 residents. Junior residents had better assembly times and scores, resection scores, and global skills scores (P < .05). Resection times did not differ between groups but differed among institutions.

DISCUSSION

Brief, hands-on training sessions, which were task-specific and repetitive facilitated short-term gains in learning operative hysteroscopy and increased the dry lab skill level of junior residents compared to that of senior residents. This curriculum was effectively implemented at 3 institutions and generated comparable results, suggesting generalizability.

Cheating experience: Guiding novices to adopt the gaze strategies of experts expedites the learning of technical laparoscopic skills

BACKGROUND

Previous research has demonstrated that trainees can be taught (via explicit verbal instruction) to adopt the gaze strategies of expert laparoscopic surgeons. The current study examined a software template designed to guide trainees to adopt expert gaze control strategies passively, without being provided with explicit instructions.

METHODS

We examined 27 novices (who had no laparoscopic training) performing 50 learning trials of a laparoscopic training task in either a discovery-learning (DL) group or a gaze-training (GT) group while wearing an eye tracker to assess gaze control. The GT group performed trials using a surgery-training template (STT); software that is designed to guide expert-like gaze strategies by highlighting the key locations on the monitor screen. The DL group had a normal, unrestricted view of the scene on the monitor screen. Both groups then took part in a nondelayed retention test (to assess learning) and a stress test (under social evaluative threat) with a normal view of the scene.

RESULTS

The STT was successful in guiding the GT group to adopt an expert-like gaze strategy (displaying more target-locking fixations). Adopting expert gaze strategies led to an improvement in performance for the GT group, which outperformed the DL group in both retention and stress tests (faster completion time and fewer errors).

CONCLUSION

The STT is a practical and cost-effective training interface that automatically promotes an optimal gaze strategy. Trainees who are trained to adopt the efficient target-locking gaze strategy of experts gain a performance advantage over trainees left to discover their own strategies for task completion.

Novel simulation for training trauma surgeons

BACKGROUND

Clinical training in operative technique is important to boost self-confidence in residents in all surgical fields but particularly in trauma surgery. The fully trained trauma surgeon must be able to provide operative intervention for any injury encountered in practice. In this report, we describe a novel training model using a human cadaver in which circulation in the major vessels can be simulated to mimic traumatic injuries seen in clinical practice.

METHODS

Fourteen human cadavers were used for simulating various life-threatening traumatic injuries. The carotid and femoral arteries and the jugular and femoral vein were cannulated and connected to perfusate reservoirs. The arterial reservoir was connected to an intra-aortic balloon pump, which adds pulsatile flow through the heart and major arteries. Fully trained trauma surgeons evaluated the utility of this model for repairing various injuries in the thoracic and abdominal cavity involving the heart, lungs, liver, and major vessels while maintaining emergent airway control.

RESULTS

Surgeons reported that this perfused cadaver model allowed simulation of the critical challenges faced during operative trauma while familiarizing the student with the operative techniques and skills necessary to gain access and control of hemorrhage associated with major vascular injuries.

CONCLUSION

In this report, we describe a novel training model that simulates the life-threatening injuries that confront trauma surgeons. An alternative to living laboratory animals, this inexpensive and readily available model offers good educational value for the acquisition and refinement of surgical skills that are specific to trauma surgery.

Perceptual impairment and psychomotor control in virtual laparoscopic surgery

BACKGROUND

It is recognised that one of the major difficulties in performing laparoscopic surgery is the translation of two-dimensional video image information to a three-dimensional working area. However, research has tended to ignore the gaze and eye-hand coordination strategies employed by laparoscopic surgeons as they attempt to overcome these perceptual constraints. This study sought to examine if measures related to tool movements, gaze strategy, and eye-hand coordination (the quiet eye) differentiate between experienced and novice operators performing a two-handed manoeuvres task on a virtual reality laparoscopic surgical simulator (LAP Mentor™).

METHODS

Twenty-five right-handed surgeons were categorised as being either experienced (having led more than 60 laparoscopic procedures) or novice (having performed fewer than 10 procedures) operators. The 10 experienced and 15 novice surgeons completed the "two-hand manoeuvres" task from the LAP Mentor basic skills learning environment while wearing a gaze registration system. Performance, movement, gaze, and eye-hand coordination parameters were recorded and compared between groups.

RESULTS

The experienced surgeons completed the task significantly more quickly than the novices, used significantly fewer movements, and displayed shorter tool paths. Gaze analyses revealed that experienced surgeons spent significantly more time fixating the target locations than novices, who split their time between focusing on the targets and tracking the tools. A more detailed analysis of a difficult subcomponent of the task revealed that experienced operators used a significantly longer aiming fixation (the quiet eye period) to guide precision grasping movements and hence needed fewer grasp attempts.

CONCLUSION

The findings of the study provide further support for the utility of examining strategic gaze behaviour and eye-hand coordination measures to help further our understanding of how experienced surgeons attempt to overcome the perceptual difficulties inherent in the laparoscopic environment.

The development of a comprehensive school-wide simulation-based procedural skills curriculum for medical students

OBJECTIVE

The purpose of this project was to assess the effectiveness of using the Delphi process to create a structured simulation-based procedural skills curriculum for all students at the Keck School of Medicine of the University of Southern California (KSOM).

METHODS

The Delphi process was used to develop a list of procedural skills that students are expected to perform competently prior to graduation. Once consensus of faculty was reached, a needs assessment was performed to poll graduating seniors' experience performing each skill. A comprehensive simulation-based curriculum was developed and implemented for all Year II students at KSOM. Student satisfaction with the curriculum was collected using a standardized end-of-session evaluation form and student self confidence was assessed using a retrospective pre- and post-self-efficacy rating for each skill.

RESULTS

The needs assessment clearly established the need for a more organized approach to teaching procedural skills at KSOM. Quantitative and qualitative data revealed that students responded favorably to the curriculum and appreciated the efforts put forth by KSOM. Student self-efficacy increased significantly for each skill.

CONCLUSIONS

The Delphi process was effective in reaching consensus among educational leaders at KSOM regarding which skills to include in the curriculum. Although there were a few minor challenges, we determined that it is feasible to develop and implement an explicit school-wide simulated-based procedural skills curriculum.

Psychomotor control in a virtual laparoscopic surgery training environment: gaze control parameters differentiate novices from experts

Background

Surgical simulation is increasingly used to facilitate the adoption of technical skills during surgical training. This study sought to determine if gaze control parameters could differentiate between the visual control of experienced and novice operators performing an eye-hand coordination task on a virtual reality laparoscopic surgical simulator (LAP Mentor™). Typically adopted hand movement metrics reflect only one half of the eye-hand coordination relationship; therefore, little is known about how hand movements are guided and controlled by vision.

Methods

A total of 14 right-handed surgeons were categorised as being either experienced (having led more than 70 laparoscopic procedures) or novice (having performed fewer than 10 procedures) operators. The eight experienced and six novice surgeons completed the eye-hand coordination task from the LAP Mentor basic skills package while wearing a gaze registration system. A variety of performance, movement, and gaze parameters were recorded and compared between groups.

Results

The experienced surgeons completed the task significantly more quickly than the novices, but only the economy of movement of the left tool differentiated skill level from the LAP Mentor parameters. Gaze analyses revealed that experienced surgeons spent significantly more time fixating the target locations than novices, who split their time between focusing on the targets and tracking the tools.

Conclusion

The findings of the study provide support for the utility of assessing strategic gaze behaviour to better understand the way in which surgeons utilise visual information to plan and control tool movements in a virtual reality laparoscopic environment. It is hoped that by better understanding the limitations of the psychomotor system, effective gaze training programs may be developed.

Using objective structured assessment of technical skills to evaluate a basic skills simulation curriculum for first-year surgical residents

BACKGROUND

In response to new Accreditation Council for Graduate Medical Education requirements about simulation skill laboratories, programs are incorporating simulation into residents' training. Despite substantial research on simulators, few data exist to support the effectiveness of simulation skills curricula. We report on an Objective Structured Assessment of Technical Skills (OSATS) used to assess residents' needs and evaluate a curriculum designed to increase proficiency.

STUDY DESIGN

The five-session (10-week) curriculum covered asepsis, skin preparation, gowning, gloving, knot-tying, suturing, and excision. Performance on a 20-minute OSATS station was measured by unblinded raters using a task-specific checklist and seven global rating scales. Interns' pre-post improvement was assessed using paired t-tests. PGY2 and PGY3 residents were used as nonequivalent controls; their scores set a benchmark for PGY1 residents postcurriculum. Percentage of possible points earned was compared with a 75% "needs" criterion; ANOVA was used to assess group differences at the p < 0.05 level.

RESULTS

Seven PGY2 and 6 PGY3 residents took the OSATS; 24 of 25 PGY1s completed both the baseline and postcurriculum OSATS. At baseline, PGY1 mean percent correct total score was 49%; they performed considerably below PGY2 (68%) and PGY3 (74%) residents. PGY1 scores improved significantly after 10 weeks (p < or = 0.001). When their postcurriculum scores were compared with PGY2 and PGY3 resident benchmarks, there were no significant differences in checklist (p = 0.38), global item (p = 0.29), or total scores (p = 0.45).

CONCLUSIONS

Our results suggest that the simulation curriculum helped PGY1 residents attain basic surgical skills at levels consistent with PGY2 and PGY3 residents as measured by an OSATS. Only PGY3 residents performed at the 75% criterion.