Identifying Priorities for Faculty Development in General Surgery Using the Delphi Consensus Method

Misperceptions and Missed Opportunities: A Qualitative Analysis of Barriers to Evaluating Surgical Teachers

OBJECTIVE

Teaching skills can be improved with written evaluations from learners. In addition to this primary purpose, teaching assessments are used in other aspects of faculty development including appointments, advancement and in some cases, compensation. Surgical trainees' willingness to provide meaningful assessments of their teachers is variable but the reasons for this are ill-defined. This study aims to elucidate surgical residents' perceptions regarding barriers to providing useful feedback to their teachers.

DESIGN

A qualitative, semi-structured confidential interview approach was used. A demographically diverse cohort of surgical residents in an urban university-based program was invited to participate. Interviews explored experiences and perceptions of teaching assessments. Specific attention was paid to understand perceptions of barriers; topics including utility, anonymity, time burden, and others were explored. Interviews were transcribed verbatim with identifiers removed from transcripts before analysis. All data was double coded to ensure accuracy with the development of a codebook until thematic exhaustion was reached.

SETTING

Yale New Haven Hospital is an academic, university-based medical center with approximately 70 residents in the general surgery program and approximately 170 surgical faculty.

PARTICIPANTS

A total of 21 residents completed individual or small group interviews. A theoretically driven sampling technique was used to recruit participants and maximize diversity. Individuals with varying backgrounds including PGY year, gender, age, IMG status, race, academic rank, research background and surgical division were asked to participate.

RESULTS

A total of 21 residents completed individual or small group interviews. Coding and analysis revealed 4 principal motifs: (1). Process- The process to complete assessment instruments is time-consuming and cumbersome to complete during the busy and acute surgical workday while failing to accurately address important aspects of surgical teaching. (2). Utility- Respondents reported uncertainty as to the downstream utility of the assessments, and a lack of confidence that the assessments would be used for faculty growth and improvement. (3). Resident Standing- Respondents described a lack of training, knowledge, skills, and empowerment to assess their teachers. (4). Perceived Consequences- Residents noted concern for identification, future autonomy, and other potential negative career consequences due to small resident sample sizes, recognizable experiences with attendings, and perceived power dynamics.

CONCLUSIONS

This study elucidates the perceptions of surgical trainees regarding barriers to providing feedback and assessments of their faculty. Although limited to a single-institution study, residents observed the current system does not allow for honest and accurate evaluations of surgical teachers. The extensive overlap between motifs highlights the need for a holistic approach to address these interconnected themes before teaching evaluations can be honest and productive. Importantly, it is also the first to identify residents' perceived lack of skill and sense of disempowerment to provide constructive faculty assessment. Due to the limited scope of the single-institution study, further verification and studies are needed to improve the quality of faculty feedback and assessment of surgical teachers.

Postgraduate Surgical Education in East, Central, and Southern Africa: A Needs Assessment Survey

BACKGROUND

The Lancet Commission on Global Surgery has identified workforce development as an important component of National Surgical Plans to advance the treatment of surgical disease in low- and middle-income countries. The goal of our study is to identify priorities of surgeon educators in the region so that collaboration and intervention may be appropriately targeted.

STUDY DESIGN

The American College of Surgeons Operation Giving Back, in collaboration with leaders of the College of Surgeons of Eastern, Central and Southern Africa (COSECSA), developed a survey to assess the needs and limitations of surgical educators working under their organizational purview. COSECSA members were invited to complete an online survey to identify and prioritize factors within 5 domains: (1) Curriculum Development, (2) Faculty Development, (3) Structured Educational Content, (4) Skills and Simulation Training, and (5) Trainee Assessment and Feedback.

RESULTS

One-hundred sixty-six responses were received after 3 calls for participation, representing all countries in which COSECSA operates. The majority of respondents (78%) work in tertiary referral centers. Areas of greatest perceived need were identified in the Faculty Development and Skills and Simulation domains. Although responses differed between domains, clinical responsibilities, cost, and technical support were commonly cited as barriers to development.

CONCLUSIONS

This needs assessment identified educational needs and priorities of COSECSA surgeons. Our study will serve as a foundation for interventions aimed at further improving graduate surgical education and ultimately patient care in the region.

Faculty Development Offered by US Medical Schools: A National Survey of Pediatric Educators

INTRODUCTION

There are limited data on the status of faculty development (FD) in the United States. Through a national survey of pediatric educators, we explored the frequency and topics of FD on teaching skills offered at US medical schools, as well as the strategies' schools use to encourage and track participation.

METHODS

Five piloted questions were included in the 2017 Annual Council on Medical Student Education in Pediatrics Survey. We used descriptive statistics.

RESULTS

Ninety-seven (66%) of the 148 surveyed US medical schools responded to at least one FD question. Ninety-eight percent of respondents reported being offered FD on teaching, with 97% of those respondents reporting that FD occurred at least annually. A variety of FD topics were reported, with feedback and precepting being most common. Incentives included continuing medical education (CME credit) (39%) and being relieved of clinical duties (23%). However, 29% reported little support for FD. Only 20% of schools reported their department tracked FD participation outside the department.

DISCUSSION

Our data suggest that the majority of medical schools offer FD on teaching skills at least yearly, with a variety of topics. Institutions utilize a variety of incentives for participation. However, a significant minority of respondents reported little support for FD. Further, departments rarely track faculty FD participation.

Trigger videos: a novel application of a tool for surgical faculty development

BACKGROUND

Trigger videos have occasionally been used in medical education; however, their application to surgical faculty development is novel. We assessed participants' attitudes towards workshops on intraoperative teaching (IOT) that were anchored by trigger videos, and studied whether they could generate discussion-for-learning among surgeons in this workshop setting.

METHODS

Surgeons from multiple specialties attended one of six faculty development workshops where IOT trigger videos were shown and discussed during break-out sessions. Participants completed questionnaires to (1) evaluate videos via survey and feedback, and (2) identify adoptable and discardable IOT techniques. Teaching techniques were collated to identify planned IOT changes and survey data and feedback were analyzed.

RESULTS

A total of 135 surgeons identified 292 adoptable and 202 discardable IOT techniques based on trigger videos and discussions, and 94% of participants reported that the trigger videos were useful and encouraged them to discuss and consider new IOT techniques in their own practice.

CONCLUSIONS

Participants reported that the trigger videos were useful and motivating. Surgeons critically reflected on IOT during the sessions, identifying numerous adoptable and discardable techniques relevant to their own teaching styles. Trigger videos can be a valuable tool for surgical faculty development and can be tailored to other medical specialties.

Unifying the Hepatopancreatobiliary Surgery Fellowship Curriculum via Delphi Consensus

BACKGROUND

Hepatopancreatobiliary (HPB) Fellowship training in the Americas consists of 3 distinctive routes with variable curricula: Surgical Oncology Fellowship via the Society of Surgical Oncology (SSO), Abdominal Transplant Surgery Fellowship via the American Society of Transplant Surgeons (ASTS), and HPB Fellowship via the Americas Hepato-Pancreato-Biliary Association (AHPBA). Our objective was to establish a pan-American consensus among HPB surgeons, surgical oncologists, abdominal transplant surgeons, and general surgery residency program directors (GSPDs) on a core knowledge curriculum for HPB fellowship, and to identify topics appropriate for general surgery residency and subspecialty beyond HPB fellowship.

STUDY DESIGN

A 3-round modified Delphi process was used. Baseline statements were developed by the Education and Training Committee of the AHPBA, in collaboration with representatives of the SSO, ASTS, and GSPDs. The expert panel, consisting of members of the 3 societies together with GSPDs, rated the statements on a 5-point Likert scale and suggested editing or adding new statements. A statement was included in the final curriculum when Cronbach's alpha value was ≥ 0.8 and ≥ 80% of the panel agreed on inclusion.

RESULTS

The response rate was 100% for the first round, and 98% for the second and third rounds. Eighty-nine of 138 proposed statements were included in the final HPB fellowship curriculum. Curricula for general surgery residency and subspecialty beyond HPB fellowship included 50 and 29 statements, respectively.

CONCLUSIONS

A multinational consensus on core knowledge for an HPB fellowship curriculum was achieved via the modified Delphi method. This core curriculum may be used to standardize HPB fellowship training across different pathways in the Americas.

Education Morbidity and Mortality: Reviving Intraoperative Teaching and Learning

BACKGROUND

Using the platform of morbidity and mortality conference, we developed and executed a combined faculty-resident intervention called "Education M&M" to discuss challenges faced by both parties in the operating room (OR), identify realistic solutions, and implement action plans. This study aimed to investigate the impact of this intervention on resident OR training.

MATERIALS AND METHODS

Two resident case presentations were followed by audience discussion and recommendations regarding actionable solutions aimed at improving resident OR training from an expert faculty panel. Postintervention surveys were completed by participants immediately and 2 mo later to assess perceived short and long-term impact on OR teaching and/or learning and the execution of two recommended solutions. Descriptive statistical analysis was applied.

RESULTS

Immediate post-intervention surveys (n = 44) indicated that 81.8% of participants enjoyed the M&M "a lot"; 90.1% said they would use some or a lot of the ideas presented. Awareness of OR teaching/learning challenges before and after the M&M improved from 3.0 to 3.7 (P = 0.00001) for faculty and 3.0 to 3.9 for trainees (P = 0.00004). Understanding of OR teaching and/or learning approaches improved from 3.1 to 3.7 for faculty (P = 0.00004) and 2.7 to 3.9 for trainees (P = 0.00001). In 2-mo post-intervention surveys, most residents had experienced two recommended solutions (71% and 88%) in the OR, but self-reported changes to faculty behavior did not reach statistical significance.

CONCLUSIONS

A department-wide education M&M could be an effective approach to enhance mutual communication between faculty members and residents around OR teaching/learning by identifying program-specific challenges and potential actionable solutions.

A multi-institutional study assessing general surgery faculty teaching evaluations

BACKGROUND

Resident evaluation of faculty teaching is an important metric in general surgery training, however considerable variability in faculty teaching evaluation (FE) instruments exists.

STUDY DESIGN

Twenty-two general surgery programs provided their FE and program demographics. Three clinical education experts performed blinded assessment of FEs, assessing adherence 2018 ACGME common program standards and if the FE was meaningful.

RESULTS

Number of questions per FE ranged from 1 to 29. The expert assessments demonstrated that no evaluation addressed all 5 ACGME standards. There were significant differences in the FEs effectiveness of assessing the 5 ACGME standards (p < 0.001), with teaching abilities and professionalism rated the highest and scholarly activities the lowest.

CONCLUSION

There was wide variation between programs regarding FEs development and adhered to ACGME standards. Faculty evaluation tools consistently built around all suggested ACGME standards may allow for a more accurate and useful assessment of faculty teaching abilities to target professional development.

Teaching and Mentoring Early in Surgical Practice

BACKGROUND

Transitioning from trainee to independent surgeon is challenging, with teaching and mentoring learners adding complexity. This study aimed to identify benefits and challenges of teaching and mentoring residents early in surgical practice.

MATERIALS AND METHODS

A mixed-methods survey with multiple choice and open-ended questions was developed for early career (first 3 years) and experienced (more than 3 years) surgeons working with residents. The survey was initially piloted by 6 surgeons, revised according to feedback and distributed on Twitter. Quantitative and qualitative analyses were performed.

RESULTS

Seventy-three valid responses were analyzed, 21 (25%) from early career surgeons and 53 (75%) from experienced surgeons. The majority were women (62%), practiced in academics (78%) and lived in the United States (76%). Most of the early career and experienced surgeons selected the operating room as the most teaching challenging location (62% early career; and 60% experienced). Top teaching challenges of early career surgeons were the need to focus on own skills (24%) and concern for patient safety/outcomes (24%), while experienced surgeons reported need to focus on own skills (23%) and difficulty giving up control (23%). Open-ended responses identified themes related to teaching and mentoring. Comments highlighted benefits of organizational support to facilitate teaching and mentoring and negative experiences due to resident learners not respecting early career surgeons.

DISCUSSION

This pilot study explores an area not previously studied. Early career surgeons and experienced surgeons find many similar challenges and benefits of teaching and mentoring residents. Challenges and themes identified can guide early career surgeons and organizations with regard to teaching and mentoring.

Tracking Surgical Education Survey Research Through the APDS Listserv

OBJECTIVE

Survey-based studies are cornerstones in medical education research. The Association of Program Directors in Surgery (APDS) listserv offers a method to contact program directors (PD) and residents for such research. To facilitate research beneficial to the APDS, improve the quality of survey-based research and minimize survey fatigue, the APDS research committee (ARC) developed a survey review process to grant access to the listserv for research. This study was conducted to determine the impact of the review process on the quality of survey-based research and eventual publication.

DESIGN

This log was systematically reviewed identifying publications resulting from accepted surveys. Publications were categorically analyzed to determine the components of their survey tool methodology, response rate (RR), and medical education research study quality instrument (MERSQI) score.

SETTING

The ARC used a 2-reviewer peer-review process for survey distribution requests. The request was either accepted, rejected, or returned for revision. Accepted surveys were distributed through the listserv with an ARC attestation of approval.

PARTICIPANTS

A log of all survey requests maintained from 2014 to 2017 and subsequent publications.

RESULTS

Thirty-five requests were accepted (40%), 30 were reviewed discovering 10 surveys that led to 12 publications (publication rate of 33%). The average RR was 60% (SD = 29%). Detailed explanations of survey development strategies were reported in 5 (42%), consisting of methods building validity evidence such as expert consensus, modified Delphi method, and pilot group sampling. Half of study participants were PD (50%). MERSQI scores averaged 10 (SD = 1.6).

CONCLUSION

Based on those survey research published to date, the ARC survey peer-review process has enabled most accepted surveys to achieve adequate RR. Although the pool of accepted requests is small, it does highlight areas of improvement. With further refinement of the process, including questioning the survey development methods, the process and listserv can be a powerful tool for further research.