1
|
Galvin D, O'Reilly B, Greene R, O'Donoghue K, O'Sullivan OE. A national survey on the impact of the coronavirus pandemic on gynecologic surgical training. Int J Gynaecol Obstet 2024. [PMID: 38958460 DOI: 10.1002/ijgo.15761] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2023] [Revised: 06/04/2024] [Accepted: 06/17/2024] [Indexed: 07/04/2024]
Abstract
OBJECTIVE The objective of this study was to assess the impact of the coronavirus pandemic on gynecology surgical training. METHODS A national cross-sectional online survey was distributed to all trainees and trainers in the higher specialist training program for obstetrics and gynecology in Ireland. The survey consisted of questions on topics which included: the volume of surgical procedures performed before and since the pandemic, confidence in performing various gynecologic procedures before and since the pandemic and questions regarding the impact of the pandemic on wellbeing and work practices. RESULTS Trainers and trainees experienced a significant reduction in operative volumes for most procedure types. Analysis showed a significant reduction in the number of minor procedures performed by trainees (z = -2.7, P = 0.007) and a significant reduction in the number of all procedure types performed by trainers (minor procedures z = -3.78, P = <0.001; intermediate procedures z = -4.48, P = < 0.001; major procedures z = -3.69, P = < 0.001). Respondents reported they had less time for research and audit, were less able to attend courses or conferences and worried about the impact of their work on their families. CONCLUSIONS In conclusion, this study has highlighted the current difficulties facing surgical trainees in gynecology because of the COVID-19 pandemic. These challenges have compounded an already challenging training environment for gynecology trainees. Efforts must be made to continue to provide high-quality tailored training to ensure the development of the next generation of gynecologic surgeons.
Collapse
Affiliation(s)
- Daniel Galvin
- Department of Obstetrics and Gynecology, College of Medicine and Health, University College Cork, Cork, Ireland
| | - Barry O'Reilly
- Department of Obstetrics and Gynecology, College of Medicine and Health, University College Cork, Cork, Ireland
| | - Richard Greene
- Department of Obstetrics and Gynecology, College of Medicine and Health, University College Cork, Cork, Ireland
| | - Keelin O'Donoghue
- Department of Obstetrics and Gynecology, College of Medicine and Health, University College Cork, Cork, Ireland
- INFANT Research Center, University College Cork, Cork, Ireland
| | - Orfhlaith E O'Sullivan
- Department of Obstetrics and Gynecology, College of Medicine and Health, University College Cork, Cork, Ireland
| |
Collapse
|
2
|
Serenari M, Lenzi J, Ricci C, Odaldi F, Maroni L, Laurenzi A, Prosperi E, Bonatti C, Fallani G, Caputo F, Rottoli M, Ravaioli M, Cescon M. The Importance of Multiorgan Procurement in the Improvement of Residents' Open Surgical Skills. J Surg Res 2024; 296:441-446. [PMID: 38320363 DOI: 10.1016/j.jss.2024.01.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Revised: 01/03/2024] [Accepted: 01/07/2024] [Indexed: 02/08/2024]
Abstract
INTRODUCTION The multiorgan procurement (MOP) represents a chance for the general surgery resident to learn the fundamental steps of open abdominal surgery. The objective of this study was to evaluate the impact of MOP on the residents' open surgical skills. METHODS Residents' surgical skills were assessed during a 6-month transplant rotation (October 2020-March 2021) using a modified Objective Structured Assessment of Technical Skills with the global rating scale. The surgeries were self-assessed by residents and tutors based on 9 specific steps (SS) and 4 general skills (GS). Each item was rated from 1 (poor) to 5 (excellent) with a maximum score of 45 points for SS and 20 for GS. A crossed-effects linear regression analysis was performed both to evaluate any associations between GS/SS scores and some prespecified covariates, and to study differences in the assessments performed by residents and tutors. RESULTS Residents actively participated in a total of 59 procurements. In general, there were no significant differences in SS/GS mean scorings between residents (n = 15) and tutors (n = 5). There was a significantly positive association between mean GS/SS scorings and the number of donor surgeries performed (at least 5). Comparing the evaluations of the tutors with the residents, this significance was retained only when scorings were assigned by the tutors. CONCLUSIONS MOP was shown to improve basic open surgical skills among residents. Awareness of the utility of a clinical rotation in transplant surgery should be raised also on an institutional level.
Collapse
Affiliation(s)
- Matteo Serenari
- Hepato-biliary and Transplant Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy; Department of Medical and Surgical Sciences, Alma Mater Studiorum - University of Bologna, Bologna, Italy.
| | - Jacopo Lenzi
- Department of Biomedical and Neuromotor Sciences, Alma Mater Studiorum - University of Bologna, Bologna, Italy
| | - Claudio Ricci
- Department of Medical and Surgical Sciences, Alma Mater Studiorum - University of Bologna, Bologna, Italy; Division of Pancreatic Surgery, IRCCS Azienda Ospedaliero-Universitaria Di Bologna, Bologna, Italy
| | - Federica Odaldi
- Hepato-biliary and Transplant Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Lorenzo Maroni
- Hepato-biliary and Transplant Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Andrea Laurenzi
- Hepato-biliary and Transplant Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Enrico Prosperi
- Hepato-biliary and Transplant Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy; Department of Medical and Surgical Sciences, Alma Mater Studiorum - University of Bologna, Bologna, Italy
| | - Chiara Bonatti
- Hepato-biliary and Transplant Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy; Department of Medical and Surgical Sciences, Alma Mater Studiorum - University of Bologna, Bologna, Italy
| | - Guido Fallani
- Hepato-biliary and Transplant Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy; Department of Medical and Surgical Sciences, Alma Mater Studiorum - University of Bologna, Bologna, Italy
| | - Francesca Caputo
- Hepato-biliary and Transplant Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy; Department of Medical and Surgical Sciences, Alma Mater Studiorum - University of Bologna, Bologna, Italy
| | - Matteo Rottoli
- Department of Medical and Surgical Sciences, Alma Mater Studiorum - University of Bologna, Bologna, Italy; Surgery of the Alimentary Tract, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Matteo Ravaioli
- Hepato-biliary and Transplant Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy; Department of Medical and Surgical Sciences, Alma Mater Studiorum - University of Bologna, Bologna, Italy
| | - Matteo Cescon
- Hepato-biliary and Transplant Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy; Department of Medical and Surgical Sciences, Alma Mater Studiorum - University of Bologna, Bologna, Italy
| |
Collapse
|
3
|
Galvin D, O'Reilly B, Greene R, O'Donoghue K, O'Sullivan O. A national survey of surgical training in gynaecology: 2014-2021. Eur J Obstet Gynecol Reprod Biol 2023; 288:135-141. [PMID: 37517105 DOI: 10.1016/j.ejogrb.2023.07.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Revised: 07/21/2023] [Accepted: 07/24/2023] [Indexed: 08/01/2023]
Abstract
OBJECTIVES Over the last decade barriers to surgical training have been identified, including reducing access to theatre lists, reducing numbers of major surgical procedures being performed, increasing numbers of trainees and reduction in working hours since the introduction of the European Work Time Directive (EWTD). We aimed to assess the impact of these challenges on training in gynaecology over time. STUDY DESIGN We designed a study which aimed to assess both trainers and trainees perception of gynaecological surgical training in Ireland. The purpose of this was to identify confidence levels and challenges and to highlight potential areas for future improvement of surgical training in gynaecology. A a cross-sectional survey was distributed to all trainees and trainers registered with the Royal College of Physicians of Ireland Obstetrics and Gynaecology higher specialist training programme in 2014, 2017 and again in 2021. RESULTS During the study period trainees' confidence that the training programme prepared them to perform gynaecological surgery fell significantly. This fall in confidence was most evident for trainees' ability to perform abdominal hysterectomy (40.9% vs 15.2%, χ2 = 4.61, p =.03) and vaginal hysterectomy (31.8% vs 12.1%, χ2 = 4.58, p =.03) when comparing 2014 with 2021. All trainees reporteded that gynaecology was not given adequate time in the training programme to prepare them to practice independently as consultants. Themes identified by participants to improve training included dedicated access to theatre time with a named trainer, increased simulation training and subspecialisation at later stages of training. CONCLUSION Our findings show an overall decrease in trainees' and trainers' confidence in the surgical training available in gynaecology over an eight-year period.. This is particularly true for major gynaecology procedures. Efforts must be made to ensure trainees have improved access to surgical training in gynaecology. Potential solutions include improving access to simulation and incorporation of subspecialist training into later stages of training.
Collapse
Affiliation(s)
- Daniel Galvin
- Department of Obstetrics and Gynaecology, College of Medicine and Health, University College Cork, Ireland.
| | - Barry O'Reilly
- Department of Obstetrics and Gynaecology, College of Medicine and Health, University College Cork, Ireland
| | - Richard Greene
- Department of Obstetrics and Gynaecology, College of Medicine and Health, University College Cork, Ireland
| | - Keelin O'Donoghue
- Department of Obstetrics and Gynaecology, College of Medicine and Health, University College Cork, Ireland; INFANT Research Centre, University College Cork, Cork, Ireland
| | - Orfhlaith O'Sullivan
- Department of Obstetrics and Gynaecology, College of Medicine and Health, University College Cork, Ireland
| |
Collapse
|
4
|
Ruiz TL, Sellers B, Devarakonda A, Wehrle CJ, Arora TK. A Novel Mock Oral Curriculum for Senior Surgery Residents: Results of a Pilot Study. J Surg Res 2022; 277:92-99. [DOI: 10.1016/j.jss.2022.03.027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Revised: 02/20/2022] [Accepted: 03/29/2022] [Indexed: 10/18/2022]
|
5
|
Chen JX, Riccardi AC, Shafique N, Gray ST. Are otolaryngology residents ready for independent practice? A survey study. Laryngoscope Investig Otolaryngol 2021; 6:1296-1299. [PMID: 34938865 PMCID: PMC8665426 DOI: 10.1002/lio2.678] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2021] [Revised: 08/04/2021] [Accepted: 09/27/2021] [Indexed: 11/07/2022] Open
Abstract
OBJECTIVE We surveyed otolaryngology program directors (PDs) and recent otolaryngology residency graduates on the operative autonomy of graduating residents and their comfort with independent practice. METHODS An anonymous survey was sent to otolaryngology PDs and recent graduates of training programs (members of the Young Physicians Section [YPS] of the American Academy of Otolaryngology-Head and Neck Surgery Foundation). Questions were developed around the 14 key indicator procedures (KIPs) defined by the Accreditation Council for Graduate Medical Education. RESULTS Fifty PDs (43% of PDs) and 152 recent graduates (6% of YPS members) responded. Over 90% of participating PDs felt their graduating residents were either somewhat or extremely comfortable performing 12 out of 14 KIPs. Among the 12 procedures PDs felt their graduating residents were comfortable performing, 57% to 95% of recent graduates also felt either somewhat or extremely comfortable performing them by graduation. Similarly, at least 90% of responding PDs felt their residents achieved meaningful autonomy in the last 2 months of residency prior to graduation for 11 of 14 KIPs. For these same 11 procedures, 74% to 95% of recent graduates indicated they achieved meaningful autonomy. The procedures that PDs and recent graduates felt required the most surgical assistance were ossiculoplasty/stapedectomy, rhinoplasty, and mastoidectomy. All PDs agreed or strongly agreed that graduating residents are comfortable operating and taking call as general otolaryngologists, compared to 86% and 93% of recent graduates. CONCLUSION Most PDs and recent graduates agree that residents are well-prepared for general otolaryngology practice with the exception of select KIPs.Level of evidence: 4.
Collapse
Affiliation(s)
- Jenny X. Chen
- Department of Otolaryngology–Head and Neck SurgeryMassachusetts Eye and EarBostonMassachusettsUSA
- Department of Otolaryngology–Head and Neck SurgeryHarvard Medical SchoolBostonMassachusettsUSA
| | - Aaliyah C. Riccardi
- Department of OtolaryngologyUniversity of Pittsburgh Medical CenterPittsburghPennsylvaniaUSA
| | - Neha Shafique
- General SurgeryUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Stacey T. Gray
- Department of Otolaryngology–Head and Neck SurgeryMassachusetts Eye and EarBostonMassachusettsUSA
- Department of Otolaryngology–Head and Neck SurgeryHarvard Medical SchoolBostonMassachusettsUSA
| |
Collapse
|
6
|
Hanrahan JG, Sideris M, Pasha T, Dedeilia A, Papalois A, Papalois V. Postgraduate Assessment Approaches Across Surgical Specialties: A Systematic Review of the Published Evidence. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:285-295. [PMID: 32889945 DOI: 10.1097/acm.0000000000003712] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
PURPOSE Ensuring competence for surgical trainees requires holistic assessment of the qualities and competencies necessary to practice safely and effectively. To determine the next steps toward achieving this aim, the authors conducted a systematic review to summarize and appraise the available evidence related to any assessment approach to postgraduate surgical training and to identify the dominant themes for assessment approaches across different specialties or countries. METHOD Medline and Embase were searched on January 10, 2019, without language or time restrictions. Any peer-reviewed study that described an assessment framework (in practice or novel) throughout postgraduate surgical training globally was included. An iterative review and thematic analysis were performed on full-text articles to determine assessment themes. Studies were then grouped by assessment themes. A tailored quality assessment of the studies included in the final analysis was conducted. Assessment themes and validity were compared across surgical specialties and countries. RESULTS From an initial 7,059 articles, 91 studies (evaluating 6,563 surgical trainees) were included in the final analysis. Ten defined assessment themes were extracted. Ten studies (11.0%) were deemed low risk of bias based on the quality assessment tool used and thus were determined to be high quality. Minor differences in assessment themes were observed between specialties and countries. Assessment themes neglected by individual surgical specialties and assessment themes that need validated assessment tools were identified. CONCLUSIONS This review highlights the low quality of evidence and fragmented efforts to develop and optimize surgical assessments. The minor differences observed demonstrate a common approach, globally and across specialties, related to surgical assessments. A paradigm shift in assessment approaches, which will require national and international collaboration, is required to optimize design and validation so that a comprehensive assessment of surgical competence can be implemented.
Collapse
Affiliation(s)
- John Gerrard Hanrahan
- J.G. Hanrahan is academic foundation doctor, Department of General Surgery, Lister Hospital, Stevenage, United Kingdom
| | - Michail Sideris
- M. Sideris is a specialty trainee in obstetrics and gynaecology, Women's Health Research Unit, Queen Mary University of London, London, United Kingdom
| | - Terouz Pasha
- T. Pasha is a final-year medical student, Faculty of Life Sciences and Medicine, King's College London, London, United Kingdom
| | - Aikaterini Dedeilia
- A. Dedeilia is a final-year medical student, School of Medicine, National and Kapodistrian University of Athens, Athens, Greece
| | - Apostolos Papalois
- A. Papalois is director, ELPEN Research & Experimental Centre, Athens, Greece
| | - Vassilios Papalois
- V. Papalois is professor of transplant surgery, Department of Surgery and Cancer, Renal Transplant Directorate, Imperial College Healthcare NHS Trust, London, United Kingdom
| |
Collapse
|
7
|
Miller BL, Azari D, Gerber RC, Radwin R, Le BV. Evidence That Female Urologists and Urology Trainees Tend to Underrate Surgical Skills on Self-Assessment. J Surg Res 2020; 254:255-260. [PMID: 32480069 DOI: 10.1016/j.jss.2020.04.027] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2020] [Revised: 04/02/2020] [Accepted: 04/11/2020] [Indexed: 01/15/2023]
Abstract
BACKGROUND Historically low, the proportion of female urology residents now exceeds 25% in recent years. Self-assessment is a widely used tool to track progress in medical education. However, the validity of its results and gender differences may influence interpretation. Simulation of surgical skills is increasingly common in modern residency training and standardizes certain objective tasks and skills. The objective of this study was to identify gender differences in self-assessment of surgeons and trainees when using simulation of surgical skills. METHODS Medical students, residents, and attending and retired surgeons completed simple interrupted suturing. Assessment was self-rated using previously tested visual analog motion scales. Tasks were video recorded and rated by blinded expert surgeons using identical motion scales. Computer vision motion tracking software was used to objectively analyze the kinematics of surgical tasks. RESULTS Proportion of female (n = 17) and male (n = 20) participants did not differ significantly by the level of training, P = 0.76. Five expert surgeons evaluated 84 video segments of simple interrupted suturing tasks (mean 3.0 segments per task per participant). Self-assessment correlated well overall with expert rating for motion economy (Pearson correlation coefficient 0.61, P < 0.001) and motion fluidity (0.55, P = 0.002). Women underrated their performance in accordance with mean individual difference of self-assessment and expert assessment scores (Δ SAS-EAS) for both economy of motion (mean ± SEM -1.1 ± 0.38, P = 0.01) and fluidity of motion (-1.3 ± 0.39, P < 0.01). On the same measures, men tended to rate themselves in accordance with experts (-0.16 ± 0.36, P = 0.63; -0.09 ± 0.41, P = 0.82, respectively). Δ SAS-EAS did not differ significantly on any rating scale across levels of training. Expert ratings did not differ significantly by gender for any domain. CONCLUSIONS Female surgeons and trainees underrate some technical skills on self-assessment when compared with expert ratings, whereas male surgeon and trainee self-ratings and expert ratings were similar. Further work is needed to determine if these differences are accentuated across increasingly difficult tasks.
Collapse
Affiliation(s)
- Brady L Miller
- Department of Urology, University of Wisconsin, Madison, Wisconsin.
| | - David Azari
- University of Wisconsin, Industrial and Systems Engineering, Madison, Wisconsin
| | - Rebecca C Gerber
- Department of Urology, University of Wisconsin, Madison, Wisconsin
| | - Robert Radwin
- University of Wisconsin, Industrial and Systems Engineering, Madison, Wisconsin; Department of Biomedical Engineering, University of Wisconsin, Madison, Wisconsin
| | - Brian V Le
- Department of Urology, University of Wisconsin, Madison, Wisconsin
| |
Collapse
|
8
|
Williams JC, Ireland T, Warman S, Cake MA, Dymock D, Fowler E, Baillie S. Instruments to measure the ability to self-reflect: A systematic review of evidence from workplace and educational settings including health care. EUROPEAN JOURNAL OF DENTAL EDUCATION : OFFICIAL JOURNAL OF THE ASSOCIATION FOR DENTAL EDUCATION IN EUROPE 2019; 23:389-404. [PMID: 31108006 DOI: 10.1111/eje.12445] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/26/2018] [Accepted: 05/16/2019] [Indexed: 05/17/2023]
Abstract
INTRODUCTION Self-reflection has become recognised as a core skill in dental education, although the ability to self-reflect is valued and measured within several professions. This review appraises the evidence for instruments available to measure the self-reflective ability of adults studying or working within any setting, not just health care. MATERIALS AND METHODS A systematic review was conducted of 20 electronic databases (including Medline, ERIC, CINAHL and Business Source Complete) from 1975 to 2017, supplemented by citation searches. Data were extracted from each study and the studies graded against quality indicators by at least two independent reviewers, using a coding sheet. Reviewers completed a utility analysis of the assessment instruments described within included studies, appraising their reported reliability, validity, educational impact, acceptability and cost. RESULTS A total of 131 studies met the inclusion criteria. Eighteen were judged to provide higher quality evidence for the review and three broad types of instrument were identified, namely: rubrics (or scoring guides), self-reported scales and observed behaviour. CONCLUSIONS Three types of instrument were identified to assess the ability to self-reflect. It was not possible to recommend a single most effective instrument due to under reporting of the criteria necessary for a full utility analysis of each. The use of more than one instrument may therefore be appropriate dependent on the acceptability to the faculty, assessor, student and cost. Future research should report on the utility of assessment instruments and provide guidance on what constitutes thresholds of acceptable or unacceptable ability to self-reflect, and how this should be managed.
Collapse
Affiliation(s)
- Julie C Williams
- Bristol Dental School, Faculty of Health Sciences, University of Bristol, Bristol, UK
| | - Tony Ireland
- Bristol Dental School, Faculty of Health Sciences, University of Bristol, Bristol, UK
| | - Sheena Warman
- Bristol Veterinary School, Faculty of Health Sciences, University of Bristol, Bristol, UK
| | - Martin A Cake
- School of Veterinary and Biomedical Sciences, Murdoch University, Perth, Western Australia, Australia
| | - David Dymock
- Bristol Dental School, Faculty of Health Sciences, University of Bristol, Bristol, UK
| | - Ellayne Fowler
- Centre for Medical Education, University of Bristol, Bristol, UK
| | - Sarah Baillie
- Bristol Veterinary School, Faculty of Health Sciences, University of Bristol, Bristol, UK
| |
Collapse
|
9
|
Johnson CE, Yates K, Sullivan ME. Building a Framework for Self-Regulated Learning in Surgical Education: A Delphi Consensus Among Experts in Surgical Education. JOURNAL OF SURGICAL EDUCATION 2019; 76:e56-e65. [PMID: 31281109 DOI: 10.1016/j.jsurg.2019.06.012] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/23/2019] [Accepted: 06/21/2019] [Indexed: 06/09/2023]
Abstract
OBJECTIVE Practice-Based Learning and Improvement is a Core Competency for surgical residents. Self-regulated learning (SRL) skills are an important component of this competency, yet are rarely taught in surgical training. Before we can teach SRL skills to residents we must understand the attributes that are essential. The purpose of this study was to develop a framework for SRL for surgical trainees. DESIGN This mixed-methods study design utilized a two-round modified-Delphi approach to develop consensus among experts in surgical education regarding SRL in surgical training. Round One included SRL constructs derived from educational, professional, and medical literature. Based upon quantitative data and thematic coding of comments, these constructs were adapted for applicability in the context of surgical residency and reorganized using a constant comparative approach. Revised constructs and groupings were presented to the expert panel in Round Two. Further survey rounds were not needed as all items in Round Two reached the predetermined consensus level of 70%. SETTING The Delphi panel was a purposeful sample of nationally recognized experts in surgical education, including members of the Association for Surgical Education and the Association for Program Directors in Surgery. PARTICIPANTS Thirty-eight of 42 experts (90.5%) responded to Round One, representing 29 academic and community medical systems nationally. The response rate for Round Two was 92%, 35 of the 38 Round One participants. RESULTS In Round One, the SRL constructs were all viewed as important with median scores ranging from 50 to 99.5, on a 100-point scale. Two hundred and ninety-one comments were coded and used to refine SRL definitions into 7 domains for Round Two, which included self-awareness, task analysis, situation awareness, strategic planning, progress evaluation, learning and performance management, and goal attainment and refinement. All Round Two items reached greater than 70% agreement, and received 51 free response comments. Several key themes emerged: clinical prioritization over learning, learner's limited control, value and reliance on external resources, low use of metacognition, and complex goal orientation. Incorporation of common themes generated a novel multi-stage framework of SRL in surgical education. CONCLUSIONS Surgical residency represents a unique learning context, in which the ideal learner is one who understands their learning environment and utilizes available resources to optimize their own learning. Experts in surgical education believe SRL skills are important in training, and a novel framework of SRL is necessary to support a learner-centered model within the demanding environment of surgical training.
Collapse
Affiliation(s)
- Cali E Johnson
- Department of Surgery, Keck School of Medicine, and Rossier School of Education, University of Southern California, Los Angeles, California.
| | - Kenneth Yates
- Department of Surgery, Keck School of Medicine, and Rossier School of Education, University of Southern California, Los Angeles, California
| | - Maura E Sullivan
- Department of Surgery, Keck School of Medicine, and Rossier School of Education, University of Southern California, Los Angeles, California
| |
Collapse
|
10
|
Bull NB, Silverman CD, Bonrath EM. Targeted surgical coaching can improve operative self-assessment ability: A single-blinded nonrandomized trial. Surgery 2019; 167:308-313. [PMID: 31570149 DOI: 10.1016/j.surg.2019.08.002] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2019] [Revised: 08/04/2019] [Accepted: 08/07/2019] [Indexed: 01/17/2023]
Abstract
BACKGROUND The aim of the study was to investigate the effect of targeted surgical coaching on self-assessment of laparoscopic operative skill. Accurate self-assessment is vital for autonomous professional development. Surgical coaching can be used for performance improvement, but its role in this domain has been insufficiently investigated. METHODS This was a single site, nonrandomized, interrupted time series design trial. Participants were residents, fellows, and attending surgeons regularly performing laparoscopic general surgery operations. Each participant was enrolled in an individualized coaching program using review of personal and peer laparoscopic videos. The program involved 3 to 5 sessions over a period of 6 to 19 weeks. Coaching used case debriefing to target self-assessment proficiency, with a focus on objective interpretation of observations and facilitative capacity building. The primary outcome measure was self-assessment accuracy and correlation to expert ratings. The Objective Structured Assessment of Technical Skill global rating scale was utilized for evaluation. RESULTS Twelve participants were recruited and completed the coaching program. At baseline, there was no correlation between self-assessment and expert ratings. After completion of the coaching program there was correlation between self-assessment and expert ratings (P = .003) and improved self-assessment accuracy compared to baseline (P = .041). CONCLUSION This study has demonstrated that targeted coaching using video review of laparoscopic cases can improve operative self-assessment accuracy using the Objective Structured Assessment of Technical Skill.
Collapse
Affiliation(s)
- Nicholas B Bull
- Department of General Surgery, The Tweed Hospital, Tweed Heads, NSW, Australia.
| | - Candice D Silverman
- Department of General Surgery, The Tweed Hospital, Tweed Heads, NSW, Australia; Practice of Candice D. Silverman, John Flynn Medical Centre, Tugun, QLD, Australia
| | - Esther M Bonrath
- Practice of Candice D. Silverman, John Flynn Medical Centre, Tugun, QLD, Australia; Department of General and Visceral Surgery, Krankenhaus Barmherzige Brueder, Regensburg, Germany
| |
Collapse
|
11
|
Srikumaran D, Tian J, Ramulu P, Boland MV, Woreta F, Wang KM, Mahoney N. Ability of Ophthalmology Residents to Self-Assess Their Performance Through Established Milestones. JOURNAL OF SURGICAL EDUCATION 2019; 76:1076-1087. [PMID: 30850245 DOI: 10.1016/j.jsurg.2018.12.004] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/16/2018] [Revised: 12/05/2018] [Accepted: 12/08/2018] [Indexed: 06/09/2023]
Abstract
OBJECTIVES Accurate self-assessment is an important aspect of practice-based learning and improvement and a critical skill for resident growth. The Accreditation Council for Graduate Medical Education mandates semiannual milestones assessments by a clinical competency committee (CCC) for all ophthalmology residents. There are six core competencies: patient care (PC), medical knowledge, systems-based practice, practice-based learning and improvement, professionalism, and interpersonal communication skills. These competencies are assessed by the milestones rubric, which has detailed behavioral anchors and are also used for trainee self-assessments. This study compares resident self-assessed (SA) and faculty CCC milestones scores. DESIGN Residents completed milestones self-assessments prior to receiving individual score reports from the CCC. Correlation coefficients were calculated comparing the SA and CCC scores. In addition, statistical models were used to determine predictors of disparities and differences between the SA and CCC scores. SETTING Wilmer Eye Institute, Johns Hopkins Hospital. PARTICIPANTS Twenty-one residents in the Wilmer Ophthalmology Residency program from July 2014 to June 2016. RESULTS Fifty-seven self-assessments were available for the analysis. For each resident's first assessment, SA and CCC scores were strongly correlated (r ≥ 0.6 and p < 0.05) for four milestones, and not correlated for the remaining 20 milestones. In multivariable models, the SA and CCC scores are less disparate for medical knowledge and systems-based practice competencies compared to practice-based learning and improvement. Higher year of training, PC and professionalism competencies were predictive of statistically significant resident overestimation of scores relative to the CCC. In addition, higher CCC scores predicted statistically significant lower SA-CCC disparities and differences. SA-CCC differences did not lower to a significant extent with repeated assessments or modification to the end-of-rotation evaluation forms. CONCLUSIONS Self-assessments by ophthalmology residents are not well-correlated with faculty assessments, emphasizing the need for improved and frequent timely feedback. Residents have the greatest difficulty self-assessing their professionalism and PC competency. In general, senior residents and underperforming residents have more inaccurate self-assessments.
Collapse
Affiliation(s)
- Divya Srikumaran
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland.
| | - Jing Tian
- Wilmer Biostatistics Center, Johns Hopkins School of Public Health, Baltimore, Maryland
| | - Pradeep Ramulu
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Michael V Boland
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Fasika Woreta
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Kendrick M Wang
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Nicholas Mahoney
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland
| |
Collapse
|
12
|
Rosenberger K, Skinner D, Monk J. Ready for Residency: A Bloomian Analysis of Competency-Based Osteopathic Medical Education. J Osteopath Med 2018; 117:529-536. [PMID: 28759095 DOI: 10.7556/jaoa.2017.103] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
Context Bloom's Taxonomy is a widely accepted tool for analyzing learning objectives, creating assessment materials, and ensuring that students move progressively through various levels of knowledge and cognition. Competency-based osteopathic medical education has, to the authors' knowledge, yet to be subjected to systematic bloomian analysis. Objective To advance a bloomian analysis of competencies published by the American Association of Colleges of Osteopathic Medicine (AACOM) and draw consequences of that analysis for undergraduate osteopathic medical education. Methods Content analysis of the verbs and contexts used in the AACOM competencies was conducted, followed by categorization by the various cognitive and knowledge dimensions as specified by the Revised Bloom's Taxonomy. Results The majority of AACOM competencies gravitate toward the taxonomy's apply and procedural cognitive and knowledge dimensions, respectively. In addition, 58% of the top 10 most-used verbs in the document were identified to be in the apply cognitive dimension. An analysis of the competencies revealed that 69% can be categorized under the procedural knowledge dimension. Of 657 code co-occurances, 45% were from the apply cognitive and procedural knowledge dimensions. These figures suggest a skewing toward application-based cognitive and procedural-based knowledge over other types of cognitive and knowledge dimensions. Conclusion Bloom's Taxonomy offers a useful framework for understanding how competency-based osteopathic medical education is organized. Such analysis underscores the importance of attending to the language and construction of competencies to better understand how the language of competencies shapes pedagogic practices and begin a broader conversation about the appropriate use and distribution of lower- and higher-order cognitive processes within competency-based osteopathic medical education.
Collapse
|
13
|
Rui M, Lee JE, Vauthey JN, Conrad C. Enhancing surgical performance by adopting expert musicians' practice and performance strategies. Surgery 2018; 163:894-900. [DOI: 10.1016/j.surg.2017.09.011] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2017] [Revised: 09/14/2017] [Accepted: 09/25/2017] [Indexed: 01/26/2023]
|
14
|
Emke AR, Cheng S, Chen L, Tian D, Dufault C. A Novel Approach to Assessing Professionalism in Preclinical Medical Students Using Multisource Feedback Through Paired Self- and Peer Evaluations. TEACHING AND LEARNING IN MEDICINE 2017; 29:402-410. [PMID: 28498003 DOI: 10.1080/10401334.2017.1306446] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
UNLABELLED Phenomenon: Professionalism is integral to the role of the physician. Most professionalism assessments in medical training are delayed until clinical rotations where multisource feedback is available. This leaves a gap in student assessment portfolios and potentially delays professional development. APPROACH A total of 246 second-year medical students (2013-2015) completed self- and peer assessments of professional behaviors in 2 courses following a series of Team-Based Learning exercises. Correlation and regression analyses were used to examine the alignment or misalignment in the relationship between the 2 types of assessments. Four subgroups were formed based on observed patterns of initial self- and peer assessment alignment or misalignment, and subgroup membership stability over time was assessed. A missing data analysis examined differences between average peer assessment scores as a function of selective nonparticipation. FINDINGS Spearman correlation demonstrated moderate to strong correlation between self-assessments completed alone (no simultaneous peer assessment) and self-assessments completed at the time of peer assessments (ρ = .59, p < .0001) but weak correlation between the two self-assessments and peer assessments (alone: ρ = .13, p < .013; at time of peer: ρ = .21, p < .0001). Generalized estimating equation models revealed that self-assessments done alone (p < .0001) were a significant predictor of self-assessments done at the time of peer. Course was also a significant predictor (p = .01) of self-assessment scores done at the time of peer. Peer assessment score was not a significant predictor. Bhapkar's test revealed subgroup membership based on the relationship between self- and peer ratings was relatively stable across Time 1 and Time 2 assessments (χ2 = 0.83, p = .84) for all but one subgroup; members of the subgroup with initially high self-assessment and low peer assessment were significantly more likely to move to a new classification at the second measurement. A missing data analysis revealed that students who completed all self-assessments had significantly higher average peer assessment ratings compared to students who completed one or no self-assessments with a difference of -0.32, 95% confidence interval [-0.48, -0.15]. Insights: Multiple measurements of simultaneous self- and peer assessment identified a subgroup of students who consistently rated themselves higher on professionalism attributes relative to the low ratings given by their peers. This subgroup of preclinical students, along with those who elected to not complete self-assessments, may be at risk for professionalism concerns. Use of this multisource feedback tool to measure perceptual stability of professionalism behaviors is a new approach that may assist with early identification of at-risk students during preclinical years.
Collapse
Affiliation(s)
- Amanda R Emke
- a Department of Pediatrics , Washington University School of Medicine , St. Louis , Missouri , USA
| | - Steven Cheng
- b Department of Internal Medicine, Division of Renal Diseases , Washington University School of Medicine , St. Louis , Missouri , USA
| | - Ling Chen
- c Division of Biostatistics , Washington University School of Medicine , St. Louis Missouri , USA
| | - Dajun Tian
- c Division of Biostatistics , Washington University School of Medicine , St. Louis Missouri , USA
| | - Carolyn Dufault
- d Department of Internal Medicine, Division of Medical Education , Washington University School of Medicine , St. Louis , Missouri , USA
| |
Collapse
|
15
|
Does One Size Fit All? Examining the Application of Neurosurgery Residency Milestones Developed in the United States to a Taiwanese Culture. World Neurosurg 2017; 104:499-508. [PMID: 28461272 DOI: 10.1016/j.wneu.2017.04.129] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2017] [Revised: 04/18/2017] [Accepted: 04/20/2017] [Indexed: 11/22/2022]
Abstract
BACKGROUND The Milestone Project was launched in 2009, charging specialties to develop specific educational accomplishments required to establish clinical competency. The milestone assessment method was first introduced to Taiwan in 2013 and before applying milestone assessments to our medical education system, the validity and reliability of these questionnaires needed to be evaluated. METHODS Twenty neurosurgical faculty members representing 3 clinical divisions and all 4 branch institutes completed milestone questionnaires for 26 residents semiannually, resulting in 435 resident assessments being collected and analyzed. RESULTS Cronbach's α, Kuder-Richardson Formula 20, and Kendall's W were used to show acceptable reliability and validity. Rater consistencies for nonskilled parts found that rater consistency progressively improved with time. Not all raters were able to assess the residents for the skilled parts resulting in nonassessable rates ranging from 9.5%-89.4%. For nonskilled and skilled items, milestone level as assessed by the staff improved as the resident progressed from residency year 3 (R3) to R6 in the residency program and showed that the milestone achievement level for an R3 was lower than that of an R6. CONCLUSIONS Milestone assessments have high reliability and may be a helpful assessment tool. Although milestone assessment can provide thorough feedback concerning performance and the content of the training program, they may not perfectly suit all residency-training programs, especially in different countries or different cultures. Modifications should be done before applying milestones to different areas; therefore the results can truly reflect the progress and condition of the training and learning process.
Collapse
|
16
|
Robinson WP, Doucet DR, Simons JP, Wyman A, Aiello FA, Arous E, Schanzer A, Messina LM. An intensive vascular surgical skills and simulation course for vascular trainees improves procedural knowledge and self-rated procedural competence. J Vasc Surg 2017; 65:907-915.e3. [DOI: 10.1016/j.jvs.2016.12.065] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2016] [Accepted: 12/16/2016] [Indexed: 11/26/2022]
|
17
|
Bradley KE, Andolsek KM. A pilot study of orthopaedic resident self-assessment using a milestones' survey just prior to milestones implementation. INTERNATIONAL JOURNAL OF MEDICAL EDUCATION 2016; 7:11-8. [PMID: 26752012 PMCID: PMC4715902 DOI: 10.5116/ijme.5682.6dfd] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/26/2015] [Accepted: 12/29/2015] [Indexed: 05/17/2023]
Abstract
OBJECTIVE To pilot test if Orthopaedic Surgery residents could self-assess their performance using newly created milestones, as defined by the Accreditation Council on Graduate Medical Education. METHODS In June 2012, an email was sent to Program Directors and administrative coordinators of the 154 accredited Orthopaedic Surgery Programs, asking them to send their residents a link to an online survey. The survey was adapted from the Orthopaedic Surgery Milestone Project. Completed surveys were aggregated in an anonymous, confidential database. SAS 9.3 was used to perform the analyses. RESULTS Responses from 71 residents were analyzed. First and second year residents indicated through self-assessment that they had substantially achieved Level 1 and Level 2 milestones. Third year residents reported they had substantially achieved 30/41, and fourth year residents, all Level 3 milestones. Fifth year, graduating residents, reported they had substantially achieved 17 Level 4 milestones, and were extremely close on another 15. No milestone was rated at Level 5, the maximum possible. Earlier in training, Patient Care and Medical Knowledge milestones were rated lower than the milestones reflecting the other four competencies of Practice Based Learning and Improvement, Systems Based Practice, Professionalism, and Interpersonal Communication. The gap was closed by the fourth year. CONCLUSIONS Residents were able to successfully self-assess using the 41 Orthopaedic Surgery milestones. Respondents' rate improved proficiency over time. Graduating residents report they have substantially, or close to substantially, achieved all Level 4 milestones. Milestone self-assessment may be a useful tool as one component of a program's overall performance assessment strategy.
Collapse
Affiliation(s)
- Kendall E. Bradley
- Orthopaedic Surgery Residency Training Program, Duke University Hospital, Box 3000 DUMC, Durham, NC 27710, USA
| | - Kathryn M. Andolsek
- Community and Family Medicine, Duke University School of Medicine, Box 3648 DUMC, 201 Trent Drive, Durham, NC 27710, USA
| |
Collapse
|
18
|
Hu Y, Kim H, Mahmutovic A, Choi J, Le I, Rasmussen S. Verification of accurate technical insight: a prerequisite for self-directed surgical training. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2015; 20:181-191. [PMID: 24903585 PMCID: PMC4258168 DOI: 10.1007/s10459-014-9519-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/30/2013] [Accepted: 05/26/2014] [Indexed: 06/01/2023]
Abstract
Simulation-based surgical skills training during preclinical education is a persistent challenge due to time constraints of trainees and instructors alike. Self-directed practice is resource-efficient and flexible; however, insight into technical proficiency among trainees is often lacking. The purpose of this study is to prospectively assess the accuracy of self-assessments among medical students learning basic surgical suturing. Over seven weekly practice sessions, preclinical medical students performed serial repetitions of a simulation-based suturing task under one-on-one observation by one of four trainers. Following each task repetition, self- and trainer-assessments (SA-TA) were performed using a 36-point weighted checklist of technical standards developed a priori by expert consensus. Upon study completion, agreement between SA and TA was measured using weighted Cohen's kappa coefficients. Twenty-nine medical students each performed a median of 25 suture task repetitions (IQR 21.5-28). Self-assessments tended to overestimate proficiency during the first tertile of practice attempts. Agreement between SA and TA improved with experience, such that the weighted kappa statistics for the two-handed and instrument ties were >0.81 after 18-21 task attempts. Inexperienced trainees frequently overestimate technical proficiency through self-assessments. However, this bias diminishes with repetitive practice. Only after trainees have attained the capacity to accurately self-assess can effective self-directed learning take place.
Collapse
Affiliation(s)
- Yinin Hu
- Department of Surgery, University of Virginia School of Medicine, PO Box 800709, Charlottesville, VA, 22908-0679, USA,
| | | | | | | | | | | |
Collapse
|