1
|
Wang TN, Woelfel IA, Huang E, Pieper H, Meara MP, Chen X(P. Behind the pattern: General surgery residsent autonomy in robotic surgery. Heliyon 2024; 10:e31691. [PMID: 38841510 PMCID: PMC11152925 DOI: 10.1016/j.heliyon.2024.e31691] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Revised: 05/12/2024] [Accepted: 05/20/2024] [Indexed: 06/07/2024] Open
Abstract
Objective Robotic surgery is increasingly utilized and common in general surgery training programs. This study sought to better understand the factors that influence resident operative autonomy in robotic surgery. We hypothesized that resident seniority, surgeon work experience, surgeon robotic-assisted surgery (RAS) case volume, and procedure type influence general surgery residents' opportunities for autonomy in RAS as measured by percentage of resident individual console time (ICT). Methods General surgery resident ICT data for robotic cholecystectomy (RC), inguinal hernia (RIH), and ventral hernia (RVH) operations performed on the dual-console Da Vinci surgical robotic system between July 2019 and June 2021 were extracted. Cases with postgraduate year (PGY) 2-5 residents participating as a console surgeon were included. A sequential explanatory mixed-methods approach was undertaken to explore the ICT results and we conducted secondary qualitative interviews with surgeons. Descriptive statistics and thematic analysis were applied. Results Resident ICT data from 420 robotic cases (IH 200, RC 121, and VH 99) performed by 20 junior residents (PGY2-3), 18 senior residents (PGY4-5), and 9 attending surgeons were extracted. The average ICT per case was 26.8 % for junior residents and 42.4 % for senior residents. Compared to early-career surgeons, surgeons with over 10 years' work experience gave less ICT to junior (18.2 % vs. 32.0 %) and senior residents (33.9 % vs. 56.6 %) respectively. Surgeons' RAS case volume had no correlation with resident ICT (r = 0.003, p = 0.0003). On average, residents had the most ICT in RC (45.8 %), followed by RIH (36.7 %) and RVH (28.6 %). Interviews with surgeons revealed two potential reasons for these resident ICT patterns: 1) Surgeon assessment of resident training year/experience influenced decisions to grant ICT; 2) Surgeons' perceived operative time pressure inversely affected resident ICT. Conclusions This study suggests resident ICT/autonomy in RC, RIH, and RVH are influenced by resident seniority level, surgeon work experience, and procedure type, but not related to surgeon RAS case volume. Design and implementation of an effective robotic training program must consider the external pressures at conflict with increased resident operative autonomy and seek to mitigate them.
Collapse
Affiliation(s)
- Theresa N. Wang
- The Ohio State University Wexner Medical Center, Department of Surgery, Columbus, OH, USA
| | - Ingrid A. Woelfel
- The Ohio State University Wexner Medical Center, Department of Surgery, Columbus, OH, USA
| | - Emily Huang
- The Ohio State University Wexner Medical Center, Department of Surgery, Columbus, OH, USA
| | - Heidi Pieper
- The Ohio State University Wexner Medical Center, Department of Surgery, Columbus, OH, USA
| | - Michael P. Meara
- The Ohio State University Wexner Medical Center, Department of Surgery, Columbus, OH, USA
| | | |
Collapse
|
2
|
D’Abbondanza JA, Shih JG, Knox ADC, Zhygan N, Brown MH, Fish JS, Courtemanche DJ. Resident Exposure and Involvement in Core Procedural Competencies within Pediatric Plastic Surgery. Plast Surg (Oakv) 2024; 32:347-354. [PMID: 38681244 PMCID: PMC11046279 DOI: 10.1177/22925503221109072] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2022] [Revised: 05/02/2022] [Accepted: 05/11/2022] [Indexed: 05/01/2024] Open
Abstract
Introduction: The implementation of competency-based residency training in plastic surgery is underway. Key competencies in plastic surgery have been previously identified, however, within the domain of pediatrics, data suggest limited exposure throughout training for Canadian graduates. This study aims to identify the exposure and involvement of residents in core pediatric cases. Methods: We performed a retrospective, multicenter review of plastic surgery resident case logs (T-Res, POWER, New Innovations) across 10 Canadian, English-speaking training programs between 2004 and 2014. Case logs were coded according to the 8 core pediatric competencies previously identified by a modified Delphi technique. Results: A total of 3061 of 59 405 cases (5.2%) logged by 55 graduating residents were core pediatric procedures with an average of 55.6 ± 23.0 cases logged per resident. The top 3 most commonly logged procedures were cleft lip repair, cleft palate repair, and setback otoplasty. The number of cases per program varied widely with the most at 731 and least at 85 logged cases. Roles across procedures have wide variation and residents are most commonly identified as the assistant rather than surgeon or co-surgeon. Conclusion: These findings highlight variability both within and across residency programs with a paucity of exposure and involvement in pediatric plastic surgery cases. This may present a conflict between current recommendations for residency-specific procedural competencies and true clinical exposure. Further curriculum development and simulation may be of benefit.
Collapse
Affiliation(s)
- Josephine A. D’Abbondanza
- Division of Plastic, Reconstructive and Aesthetic Surgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada
| | - Jessica G. Shih
- Division of Plastic, Reconstructive and Aesthetic Surgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada
| | - Aaron D. C. Knox
- Division of Plastic and Reconstructive Surgery, Department of Surgery, University of Calgary, Calgary, Alberta, Canada
| | - Nick Zhygan
- Division of Plastic and Reconstructive Surgery, Department of Surgery, University of British Columbia, Victoria, British Columbia, Canada
| | - Mitchell H. Brown
- Division of Plastic, Reconstructive and Aesthetic Surgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada
| | - Joel S. Fish
- Division of Plastic, Reconstructive and Aesthetic Surgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada
| | - Douglas J. Courtemanche
- Division of Plastic and Reconstructive Surgery, Department of Surgery, University of British Columbia, Victoria, British Columbia, Canada
| |
Collapse
|
3
|
Sevestre A, Dochez V, Souron R, Deschamps T, Winer N, Thubert T. Evaluation Tools for Assessing Autonomy of Surgical Residents in the Operating Room and Factors Influencing Access to Autonomy: A Systematic Literature Review. JOURNAL OF SURGICAL EDUCATION 2024; 81:182-192. [PMID: 38160113 DOI: 10.1016/j.jsurg.2023.11.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/03/2023] [Revised: 10/16/2023] [Accepted: 11/01/2023] [Indexed: 01/03/2024]
Abstract
BACKGROUND Surgical residents in France lack a clear pedagogical framework for achieving autonomy in the operating room. The progressive acquisition of surgical autonomy is a determining factor in the confidence of operators for their future independent practice. Currently, there is no autonomy scale commonly used in Europe. The objective of this study is to identify existing tools for quantifying the autonomy of residents and the factors that influence it. MATERIALS AND METHODS We conducted a qualitative systematic review following the recommendations of the Systematic Review Without Meta-Analysis (SWiM) guidelines. Publications were extracted from the MEDLINE (PubMed), EMBASE, and PSYCINFO databases. All publications without date restrictions up to July 2022 were identified. RESULTS Among the 231 identified publications, 21 met the inclusion criteria. Seventeen publications used a graded autonomy assessment tool by the student and/or the teacher, while 4 used evaluations by an observing third party. We found 8 different autonomy scales, with the Zwisch Scale representing 57.1% of the cases. Factors influencing autonomy were diverse, including the work context, experience, and gender of the resident and their teacher. DISCUSSION We found heterogeneity in the tools used to "measure" the autonomy of a resident in the operating room. The SIMPL tool or the Zwisch Scale appear to be the most frequently used tools. The relationship between autonomy, performance, confidence, and knowledge may require multidimensional tools that encompass various areas of competence, but this could make their daily application more challenging. The factors influencing autonomy are numerous; and understanding them would improve teaching in the operating room. There is a significant lack of data on surgical autonomy in France, as well as a lack of evaluation in the field of gynecology-obstetrics worldwide.
Collapse
Affiliation(s)
- Anaïs Sevestre
- Nantes Université, CHU Nantes, Service de Gynécologie-Obstétrique, Nantes, France; Nantes Université, CHU Nantes, Movement - Interactions - Performance, Nantes, France
| | - Vincent Dochez
- Nantes Université, CHU Nantes, Service de Gynécologie-Obstétrique, Nantes, France; Nantes Université, CHU Nantes, Movement - Interactions - Performance, Nantes, France.
| | - Robin Souron
- Nantes Université, CHU Nantes, Movement - Interactions - Performance, Nantes, France
| | - Thibault Deschamps
- Nantes Université, CHU Nantes, Movement - Interactions - Performance, Nantes, France
| | - Norbert Winer
- Nantes Université, CHU Nantes, Service de Gynécologie-Obstétrique, Nantes, France; Nantes Université, CHU Nantes, INRAE, Nantes, France
| | - Thibault Thubert
- Nantes Université, CHU Nantes, Service de Gynécologie-Obstétrique, Nantes, France; Nantes Université, CHU Nantes, Movement - Interactions - Performance, Nantes, France
| |
Collapse
|
4
|
Keuning MC, Lambert B, Nieboer P, Huiskes M, Diemers AD. Perceptions and Guiding Strategies to Regulate Entrusted Autonomy of Residents in the Operating Room: A Systematic Literature Review. JOURNAL OF SURGICAL EDUCATION 2024; 81:93-105. [PMID: 37838573 DOI: 10.1016/j.jsurg.2023.09.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/17/2022] [Revised: 12/24/2022] [Accepted: 09/11/2023] [Indexed: 10/16/2023]
Abstract
OBJECTIVE To provide a systematic literature review of intraoperative entrusted autonomy for surgical residents. Specifically, perceptions from residents and supervising surgeons, supervising behavior and influencing factors on intraoperative teaching and learning are analyzed. BACKGROUND Increasing demands on surgical training and the need for effective development of technical skills, amplify the importance of making the most of intraoperative teaching and learning opportunities in the operating room. It is critical for residents to gain the greatest benefit from every surgical case and to achieve operative competence. METHODS A systematic literature search identified 921 articles from 2000 to 2022 that addressed surgical education/training, intraoperative supervision/teaching, autonomy and entrustment. 40 studies with heterogeneous designs and methodologies were included. RESULTS Four themes were established in the analysis: patient safety, learner, learning environment and supervising surgeon. The patient is identified as the primary responsibility during intraoperative teaching and learning. Supervisors continuously guard patient safety as well as the resident's learning process. Ideal intraoperative learning occurs when the resident has optimal entrusted autonomy during the procedure matching with the current surgical skills level. A safe learning environment with dedicated time for learning are prerequisites for both supervising surgeons and residents. Supervising surgeons' own preferences and confidence levels also play an important role. CONCLUSIONS This systematic literature review identifies patient safety as the overriding principle for supervising surgeons when regulating residents' entrusted autonomy. When the supervisor's responsibility toward the patient has been met, there is room for intraoperative teaching and learning. In this process the learner, the learning environment and the supervising surgeon's own preferences all intertwine, creating a triangular responsibility. This review outlines the challenge of establishing an equilibrium in this triangle and the broad arsenal of strategies supervising surgeons use to keep it in balance.
Collapse
Affiliation(s)
- Martine C Keuning
- Department of Orthopedic Surgery, University Medical Center Groningen, Groningen, The Netherlands.
| | - Bart Lambert
- Department of Orthopedic Surgery, University Medical Center Groningen, Groningen, The Netherlands
| | - Patrick Nieboer
- Department of Orthopedic Surgery, University Medical Center Groningen, Groningen, The Netherlands
| | - Mike Huiskes
- Center for Language and Cognition Groningen, University of Groningen, Groningen, The Netherlands
| | - Agnes D Diemers
- Lifelong Learning, Education and Assessment Research Network (LEARN), University Medical Center Groningen, Groningen, The Netherlands
| |
Collapse
|
5
|
Farid H, Rajagopalan S, Dalrymple JL. Procedural Teaching: Focusing on the Conscious Patient. JOURNAL OF SURGICAL EDUCATION 2023; 80:1745-1747. [PMID: 37634976 DOI: 10.1016/j.jsurg.2023.08.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 07/31/2023] [Accepted: 08/01/2023] [Indexed: 08/29/2023]
Abstract
OBJECTIVE In this perspective, the authors discuss how to teach procedures at the bedside with an awake patient. DESIGN/SETTING The teaching process is divided into 3 stages: preprocedural, intraprocedural, and postprocedural. PARTICIPANTS Each stage focuses on a specific set of educational goals and aims for the learner, with specific tips for how the faculty member should be teaching the learner. We discuss how to deal with challenging situations, such as when the faculty member needs to take over the procedure, and how to allow the learner to troubleshoot if they come across the unexpected. CONCLUSION With the guidance provided in this perspective, we aim to make procedural teaching at the bedside less daunting for faculty members.
Collapse
Affiliation(s)
- Huma Farid
- Department of Obstetrics and Gynecology, Beth Israel Deaconess Medical School, Harvard Medical School, Boston, Massachusetts.
| | - Supraja Rajagopalan
- Department of Obstetrics and Gynecology, Beth Israel Deaconess Medical School, Harvard Medical School, Boston, Massachusetts
| | - John L Dalrymple
- Department of Obstetrics and Gynecology, Beth Israel Deaconess Medical School, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
6
|
Wang TN, Woelfel IA, Pieper H, Haisley KR, Meara MP, Chen XP. Is Robotic Console Time a Surrogate for Resident Operative Autonomy? JOURNAL OF SURGICAL EDUCATION 2023; 80:1711-1716. [PMID: 37296003 DOI: 10.1016/j.jsurg.2023.05.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/16/2023] [Revised: 04/18/2023] [Accepted: 05/08/2023] [Indexed: 06/12/2023]
Abstract
OBJECTIVE Robotic-assisted surgery is an increasing part of general surgery training, but resident autonomy on the robotic platform can be hard to quantify. Robotic console time (RCT), the percentage of time the resident controls the console, may be an appropriate measure of resident operative autonomy. This study aims to characterize the correlation between objective resident RCT and subjectively scored operative autonomy. METHODS Using a validated resident performance evaluation instrument, we collected resident operative autonomy ratings from residents and attendings performing robotic cholecystectomy (RC) and robotic inguinal hernia repair (IH) at a university-based general surgery program between 9/2020-6/2021. We then extracted RCT data from the Intuitive surgical system. Descriptive statistics, t-tests and ANOVA were performed. RESULTS A total of 31 robotic operations (13 RC, 18 IH) performed by 4 attending surgeons and 8 residents (4 junior, 4 senior) were matched and included. 83.9% of cases were scored by both attending and resident. The average RCT per case was 35.6%(95% CI 13.0%,58.3%) for junior residents (PGY 2-3) and 59.7%(CI 51.1%,68.3%) for senior residents (PGY 4-5). The mean autonomy evaluated by residents was 3.29(CI 2.85,3.73) out of a maximum score of 5, while the mean autonomy evaluated by attendings was 4.12(CI 3.68,4.55). RCT significantly correlated with subjective evaluations of resident autonomy (r=0.61, p=0.0003). RCT also moderately correlated with resident training level (r=0.5306, p<0.0001). Neither attending robotic experience nor operation type significantly correlated with RCT or autonomy evaluation scores. CONCLUSIONS Our study suggests that resident console time is a valid surrogate for resident operative autonomy in robotic cholecystectomy and inguinal hernia repair. RCT may be a valuable measure in objective assessment of residents' operative autonomy and training efficiency. Future investigation into how RCT correlates with subjective and objective autonomy metrics such as verbal guidance or distinguishing critical operative steps is needed to validate the study findings further.
Collapse
Affiliation(s)
- Theresa N Wang
- Department of Surgery, The Ohio State University Wexner Medical Center, Columbus, Ohio.
| | - Ingrid A Woelfel
- Department of Surgery, The Ohio State University Wexner Medical Center, Columbus, Ohio
| | - Heidi Pieper
- Department of Surgery, The Ohio State University Wexner Medical Center, Columbus, Ohio
| | - Kelly R Haisley
- Department of Surgery, The Ohio State University Wexner Medical Center, Columbus, Ohio
| | - Michael P Meara
- Department of Surgery, The Ohio State University Wexner Medical Center, Columbus, Ohio
| | - Xiaodong Phoenix Chen
- Department of Surgery, The Ohio State University Wexner Medical Center, Columbus, Ohio
| |
Collapse
|
7
|
Woelfel I, Wang T, Pieper H, Meara M, Chen XP. Distortions in the Balance Between Teaching and Efficiency in the Operating Room. J Surg Res 2023; 283:110-117. [PMID: 36402083 DOI: 10.1016/j.jss.2022.10.032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2022] [Revised: 08/20/2022] [Accepted: 10/17/2022] [Indexed: 11/18/2022]
Abstract
INTRODUCTION The balance between teaching and operative efficiency (i.e., continuing operative case progression) is difficult for even the most experienced master surgeon educators. The purpose of this study was to explore influencing factors behind attending surgeons' decisions to break the balance between operative efficiency and teaching in the operating room. METHODS Semistructured interviews were conducted with surgeons across the United States via Web-based video conferencing. The interviews were audio-recorded and transcribed. Qualitative analysis using the framework method was utilized, and emergent themes were identified. RESULTS Twenty-three attending surgeons from 8 academic institutions and 11 surgical specialties completed interviews (14 men and 9 women). Attending surgeons consider a variety of factors associated with their dual roles (surgeon versus teacher) when balancing operative efficiency and providing appropriate independence for residents with oversight to promote autonomy. These were divided into surgeon-role-related factors (patient safety, financial factors, scheduling factors, preservation of faculty reputation for efficiency, and mode of operation) as well as teacher-role-related factors (preparation, level, and technical skill of the resident). These factors then informed attending surgeons' determinations about how the case was progressing, which prompted them to intervene and reduce resident autonomy or allow the resident to continue. CONCLUSIONS Surgeons consider numerous factors when deciding how to balance resident teaching and autonomy while preserving operative efficiency. These findings provide helpful insights for surgical departments to consider inclusion in faculty-development programs, resident education, and systematic improvements.
Collapse
Affiliation(s)
- Ingrid Woelfel
- Department of Surgery, Ohio State University, Columbus, Ohio.
| | - Theresa Wang
- Department of Surgery, Ohio State University, Columbus, Ohio
| | - Heidi Pieper
- Department of Surgery, Ohio State University, Columbus, Ohio
| | - Michael Meara
- Department of Surgery, Ohio State University, Columbus, Ohio
| | | |
Collapse
|
8
|
Cost analysis of training residents in robotic-assisted surgery. Surg Endosc 2022; 37:2765-2769. [PMID: 36471060 DOI: 10.1007/s00464-022-09794-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Accepted: 11/27/2022] [Indexed: 12/12/2022]
Abstract
INTRODUCTION Use of robotic-assisted surgery is increasing, and resident involvement may lead to higher costs. We investigated whether senior resident involvement in noncomplex robotic cholecystectomy (RC) and inguinal hernia (RIH) would take more time and cost more when compared to non-robotic cholecystectomy (NRC) and inguinal hernia repair (NRIH). METHODS We extracted surgery duration and total cost of NRC, NRIH, RC, and RIH from 7/2016 to 6/2020 with senior resident (PGY4-5) involvement. We excluded complex cases as well as prisoner cases and those with new faculty and research residents. We assessed differences between robotic and non-robotic cases in surgery duration and total cost per minute, using one-way ANOVA. RESULTS We included 1608 cases (non-robotic 1145 vs. robotic 463). On average, RC cases with a senior resident took less time than NRC (179.4 < 185.8, p = 0.401); surgery duration of RIH cases was similar with NRIH cases. The total cost per minute of RC cases with a senior resident on average was $9.30 higher than NRC cases for each minute incurred in the operating room but did not lead to a significant change in overall cost. RIH cases, on the other hand, cost less per minute than NRIH cases (114.1 < 126.5, p = 0.399). CONCLUSION Training in robotic surgery is important. Noncomplex RC and RIH involving senior residents were not significantly longer nor did they incur significantly more cost than non-robotic procedures. Senior resident training in noncomplex robotic surgery can be efficient and can be included in the residency curriculum.
Collapse
|
9
|
Stoeckl EM, Garren ME, Nishii A, Evans J, Minter RM, Sandhu G, Jung SA. TrustEd: A Tool for Developing Intraoperative Entrustment Skills. JOURNAL OF SURGICAL EDUCATION 2022; 79:574-578. [PMID: 34972669 DOI: 10.1016/j.jsurg.2021.12.007] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/10/2021] [Revised: 11/09/2021] [Accepted: 12/04/2021] [Indexed: 06/14/2023]
Abstract
OBJECTIVE Toolkits to assess progressive resident autonomy are integral to the movement toward competency-based surgical education. OpTrust is one such tool validated for intraoperative assessment of both faculty and resident entrustment behaviors. We developed a supplementary tool to OpTrust that would aid faculty and residents in making meaningful improvements in entrustment behavior by providing talking points and reflection items tailored to different motivational styles as defined by Regulatory Focus Theory (RFT). DESIGN Existing literature about surgical entrustment was used to build a list of sample dialogue and self-reflection items to use in the operating room. This list was distributed as a survey to individuals familiar with OpTrust and RFT, asking them to categorize each item as Promotion-oriented, Prevention-oriented, or Either. The respondents then met to discuss survey items that did not reach a consensus until the group agreed on their categorization. SETTING University of Wisconsin, School of Medicine and Public Health, Madison, Wisconsin Michigan Medicine, Ann Arbor, Michigan PARTICIPANTS: Clinician and education researchers familiar with intraoperative entrustment and RFT RESULTS: Eight respondents completed the survey categorizing the talking points and reflection items by RFT (100% response rate). Six of these respondents attended the additional meeting to discuss discordant items. The input from this panel was used to develop "TrustEd," the supplementary tool that faculty and residents can quickly reference before beginning a case. CONCLUSION Although tools such as OpTrust allow intraoperative entrustment behaviors to be quantified, TrustEd offers concrete strategies for faculty and residents who are interested in improving those behaviors over time. Further study is needed to assess whether the use of TrustEd does in fact lead to durable behavior change and improvement in OpTrust scores.
Collapse
Affiliation(s)
- Elizabeth M Stoeckl
- Department of Surgery, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Margaret E Garren
- Department of Surgery, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Akira Nishii
- Department of Surgery, Michigan Medicine, Ann Arbor, Michigan
| | - Julie Evans
- Department of Surgery, Michigan Medicine, Ann Arbor, Michigan
| | - Rebecca M Minter
- Department of Surgery, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Gurjit Sandhu
- Department of Surgery, Michigan Medicine, Ann Arbor, Michigan
| | - Sarah A Jung
- Department of Surgery, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin.
| |
Collapse
|
10
|
Orlando MS, Greenberg CC, Pavuluri Quamme SR, Yee A, Faerber AE, King CR. Surgical coaching in obstetrics and gynecology: an evidence-based strategy to elevate surgical education and promote lifelong learning. Am J Obstet Gynecol 2022; 227:51-56. [PMID: 35176285 DOI: 10.1016/j.ajog.2022.02.006] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Revised: 01/22/2022] [Accepted: 02/08/2022] [Indexed: 11/24/2022]
Abstract
The American Board of Medical Specialties, of which the American Board of Obstetrics and Gynecology is a member, released recommendations in 2019 reimagining specialty certification and highlighting the importance of individualized feedback and data-driven advances in clinical practice throughout the physicians' careers. In this article, we presented surgical coaching as an evidence-based strategy for achieving lifelong learning and practice improvement that can help to fulfill the vision of the American Board of Medical Specialties. Surgical coaching involves the development of a partnership between 2 surgeons in which 1 surgeon (the coach) guides the other (the participant) in identifying goals, providing feedback, and facilitating action planning. Previous literature has demonstrated that surgical coaching is viewed as valuable by both coaches and participants. In particular, video-based coaching involves reviewing recorded surgical cases and can be integrated into the physicians' busy schedules as a means of acquiring and advancing both technical and nontechnical skills. Establishing surgical coaching as an option for continuous learning and improvement in practice has the potential to elevate surgical performance and patient care.
Collapse
|
11
|
Woelfel IA, Smith BQ, Salani R, Harzman AE, Cochran AL, Chen X(P. The long game: Evolution of clinical decision making throughout residency and fellowship. Am J Surg 2022; 223:266-272. [PMID: 33752873 PMCID: PMC9045150 DOI: 10.1016/j.amjsurg.2021.03.023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2020] [Revised: 02/10/2021] [Accepted: 03/10/2021] [Indexed: 02/03/2023]
Abstract
BACKGROUND The purpose of this study was to explore the trajectory of autonomy in clinical decision making. METHODS We conducted a qualitative secondary analysis of interviews with 45 residents and fellows from the General Surgery and Obstetrics & Gynecology departments across all clinical postgraduate years (PGY) using convenience sampling. Each interview was recorded, transcribed and iteratively analyzed using a framework method. RESULTS A total of 16 junior residents, 22 senior residents and 7 fellows participated in 12 original interviews. Early in training residents take their abstract ideas about disease processes and make them concrete in their applications to patient care. A transitional stage follows in which residents apply concepts to concrete patient care. Chief residents re-abstract their concrete technical and clinical knowledge to prepare for future surgical practice. CONCLUSIONS Understanding where each learner is on this pathway will assist development of curriculum that fosters resident readiness for practice at each PGY level.
Collapse
Affiliation(s)
- Ingrid A. Woelfel
- Department of Surgery, The Ohio State University, 395 W 12th Ave Suite 670, Columbus, OH, 43201, USA,Corresponding author. Department of Surgery, 395 W 12th Ave Suite 670, Columbus, OH, 43201, USA. (I.A. Woelfel)
| | - Brentley Q. Smith
- Division of Gynecologic Oncology, Department of Obstetrics & Gynecology, The Ohio State University, Starling-Loving Hall, 320 West 10th Ave, Columbus, OH, 43210, USA
| | - Ritu Salani
- Division of Gynecologic Oncology, Department of Obstetrics & Gynecology, The Ohio State University, Starling-Loving Hall, 320 West 10th Ave, Columbus, OH, 43210, USA
| | - Alan E. Harzman
- Department of Surgery, The Ohio State University, 395 W 12th Ave Suite 670, Columbus, OH, 43201, USA
| | - Amalia L. Cochran
- Department of Surgery, The Ohio State University, 395 W 12th Ave Suite 670, Columbus, OH, 43201, USA
| | - Xiaodong (Phoenix) Chen
- Department of Surgery, The Ohio State University, 395 W 12th Ave Suite 670, Columbus, OH, 43201, USA
| |
Collapse
|
12
|
Chen XP, Cochran A, Harzman AE, Ellison EC. A Novel Operative Coaching Program for General Surgery Chief Residents Improves Operative Efficiency. JOURNAL OF SURGICAL EDUCATION 2021; 78:1097-1102. [PMID: 33358340 PMCID: PMC8217072 DOI: 10.1016/j.jsurg.2020.12.006] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/12/2020] [Revised: 11/29/2020] [Accepted: 12/05/2020] [Indexed: 06/01/2023]
Abstract
INTRODUCTION We evaluated the effect of an operative coaching (OC) model on general surgery chief residents' operative efficiency (OE) measured by operative times. We hypothesized that higher levels of entrustment surgeons intend to offer resident in future similar cases are associated with improved OE. MATERIALS AND METHODS From July 2018 to June 2019, we used a validated instrument to score prospective resident entrustment in 228 evaluations of 6 chief residents during 12 OC sessions each (3 lap colectomy, 3 lap cholecystectomy, 3 ventral hernia, 3 inguinal hernia). Operative times of matched case CPT codes performed by coached chiefs (N = 500) were matched via CPT code to the cases of uncoached chiefs in the academic year 2016-2017 (N = 478). Statistical analysis was performed using Pearson correlation and one-way ANOVA. RESULTS Prospective entrustment scores from coached chief residents were associated with significantly shorter operative times in matched complex cases (CC) (r = -0.58, p = 0.0047). A similar trend was observed in noncomplex cases (NCC) (r = -0.29, p = 0.18). Compared to the historical cohort, coached chief residents showed a decrease in mean operative time during complex cases (p = 0.0008, d = 0.44), but an increase in mean operative times for noncomplex cases (p < 0.0001, d = 0.33). CONCLUSIONS An OC model improves chief residents' prospective entrustment leading to increased OE in cases with greater levels of operative complexity, showing a decrease in mean operative time compared to uncoached residents in certain procedures. This is the first report showing formal coaching may be a method to enhance chief resident OE.
Collapse
Affiliation(s)
| | - Amalia Cochran
- Department of Surgery, The Ohio State University, Columbus Ohio
| | - Alan E Harzman
- Department of Surgery, The Ohio State University, Columbus Ohio
| | | |
Collapse
|
13
|
Kinnear B, Warm EJ, Caretta-Weyer H, Holmboe ES, Turner DA, van der Vleuten C, Schumacher DJ. Entrustment Unpacked: Aligning Purposes, Stakes, and Processes to Enhance Learner Assessment. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:S56-S63. [PMID: 34183603 DOI: 10.1097/acm.0000000000004108] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Educators use entrustment, a common framework in competency-based medical education, in multiple ways, including frontline assessment instruments, learner feedback tools, and group decision making within promotions or competence committees. Within these multiple contexts, entrustment decisions can vary in purpose (i.e., intended use), stakes (i.e., perceived risk or consequences), and process (i.e., how entrustment is rendered). Each of these characteristics can be conceptualized as having 2 distinct poles: (1) purpose has formative and summative, (2) stakes has low and high, and (3) process has ad hoc and structured. For each characteristic, entrustment decisions often do not fall squarely at one pole or the other, but rather lie somewhere along a spectrum. While distinct, these continua can, and sometimes should, influence one another, and can be manipulated to optimally integrate entrustment within a program of assessment. In this article, the authors describe each of these continua and depict how key alignments between them can help optimize value when using entrustment in programmatic assessment within competency-based medical education. As they think through these continua, the authors will begin and end with a case study to demonstrate the practical application as it might occur in the clinical learning environment.
Collapse
Affiliation(s)
- Benjamin Kinnear
- B. Kinnear is associate professor of internal medicine and pediatrics, Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0003-0052-4130
| | - Eric J Warm
- E.J. Warm is professor of internal medicine and program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0002-6088-2434
| | - Holly Caretta-Weyer
- H. Caretta-Weyer is assistant professor of emergency medicine, Department of Emergency Medicine, Stanford University School of Medicine, Palo Alto, California; ORCID: https://orcid.org/0000-0002-9783-5797
| | - Eric S Holmboe
- E.S. Holmboe is chief, research, milestones development and evaluation officer, Accreditation Council for Graduate Medical Education, Chicago, Illinois; ORCID: https://orcid.org/0000-0003-0108-6021
| | - David A Turner
- D.A. Turner is vice president, Competency-Based Medical Education, American Board of Pediatrics, Chapel Hill, North Carolina
| | - Cees van der Vleuten
- C. van der Vleuten is professor of education, Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands; ORCID: https://orcid.org/0000-0001-6802-3119
| | - Daniel J Schumacher
- D.J. Schumacher is associate professor of pediatrics, Cincinnati Children's Hospital Medical Center/University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0001-5507-8452
| |
Collapse
|
14
|
Chen XP, Cochran A, Harzman AE, Ellison EC. Predicting prospective resident entrustment: From evaluation to action. Am J Surg 2021; 222:536-540. [PMID: 33485620 DOI: 10.1016/j.amjsurg.2021.01.020] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2020] [Revised: 12/27/2020] [Accepted: 01/11/2021] [Indexed: 11/18/2022]
Abstract
OBJECTIVE We aimed to identify potential variables predictive of a resident achieving faculty future entrustment as a way to enhance attending surgeons' planning of teaching in the operating room leading to improved resident operative autonomy in practice. METHODS We reviewed 273 resident performance evaluations from 91 surgical cases that were collected from 11 general surgery chief residents and 16 attending surgeons between April 2018 and June 2019 using a validated evaluation instrument. The primary outcome measure was prospective resident entrustment estimated by the rater for future similar cases. We used descriptive statistics and the boosted tree analysis model to find potential predictors for the outcome measure and examine test-retest reliability by procedure. RESULTS Step-specific guidance (r = 0.77, p < 0.0001) was the variable most highly associated with prospective resident entrustment in bivariate linear analysis. The boosted tree analysis demonstrated step-specific guidance was the strongest predictor for prospective resident entrustment in the OR, and its predictive importance was much higher than the overall guidance (0.64 > 0.18). Test-retest reliability was from 0.93 to 0.98 across procedures, indicating the likelihood that attending surgeons granted future autonomy complied with their evaluation of prospective resident entrustment was high. CONCLUSIONS By assessing step-specific guidance, attending surgeons can reliably judge residents' future entrustment and potentially better plan for operative teaching/supervision that may lead to granting a surgical resident operative autonomy on similar cases in the future. Our findings provide insight into prospective faculty development of surgical teaching aimed at improving resident readiness for independent practice.
Collapse
|
15
|
Smith BQ, Woelfel I, Salani R, Harzman A, Chen X. Resident Self-Entrustment and Expectations of Autonomy: OB > GYN? JOURNAL OF SURGICAL EDUCATION 2021; 78:275-281. [PMID: 32753260 DOI: 10.1016/j.jsurg.2020.07.019] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/27/2020] [Revised: 06/17/2020] [Accepted: 07/14/2020] [Indexed: 06/11/2023]
Abstract
INTRODUCTION Entrustment is a key component connecting to resident preparedness for surgical practice in the operating room (OR). Residents' self-entrustment of their surgical competencies closely associates with their OR training experience and granted autonomy. Some recent studies have investigated how attending surgeons entrusted residents in the OR. There is little to no data, however, in examining these issues from the resident perspective. The goal of this study was to identify the perception and expectations of autonomy from residents' perspective, as well as the self-entrustment of their surgical competencies in obstetrics (OB) and gynecologic (GYN) procedures. METHODS Focus group interviews of OB/GYN residents were performed. Residents were selected by convenience sampling. Audio recordings of each interview were transcribed, iteratively analyzed, and emergent themes identified, using a framework method. RESULTS A total of 123 minutes of interviews were recorded. Eight junior residents (PGY1-2) and 12 senior residents (PGY3-4) participated. Our data illustrated that (1) the perception of autonomy shifted significantly throughout residency training; (2) residents demonstrated higher expectations and self-entrustment for OB surgical procedures than for GYN surgical procedures upon graduation; and (3) case volume, modalities of OR teaching and mutual communication are 3 factors influencing resident self-entrustment of their surgical competencies. CONCLUSIONS Residents showed disparities in their self-entrustment and expectations of autonomy between OB and GYN surgical procedures. Better understanding these differences and the 3 influencing factors could help programs develop a potential solution for improvement in resident entrustment and autonomy upon graduation.
Collapse
Affiliation(s)
- Brentley Q Smith
- Division of Gynecologic Oncology, The Ohio State University The James Comprehensive Cancer Center, Columbus, Ohio.
| | - Ingrid Woelfel
- Department of General Surgery, The Ohio State University Wexner Medical Center, The Ohio State University Wexner Medical Center, Columbus, Ohio
| | - Ritu Salani
- Division of Gynecologic Oncology, The Ohio State University The James Comprehensive Cancer Center, Columbus, Ohio
| | - Alan Harzman
- Department of General Surgery, The Ohio State University Wexner Medical Center, The Ohio State University Wexner Medical Center, Columbus, Ohio
| | - Xiaodong Chen
- Department of General Surgery, The Ohio State University Wexner Medical Center, The Ohio State University Wexner Medical Center, Columbus, Ohio
| |
Collapse
|
16
|
Sutkin G, Littleton EB, Arnold L, Kanter SL. Micro-relational interdependencies are the essence of teaching and learning in the OR. MEDICAL EDUCATION 2020; 54:1137-1147. [PMID: 32794212 DOI: 10.1111/medu.14353] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/06/2020] [Revised: 07/23/2020] [Accepted: 08/07/2020] [Indexed: 06/11/2023]
Abstract
CONTEXT In the high-stakes, time-critical environment of the operating room (OR), attendings and residents strive to complete safe, effective surgeries and ensure that learning occurs. Yet meaningful resident participation often receives less attention, and that impedes residents' ability to learn and achieve autonomous operative practice. We need a new conceptual framework for understanding progression to autonomous practice that can guide both faculty and residents. Thus, we sought a new conceptualisation of intraoperative teaching and learning (IOT&L) through the lens of Eraut's notion of informal workplace learning and Billett's theory of relational interdependence between social and individual agency. METHODS We viewed authentic examples of IOT&L in video and transcripts of live OR cases and interviews with participating attendings and residents. By systematically applying Eraut and Billet's theories to the transcripts and interviews, we developed concrete descriptions about how IOT&L occurs, categorised them into theory-based principles and derived a conceptualisation and related research ideas about IOT&L. RESULTS Established workplace learning theories frame IOT&L as socially negotiated processes transpiring in distinct interdependent interactions between residents' individual cognitive experiences and their OR social experiences that direct their learning. As the surgery unfolds, spontaneous events and the rules of surgery create opportunities for unplanned and informal learning. These authentic interrelated cognitive and social experiences are stimulated when residents reveal a learning need or attendings recognise a learning gap, and efforts ensue to bridge that gap. Through these minute distinct exchanges, labelled here as 'atomic' IOT&L, residents gain crucial knowledge and skill. CONCLUSION Framing authentic OR interactions between attendings and residents in terms of micro-relational interdependencies shows how granular teaching/learning exchanges yield high-value informal learning. To improve IOT&L, we must examine and change it at this fundamental level by using and testing this new theoretical conceptualisation. These insights produced ideas about IOT&L to test and research.
Collapse
Affiliation(s)
- Gary Sutkin
- University of Missouri-Kansas City School of Medicine, Kansas City, MO, USA
| | | | - Louise Arnold
- University of Missouri-Kansas City School of Medicine, Kansas City, MO, USA
| | | |
Collapse
|
17
|
Chen X, Harzman A, Cochran A, Ellison E. Evaluation of an instrument to assess resident surgical entrustable professional activities (SEPAs). Am J Surg 2020; 220:4-7. [DOI: 10.1016/j.amjsurg.2019.08.026] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2019] [Revised: 08/09/2019] [Accepted: 08/26/2019] [Indexed: 10/26/2022]
|
18
|
Woelfel I, Strosberg D, Smith B, Harzman A, Salani R, Cochran A, Chen X. The Construction of Case-Specific Resident Learning Goals. JOURNAL OF SURGICAL EDUCATION 2020; 77:859-865. [PMID: 32201144 DOI: 10.1016/j.jsurg.2020.02.021] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/25/2019] [Revised: 02/07/2020] [Accepted: 02/21/2020] [Indexed: 06/10/2023]
Abstract
OBJECTIVE Developing resident autonomy in the operating room is a complex process and resident established case specific learning goals may increase resident operating room training efficiency. However, little is understood about residents' experience identifying learning goals for a given case. The aim of this study was to explore the essential components contributing to surgery residents' identification of specific learning goals for surgical cases. DESIGN We conducted focus group interviews with general surgery residents across all post-graduate years (PGY) through convenience sampling. Audio recordings of each interview were transcribed and iteratively analyzed. Emerging themes were identified using a framework method. SETTING The study was conducted within the Department of General Surgery at the Ohio State University Medical Center, a tertiary academic medical center. PARTICIPANTS Eight junior (PGY 1-2) and 10 senior (PGY 3-5) residents participated, of whom 10 were female and 8 were male. RESULTS On average, each focus group interview lasted 57.00 (SD ± 12.99) minutes. Three essential components of residents' creation of case-specific learning goals emerged from the focus group interviews: medical knowledge, surgical experience and entrustment. Residents require baseline knowledge and surgical experience with an operation to identify the learning goal they would aim to execute. They also require entrustment of themselves and support of the attending to accomplish the case specific learning goal. Differences in the possession of these three components would likely influence differences in the ability to create learning goals between junior and senior residents. CONCLUSIONS Medical knowledge, surgical experience and entrustment are 3 factors that are imperative to the creation of a resident's case specific learning goal. The complex combination of these three components contributes to the building of the learning goal prior to the start of the operation. Elucidating these aspects provides additional information for targeted interventions in the future.
Collapse
Affiliation(s)
- Ingrid Woelfel
- Department of Surgery, The Ohio State University, Columbus, Ohio.
| | - David Strosberg
- Department of Surgery, The Ohio State University, Columbus, Ohio
| | - Brentley Smith
- Division of Gynecologic Oncology, Department of Obstetrics & Gynecology, The Ohio State University, Columbus, Ohio
| | - Alan Harzman
- Department of Surgery, The Ohio State University, Columbus, Ohio
| | - Ritu Salani
- Division of Gynecologic Oncology, Department of Obstetrics & Gynecology, The Ohio State University, Columbus, Ohio
| | - Amalia Cochran
- Department of Surgery, The Ohio State University, Columbus, Ohio
| | - Xiaodong Chen
- Department of Surgery, The Ohio State University, Columbus, Ohio
| |
Collapse
|
19
|
Chen XP, Sullivan AM, Smink DS, Alseidi A, Bengtson JM, Kwakye G, Dalrymple JL. Resident Autonomy in the Operating Room: How Faculty Assess Real-time Entrustability. Ann Surg 2019; 269:1080-1086. [PMID: 31082905 DOI: 10.1097/sla.0000000000002717] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
OBJECTIVE This study aimed to identify the empirical processes and evidence that expert surgical teachers use to determine whether to take over certain steps or entrust the resident with autonomy to proceed during an operation. BACKGROUND Assessing real-time entrustability is inherent in attending surgeons' determinations of residents' intraoperative autonomy in the operating room. To promote residents' autonomy, it is necessary to understand how attending surgeons evaluate residents' performance and support opportunities for independent practice based on the assessment of their entrustability. METHODS We conducted qualitative semi-structured interviews with 43 expert surgical teachers from 21 institutions across 4 regions of the United States, using purposeful and snowball sampling. Participants represented a range of program types, program size, and clinical expertise. We applied the Framework Method of content analysis to iteratively analyze interview transcripts and identify emergent themes. RESULTS We identified a 3-phase process used by most expert surgical teachers in determining whether to take over intraoperatively or entrust the resident to proceed, including 1) monitoring performance and "red flags," 2) assessing entrustability, and 3) granting autonomy. Factors associated with individual surgeons (eg, level of comfort, experience, leadership role) and the context (eg, patient safety, case, and time) influenced expert surgical teachers' determinations of entrustability and residents' final autonomy. CONCLUSION Expert surgical teachers' 3-phase process of decisions on take-over provides a potential framework that may help surgeons identify appropriate opportunities to develop residents' progressive autonomy by engaging the resident in the determination of entrustability before deciding to take over.
Collapse
Affiliation(s)
- Xiaodong Phoenix Chen
- Brigham and Women's Hospital, Boston, MA
- Harvard Medical School, Boston, MA
- The Ohio State University, Columbus, OH
| | - Amy M Sullivan
- Beth Israel Deaconess Medical Center, Boston, MA
- Harvard Medical School, Boston, MA
| | - Douglas S Smink
- Brigham and Women's Hospital, Boston, MA
- Harvard Medical School, Boston, MA
| | | | - Joan M Bengtson
- Brigham and Women's Hospital, Boston, MA
- Harvard Medical School, Boston, MA
| | | | - John L Dalrymple
- Beth Israel Deaconess Medical Center, Boston, MA
- Harvard Medical School, Boston, MA
| |
Collapse
|
20
|
Chen X(P, Cochran A, Dalrymple JL. Framework for Faculty Development in Resident Autonomy and Entrustment in the Operating Room. JAMA Surg 2019; 154:5-6. [DOI: 10.1001/jamasurg.2018.3529] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Affiliation(s)
| | - Amalia Cochran
- The Ohio State University, Columbus
- Web and Social Media Editor, JAMA Surgery
| | - John L. Dalrymple
- Beth Israel Deaconess Medical Center, Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
| |
Collapse
|