1
|
Kim E, Rosenthal LS, Ryder CY, Anidi C, Bidwell SS, Rooney DM, Yu J, Forczmanski P, Jeffcoach DR, Kim GJ. Generalizability of Artificial Intelligence Assessments in Laparoscopic Surgery Simulation. J Surg Res 2025; 309:249-256. [PMID: 40279911 DOI: 10.1016/j.jss.2025.03.030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2024] [Revised: 03/19/2025] [Accepted: 03/22/2025] [Indexed: 04/29/2025]
Abstract
INTRODUCTION The application of artificial intelligence (AI) in the assessment of procedural skills on a simulation platform using the global rating scale (GRS) has shown promise. Our team developed an open-source, low-cost simulation platform for the development of laparoscopic skills in low-resource settings, with skill assessment provided by video-based peer review and AI. The generalizability of AI trained on one procedure to evaluate general procedural skills within a single training system is unknown. This study examines the feasibility of generalizing AI-based assessments across procedures in a training system. METHODS AI was trained, with varied combinations of procedures, to score 111 laparoscopic performance videos of four procedures (57 salpingostomies, 20 appendectomies, 15 enterectomies, and 19 diaphragmatic repairs), using time and distance-based calculations. Predicted scores were generated using five-fold cross-validation and K-nearest neighbors, with both 5-class (scored 1-5) and 2-class (pass/fail) scoring systems. Videos were also scored in a conventional fashion using human video-based review, based on GRS competencies. RESULTS AI assessments achieved 42%-100% concordance with human reviews in the 5-class system and 68%-100% in the 2-class system, P = 0.005. Within the 5-class system, 100% accuracy was reached when AI trained on multiple procedures evaluated appendectomy. The 2-class system attained 100% accuracy in three procedures across the GRS competencies. CONCLUSIONS AI assessment trained on procedures using video-based review evaluated laparoscopic skills across different procedures within a simulation-based training system. Dichotomizing scoring to pass/fail improved accuracy, while supporting the potential to assess procedural competence.
Collapse
Affiliation(s)
- Erin Kim
- University of Michigan, Ann Arbor, Michigan
| | | | | | | | | | - Deborah M Rooney
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, Michigan
| | - Joon Yu
- Department of Surgery, University of Michigan, Ann Arbor, Michigan
| | - Pawel Forczmanski
- Department of Computer Science and Information Technology, West Pomeranian University of Technology in Szczecin, Szczecin, Poland
| | - David R Jeffcoach
- Department of Surgery, University of California San Francisco Fresno, Fresno, California
| | - Grace J Kim
- Department of Surgery, University of Michigan, Ann Arbor, Michigan.
| |
Collapse
|
2
|
Perri G, Di Martino M, Minter R, Srinivasa S, Newhook T, Roch A, Guest R, Fretland A, Prodehl L, Thepbunchonchai A, Hallet J. Challenges in early career for HPB surgeons: an international practice survey. HPB (Oxford) 2025:S1365-182X(25)00068-1. [PMID: 40024851 DOI: 10.1016/j.hpb.2025.02.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/19/2024] [Accepted: 02/15/2025] [Indexed: 03/04/2025]
Abstract
BACKGROUND Understanding the needs of early-career surgeons is essential for developing strategies for support. This study examined perceived needs of early-career HPB surgeons worldwide. METHODS A self-administered web-based survey of early career HPB surgeons (aged ≤45 years old or practicing for ≤5 years) was conducted. A questionnaire was developed through items generation and reduction, followed by pilot testing. Quantitative data were reported with descriptive statistics and qualitative responses analysed using open coding. RESULTS There were 282 respondents. Six main areas of challenges were identified: mentorship opportunities, continued skills acquisition and training, research and collaboration, leadership and career development, clinical knowledge exchange, and accessibility including financial barriers. Specifically, highest rated challenges were ebalancing work and personal life (mean 6.98, SD 2.92), research (mean 6.79, SD 2.50), career development/mentorship (mean 6.70, SD 2.30), networking/collaboration (mean 6.16, SD 2.71), leadership and (mean 6.09, SD 2.53). Rating of research as a challenge was higher in Africa and Central/South America (p = 0.01), and that of leadership was higher in Asia, Oceania, and the Middle East (p = 0.02). CONCLUSIONS This survey highlights the multiple challenges faced by early-career HPB surgeons worldwide. These results provide insights into how to better support early-career HPB surgeons to fully develop their specialty.
Collapse
Affiliation(s)
- Giampaolo Perri
- Department of Surgery, Oncology and Gastroenterology (DiSCOG), University of Padua, Padua, Italy
| | - Marcello Di Martino
- Department of Health Sciences, University of Piemonte Orientale, Novara, Italy
| | - Rebecca Minter
- University of Wisconsin School of Medicine and Public Health, Department of Surgery, Madison, WI, USA
| | - Sanket Srinivasa
- Department of Surgery, North Shore Hospital, Auckland, New Zealand
| | - Timothy Newhook
- Department of Surgery, The University of Texas MD Anderson Cancer Centre, Houston, TX, USA
| | - Alexandra Roch
- Division of Surgical Oncology, Department of Surgery, Indiana University School of Medicine, Indianapolis, IN, USA
| | - Rachel Guest
- Department of Clinical Surgery, University of Edinburgh, Edinburgh, UK
| | - Asmund Fretland
- Oslo University Hospital, Department of Hepato-Pancreato-Biliary Surgery, Oslo, Norway
| | - Leanne Prodehl
- Department of Surgery, Faculty of Health Sciences University of the Witwatersrand, Johannesburg, South Africa
| | - Asara Thepbunchonchai
- Department of Surgery, Faculty of Medicine, Chiang Mai University, Chiang Mai, Thailand
| | - Julie Hallet
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Surgical Oncology, Odette Cancer Centre - Sunnybrook Health Sciences Centre, Toronto, ON, Canada.
| |
Collapse
|
3
|
Bhaskar SM. Medicine Meets Science: The Imperative of Scientific Research and Publishing for Physician-Scientists. Indian J Radiol Imaging 2025; 35:S9-S17. [PMID: 39802717 PMCID: PMC11717469 DOI: 10.1055/s-0044-1800803] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/16/2025] Open
Abstract
Physician-scientists serve as conduits between clinical practice and scientific research, leveraging their unique expertise to improve patient care and drive medical innovation. This article highlights the indispensable role of research and publishing in promoting evidence-based practices, facilitating professional growth, and shaping public health policy. Drawing on historical and contemporary examples, I examine the challenges faced by physician-scientists, such as ethical dilemmas and declining engagement in research, particularly in resource-constrained settings. I suggest pragmatic strategies to overcome these barriers, emphasizing the need for systemic support, ethical integrity, and the equitable dissemination of advancements. This piece aims to inspire a new generation of physician-scientists to engage deeply with both clinical and research domains, thus advancing global health equity and resilience.
Collapse
Affiliation(s)
- Sonu M.M. Bhaskar
- Department of Neurology, Division of Cerebrovascular Medicine and Neurology, National Cerebral and Cardiovascular Center (NCVC), Suita, Osaka, Japan
- Global Health Neurology Lab, Sydney, New South Wales, Australia
- Ingham Institute for Applied Medical Research, Clinical Sciences Stream, Liverpool, New South Wales, Australia
- NSW Brain Clot Bank, NSW Health Pathology, Sydney, New South Wales, Australia
- Department of Neurology & Neurophysiology, Liverpool Hospital, South Western Sydney Local Health District, Liverpool, New South Wales, Australia
| |
Collapse
|
4
|
Nadeem A. Surgical Training Pathways in the United Kingdom and the United States: Lessons for Resource-Limited Settings From High-Income Countries. Cureus 2024; 16:e73285. [PMID: 39524163 PMCID: PMC11546745 DOI: 10.7759/cureus.73285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/08/2024] [Indexed: 11/16/2024] Open
Abstract
Background Surgical training is a critical component of healthcare, especially in high-income countries such as the United Kingdom (UK) and the United States (US), which have established distinct, well-organised training frameworks. Comparing these systems provides valuable insights that may enhance global surgical education, particularly in low- and middle-income countries, where training and retaining proficient surgeons are considerable challenges. Methodology This comparative study examines the surgical training systems in the UK and the US, focusing on key aspects, including training structure, competency-based assessments, and work-hour regulations. Data were sourced from regulatory organisations such as the General Medical Council, the Royal College of Surgeons, the Accreditation Council for Graduate Medical Education, and the American Board of Surgery. The analysis explores how elements of these models might be adapted to support sustainable surgical education frameworks in resource-limited environments. Ethical approval was not required due to the use of publicly accessible data and no patient involvement. Results The UK and US surgical training systems differ substantially in their structure, training duration, and specialisation timing. The UK employs a tiered approach, offering generalist experience before specialisation, while the US favours early specialisation directly after medical school. Both systems implement competency-based evaluations, though the US system places a greater emphasis on case volume and procedural exposure. Work-hour regulations also vary, with the UK capping weekly hours at 48 under the European Working Time Directive, compared to an 80-hour maximum in the US, which results in differing levels of trainee satisfaction and burnout rates. Conclusions The competency-based assessments in both the UK and the US offer adaptable frameworks for resource-limited settings. The phased training approach in the UK is well-suited for environments requiring versatile surgeons capable of handling a wide range of cases. By implementing these adaptable elements, along with cost-effective training innovations such as simulation tools, e-learning platforms, and international partnerships, resource-constrained regions can foster a sustainable, skilled surgical workforce. These insights offer pathways to improve healthcare outcomes and equity globally by enhancing surgical capacity in regions with limited resources.
Collapse
Affiliation(s)
- Adeel Nadeem
- Orthopaedics, Royal National Orthopaedic Hospital, London, GBR
| |
Collapse
|