1
|
Woo HY, Cho A, Ko M, Shin J, Min SK, Min S, Han A, Ha J, Ahn S. The impact of simulator training of vascular anastomosis and video assessment for surgical residents. Ann Vasc Surg 2024:S0890-5096(24)00482-5. [PMID: 39122209 DOI: 10.1016/j.avsg.2024.03.034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2023] [Revised: 03/28/2024] [Accepted: 03/31/2024] [Indexed: 08/12/2024]
Abstract
INTRODUCTION The rapid increase of minimally invasive surgery and the shortened training period for surgical residents has resulted in limited opportunities to acquire proficiency in open surgical techniques, such as vascular anastomosis. However, vascular anastomosis remains an essential skill in every surgery for bleeding control. This study aimed to validate the effectiveness of surgical education model for vascular anastomosis and assess the impact on the comprehension, skill, and confidence of surgical residents in performing vascular anastomosis. METHODS A total of 21 surgical residents with 1st to 3rd years of experience at Seoul National University Hospital participated in a four-week vascular anastomosis training program. The program included an educational lecture and the performance of an end-to-side anastomosis on a procedural model, with evaluations being conducted using the Objective Structured Assessment of Technical Skills (OSATS) and the End Product Rating Score (EPRS) in pre- and post-training surveys. RESULTS Significant improvement was observed in the OSATS score (from 9.22 ± 2.4 in week 1 to 12.87 ± 3.1 in week 4; P < 0.001) and the EPRS score (from 12.47 ± 4.1 in week 1 to 17.57 ± 2.2 in week 4; P < 0.001). Additionally, the surgical performance time significantly decreased from 20.99 ± 4.6 minutes to 16.33 ± 4.2 minutes (P = 0.019) CONCLUSIONS: Simulator training of in vitro vascular anastomosis, when accompanied by expert-led instruction, can effectively enhance the surgical proficiency, confidence, and overall surgical outcomes of residents, as inferred from the observed improvements in OSATS and EPRS scores. The results suggest that integration of this training model into surgical curricula could be a promising strategy for enhancing vascular surgical training.
Collapse
Affiliation(s)
- Hye Young Woo
- Department of Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Ara Cho
- Department of Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Myeonghyeon Ko
- Department of Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Jiyoung Shin
- Department of Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Seung-Kee Min
- Department of Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Sangil Min
- Department of Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Ahram Han
- Department of Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Jongwon Ha
- Department of Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Sanghyun Ahn
- Department of Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
2
|
Cold KM, Vamadevan A, Vilmann AS, Svendsen MBS, Konge L, Bjerrum F. Computer-aided quality assessment of endoscopist competence during colonoscopy: a systematic review. Gastrointest Endosc 2024; 100:167-176.e1. [PMID: 38580134 DOI: 10.1016/j.gie.2024.04.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/16/2024] [Revised: 03/28/2024] [Accepted: 03/28/2024] [Indexed: 04/07/2024]
Abstract
BACKGROUND AND AIMS Endoscopists' competence can vary widely, as shown in the variation in the adenoma detection rate (ADR). Computer-aided quality assessment (CAQ) can automatically assess performance during individual procedures. In this review we identified and described different CAQ systems for colonoscopy. METHODS A systematic review of the literature was done using MEDLINE, EMBASE, and Scopus based on 3 blocks of terms according to the inclusion criteria: colonoscopy, competence assessment, and automatic evaluation. Articles were systematically reviewed by 2 reviewers, first by abstract and then in full text. The methodological quality was assessed using the Medical Education Research Study Quality Instrument (MERSQI). RESULTS Of 12,575 identified studies, 6831 remained after removal of duplicates and 6806 did not pass the eligibility criteria and were excluded, leaving 25 studies, of which 13 studies were included in the final analysis. Five categories of CAQ systems were identified: withdrawal speedometer (7 studies), endoscope movement analysis (3 studies), effective withdrawal time (1 study), fold examination quality (1 study), and visual gaze pattern (1 study). The withdrawal speedometer was the only CAQ system that tested its feedback by examining changes in ADR. Three studies observed an improvement in ADR, and 2 studies did not. The methodological quality of the studies was high (mean MERSQI, 15.2 points; maximum, 18 points). CONCLUSIONS Thirteen studies developed or tested CAQ systems, most frequently by correlating it to the ADR. Only 5 studies tested feedback by implementing the CAQ system. A meta-analysis was impossible because of the heterogeneous study designs, and more studies are warranted.
Collapse
Affiliation(s)
- Kristoffer Mazanti Cold
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR & Education, the Capital Region of Denmark, Copenhagen, Denmark; Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Anishan Vamadevan
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR & Education, the Capital Region of Denmark, Copenhagen, Denmark
| | - Andreas Slot Vilmann
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR & Education, the Capital Region of Denmark, Copenhagen, Denmark; Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark; Gastrounit, Surgical Section, Copenhagen University Hospital-Herlev and Gentofte, Herlev, Denmark
| | - Morten Bo Søndergaard Svendsen
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR & Education, the Capital Region of Denmark, Copenhagen, Denmark; Department of Computer Science, Faculty of Science, University of Copenhagen, Copenhagen, Denmark
| | - Lars Konge
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR & Education, the Capital Region of Denmark, Copenhagen, Denmark; Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Flemming Bjerrum
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR & Education, the Capital Region of Denmark, Copenhagen, Denmark; Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark; Gastrounit, Surgical Section, Copenhagen University Hospital-Amager and Hvidovre, Hvidovre, Denmark
| |
Collapse
|
3
|
Feng L, Li W, Lai J, Yang W, Wu S, Liu J, Ma R, Lee S, Tian J. Validity of A Novel Simulator for Percutaneous Transforaminal Endoscopic Discectomy. World Neurosurg 2024; 187:e220-e232. [PMID: 38641247 DOI: 10.1016/j.wneu.2024.04.070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2024] [Revised: 04/11/2024] [Accepted: 04/12/2024] [Indexed: 04/21/2024]
Abstract
BACKGROUND Percutaneous transforaminal endoscopic discectomy (PTED) has steep learning curves and a high incidence of complications, but currently, efficient and economical training methods are lacking. This study aimed to validate a novel simulator for PTED. METHODS The simulated PTED included puncturing and establishing the working channel (PEWC) and endoscopic discectomy, with the PEWC being the tested module. Eleven experts and 21 novices were included and introduced to the simulator and tasks; all participants completed the PEWC. Outcomes included: total operation time, number of fluoroscopy for positioning the working sheath, number of spinal risk region invasion, Global Rating Scale (GRS) and a modified GRS, etc. The Mann-Whitney U test was used to compare 2 groups. Spearman's correlation coefficient analyzed continuous variables. RESULTS Experts outperformed novices in total operation time (P = 0.001), requiring fewer number of fluoroscopies for positioning the working sheath (P = 0.003). Additionally, experts had a lower number of spinal risk region invasions (P = 0.016) and higher scores on both the GRS (P < 0.001) and modified GRS (P < 0.001). PTED experience correlated with GRS scores (P = 0.001) and modified GRS (P < 0.001). The overall realism scored a median of 4 (3.75-5), and educational value had a median of 4 (range 3-5). CONCLUSIONS This study demonstrates the validity of the novel simulator, revealing significant associations between PTED experience and performance metrics in a simulated PEWC setting. Furthermore, the PEWC module also offers a good realistic design and high education value according to experts.
Collapse
Affiliation(s)
- Lei Feng
- Department of Orthopedics, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Wei Li
- Department of Orthopedics, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Jianming Lai
- The Second School of Clinical Medicine, Southern Medical University, Guangzhou, China
| | - Weihao Yang
- Department of Orthopedics, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Shangxing Wu
- Department of Orthopedics, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Jiajia Liu
- Department of Orthopedics, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Ruixin Ma
- The Second School of Clinical Medicine, Southern Medical University, Guangzhou, China
| | - Soomin Lee
- Clinical Skills Training Center, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Jing Tian
- Clinical Skills Training Center, Zhujiang Hospital, Southern Medical University, Guangzhou, China.
| |
Collapse
|
4
|
Lanser C, Fisher DM, Kasrai L, Fisher K, Podolsky DJ. Development and Preliminary Evaluation of A Soft Tissue Microtia Simulator. J Craniofac Surg 2024:00001665-990000000-01714. [PMID: 38869279 DOI: 10.1097/scs.0000000000010322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2024] [Accepted: 04/19/2024] [Indexed: 06/14/2024] Open
Abstract
Surgical simulation has been used extensively for learning microtia reconstruction and has almost exclusively involved framework creation. However, soft tissue reconstruction in microtia is equally challenging and would benefit from a simulation platform. This study aimed to describe the development and preliminary evaluation of a high-fidelity soft tissue microtia simulator. Three-dimensional modeling software, fused deposition 3-dimensional printing, adhesive techniques, silicones, and polyurethane rubbers were utilized to create a right lobular-type microtia simulator that comprises skin, subcutaneous tissue, and cartilage. Two expert microtia surgeons performed a microtia reconstruction on the simulator and evaluated its value and realism using a Likert-type questionnaire. The surgeons utilized a previously developed synthetic framework and successfully performed the critical steps of the soft tissue reconstruction, including marking, incising, dissection, removal of the cartilage remnant, drain insertion, insertion of the framework, closing of the skin, and demonstration of the soft tissue conforming over the framework using suction. A preliminary assessment of the simulator demonstrated that the simulator is anatomically accurate, realistic, and highly valuable as a training tool. A high-fidelity soft tissue microtia simulator was successfully developed and tested. The simulator provides a valuable training platform for learning a critical component of microtia reconstruction.
Collapse
Affiliation(s)
| | - David M Fisher
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children
| | - Leila Kasrai
- Division of Plastic Surgery, St Josephs Health Center
| | - Keon Fisher
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children
| | - Dale J Podolsky
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children
- Posluns Center for Image Guided Innovation and Therapeutic Intervention (PCIGITI), Toronto, Ontario, Canada
| |
Collapse
|
5
|
Boberg-Ans LC, Hutter DE, La Cour M, Konge L, Le A, Vangsted A, Thomsen ASS. Comparing the impact of surgical expert versus non-ophthalmologist instructors on virtual-reality surgical performance: A randomized controlled trial. Acta Ophthalmol 2024. [PMID: 38819012 DOI: 10.1111/aos.16719] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Accepted: 05/08/2024] [Indexed: 06/01/2024]
Abstract
PURPOSE To compare Manual Small Incision Cataract Surgery (MSICS) microsurgical performance in course participants who received virtual reality simulation-based training by either a surgical expert or a non-ophthalmologist instructor. SETTING Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark. DESIGN Randomized controlled trial. METHODS Residents and specialists in ophthalmology with no prior MSICS experience were included to receive virtual reality simulation training in MSICS using the HelpMeSee simulator. The participants were randomly allocated to receive training from either an experienced MSICS surgeon or a non-ophthalmologist, also known as near-peer teaching. The performances of the participants were evaluated at baseline and post-training using a MSICS proficiency-based test with evidence of validity. RESULTS Thirty participants were included in the study and 29 completed the course. There was no significant difference in final test score between the two groups (p = 0.13). The performance score of both groups of participants increased significantly after receiving the training (p < 0.001). All participants passed the proficiency-based test after receiving the training. CONCLUSION We found no significant difference in surgical proficiency-level whether the participants were trained by a surgical expert or a non-ophthalmologist instructor for MSICS in a virtual-reality based setting. The findings of this study suggest that near-peer teaching within microsurgical performance potentially could be applied with teaching outcomes comparable to a surgical expert-instructor.
Collapse
Affiliation(s)
- Lars Christian Boberg-Ans
- Department of Ophthalmology, Rigshospitalet-Glostrup, Glostrup, Denmark
- Copenhagen Academy for Medical Education and Simulation, Denmark, Denmark
| | - Daniel Ethan Hutter
- Copenhagen Academy for Medical Education and Simulation, Denmark, Denmark
- Bascom Palmer Eye Institute, Voluntary Faculty, Miami, Florida, USA
| | - Morten La Cour
- Department of Ophthalmology, Rigshospitalet-Glostrup, Glostrup, Denmark
- Department of Clinical Medicine, University of Copenhagen, Glostrup, Denmark
| | - Lars Konge
- Copenhagen Academy for Medical Education and Simulation, Denmark, Denmark
| | - Anton Le
- Department of Ophthalmology, Rigshospitalet-Glostrup, Glostrup, Denmark
- Copenhagen Academy for Medical Education and Simulation, Denmark, Denmark
| | - Andreas Vangsted
- Department of Ophthalmology, Rigshospitalet-Glostrup, Glostrup, Denmark
| | - Ann Sofia Skou Thomsen
- Department of Ophthalmology, Rigshospitalet-Glostrup, Glostrup, Denmark
- Copenhagen Academy for Medical Education and Simulation, Denmark, Denmark
- Department of Clinical Medicine, University of Copenhagen, Glostrup, Denmark
| |
Collapse
|
6
|
Toale C, Morris M, Konge L, Nayahangan LJ, Roche A, Heskin L, Kavanagh DO. Generating a Prioritized List of Operative Procedures for Simulation-based Assessment of General Surgery Trainees Through Consensus. Ann Surg 2024; 279:900-905. [PMID: 37811854 DOI: 10.1097/sla.0000000000006118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/10/2023]
Abstract
OBJECTIVE To develop appropriate content for high-stakes simulation-based assessments of operative competence in general surgery training through consensus. BACKGROUND Valid methods of summative operative competence assessment are required by competency-based training programs in surgery. METHOD An online Delphi consensus study was conducted. Procedures were derived from the competency expectations outlined by the Joint Committee on Surgical Training Curriculum 2021, and subsequent brainstorming. Procedures were rated according to their perceived importance, perceived procedural risk, how frequently they are performed, and simualtion feasibility by a purposive sample of 30 surgical trainers and a 5-person steering group. A modified Copenhagen Academy for Medical Education and Simulation Needs Assessment Formula was applied to the generated data to produce ranked procedural lists, which were returned to participants for re-prioritization. RESULTS Prioritized lists were generated for simulation-based operative competence assessments at 2 key stages of training; the end of 'phase 2' prior to the development of a sub-specialty interest, and the end of 'phase 3', that is, end-of-training certification. A total of 21 and 16 procedures were deemed suitable for assessments at each of these stages, respectively. CONCLUSIONS This study describes a national needs assessment approach to content generation for simulation-based assessments of operative competence in general surgery using Delphi consensus methodology. The prioritized procedural lists generated by this study can be used to further develop operative skill assessments for use in high-stakes scenarios, such as trainee progression, entrustment, and end-of-training certification, before subsequent validity testing.
Collapse
Affiliation(s)
- Conor Toale
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Marie Morris
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Lars Konge
- Copenhagen Academy for Medical Education and Simulation, University of Copenhagen, Copenhagen, Denmark
| | - Leizl Joy Nayahangan
- Copenhagen Academy for Medical Education and Simulation, University of Copenhagen, Copenhagen, Denmark
| | - Adam Roche
- Department of Simulation, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Leonie Heskin
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, Dublin, Ireland
- Department of Simulation, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Dara O Kavanagh
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, Dublin, Ireland
| |
Collapse
|
7
|
Hertz P, Rattenborg S, Haug TR, Houlind K, Konge L, Bjerrum F. Training and assessment for colorectal surgery and appendicectomy- a systematic review. Colorectal Dis 2024; 26:597-608. [PMID: 38396135 DOI: 10.1111/codi.16905] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Revised: 11/07/2023] [Accepted: 12/28/2023] [Indexed: 02/25/2024]
Abstract
AIM There is currently an increased focus on competency-based training, in which training and assessment play a crucial role. The aim of this systematic review is to create an overview of hands-on training methods and assessment tools for appendicectomy and colon and rectal surgery procedures using either an open, laparoscopic or robot-assisted approach. METHOD A systematic review of Medline, Embase, Cochrane and Scopus databases was conducted following the PRISMA guidelines. We conducted the last search on 9 March 2023. All published papers describing hands-on training, evaluation of performance data and development of assessment tools were eligible. The quality of studies and the validity evidence of assessment tools are reported. RESULTS Fifty-one studies were identified. Laparoscopic assessment tools are abundant, but the literature still lacks good-quality assessment tools for open appendicectomy, robotic colectomy and open rectal surgery. Overall, there is a lack of discussion regarding the establishment of pass/fail standards and the consequences of assessment. Virtual reality simulation is used more for appendicectomy than colorectal procedures. Only a few of the studies investigating training were of acceptable quality. There is a need for high-quality studies in open and robotic-assisted colon surgery and all approaches to rectal surgery. CONCLUSION This review provides an overview of current training methods and assessment tools and identifies where more research is needed based on the quality of the studies and the current validity evidence.
Collapse
Affiliation(s)
- Peter Hertz
- Department of Surgery, Hospital Lillebaelt, University of Southern Denmark, Kolding, Denmark
- Department of Regional Health Research, University of Southern Denmark, Kolding, Denmark
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR and Education, Copenhagen, Denmark
| | - Søren Rattenborg
- Department of Regional Health Research, University of Southern Denmark, Kolding, Denmark
- Department of Surgery, Hospital Lillebaelt Vejle, Colorectal Cancer Center South, University of Southern Denmark DK, Kolding, Denmark
| | - Tora R Haug
- Department of Surgery, Gødstrup Hospital, Herning, Denmark
- Aarhus University, Aarhus, Denmark
| | - Kim Houlind
- Department of Regional Health Research, University of Southern Denmark, Kolding, Denmark
- Department of Vascular Surgery, Hospital Lillebaelt, University of Southern Denmark, Kolding, Denmark
| | - Lars Konge
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR and Education, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | - Flemming Bjerrum
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR and Education, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
- Gastrounit, Surgical Section, Copenhagen University Hospital - Amager and Hvidovre, Hvidovre, Denmark
| |
Collapse
|
8
|
Eley CL, Palaniappan V, Carter A, Sogaolu O, Horwood J, Davies M, Torkington J, Ansell J. Randomized controlled trial of the CMR immersive virtual reality (IVR) headset training compared to e-learning for operating room configuration of the CMR versius robot. J Robot Surg 2024; 18:143. [PMID: 38554218 PMCID: PMC10981610 DOI: 10.1007/s11701-024-01885-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 02/26/2024] [Indexed: 04/01/2024]
Abstract
Robotic surgery offers potential advantages over laparoscopic procedures, but the training for configuring robotic systems in the operating room remains underexplored. This study seeks to validate immersive virtual reality (IVR) headset training for setting up the CMR Versius in the operating room. This single-blinded randomized control trial randomised medical students with no prior robotic experience using an online randomiser. The intervention group received IVR headset training, and the control group, e-learning modules. Assessors were blinded to participant group. Primary endpoint was overall score (OS): Likert-scale 1-5: 1 reflecting independent performance, with increasing verbal prompts to a maximum score of 5, requiring physical assistance to complete the task. Secondary endpoints included task scores, time, inter-rater reliability, and concordance with participant confidence scores. Statistical analysis was performed using IBM SPSS Version 27. Of 23 participants analysed, 11 received IVR and 12 received e-learning. The median OS was lower in the IVR group than the e-learning group 53.5 vs 84.5 (p < 0.001). VR recipients performed tasks independently more frequently and required less physical assistance than e-learning participants (p < 0.001). There was no significant difference in time to completion (p = 0.880). Self-assessed confidence scores and assessor scores differed for e-learning participants (p = 0.008), though not IVR participants (p = 0.607). IVR learning is more effective than e-learning for preparing robot-naïve individuals in operating room set-up of the CMR Versius. It offers a feasible, realistic, and accessible option in resource-limited settings and changing dynamics of operating theatre teams. Ongoing deliberate practice, however, is still necessary for achieving optimal performance. ISCRTN Number 10064213.
Collapse
Affiliation(s)
- Catherine L Eley
- University Hospital of Wales, Heath Park, Cardiff, CF144XY, United Kingdom.
- School of Medicine, Cardiff University, Heath Park, Cardiff, CF14 4XN, United Kingdom.
| | - Varun Palaniappan
- Princess of Wales Hopsital, Coity Road, Bridgend, CF31 1RQ, United Kingdom
| | - Abbie Carter
- School of Medicine, Cardiff University, Heath Park, Cardiff, CF14 4XN, United Kingdom
| | - Opeyemi Sogaolu
- University Hospital of Wales, Heath Park, Cardiff, CF144XY, United Kingdom
| | - James Horwood
- University Hospital of Wales, Heath Park, Cardiff, CF144XY, United Kingdom
| | - Michael Davies
- University Hospital of Wales, Heath Park, Cardiff, CF144XY, United Kingdom
| | - Jared Torkington
- University Hospital of Wales, Heath Park, Cardiff, CF144XY, United Kingdom
| | - James Ansell
- University Hospital of Wales, Heath Park, Cardiff, CF144XY, United Kingdom
| |
Collapse
|
9
|
Hertz P, Bertelsen CA, Houlind K, Bundgaard L, Konge L, Bjerrum F, Svendsen MBS. Developing a phantom for simulating robotic-assisted complete mesocolic excision using 3D printing and medical imaging. BMC Surg 2024; 24:72. [PMID: 38408998 PMCID: PMC10897992 DOI: 10.1186/s12893-024-02353-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Accepted: 02/07/2024] [Indexed: 02/28/2024] Open
Abstract
BACKGROUND Robotic-assisted complete mesocolic excision is an advanced procedure mainly because of the great variability in anatomy. Phantoms can be used for simulation-based training and assessment of competency when learning new surgical procedures. However, no phantoms for robotic complete mesocolic excision have previously been described. This study aimed to develop an anatomically true-to-life phantom, which can be used for training with a robotic system situated in the clinical setting and can be used for the assessment of surgical competency. METHODS Established pathology and surgical assessment tools for complete mesocolic excision and specimens were used for the phantom development. Each assessment item was translated into an engineering development task and evaluated for relevance. Anatomical realism was obtained by extracting relevant organs from preoperative patient scans and 3D printing casting moulds for each organ. Each element of the phantom was evaluated by two experienced complete mesocolic excision surgeons without influencing each other's answers and their feedback was used in an iterative process of prototype development and testing. RESULTS It was possible to integrate 35 out of 48 procedure-specific items from the surgical assessment tool and all elements from the pathological evaluation tool. By adding fluorophores to the mesocolic tissue, we developed an easy way to assess the integrity of the mesocolon using ultraviolet light. The phantom was built using silicone, is easy to store, and can be used in robotic systems designated for patient procedures as it does not contain animal-derived parts. CONCLUSIONS The newly developed phantom could be used for training and competency assessment for robotic-assisted complete mesocolic excision surgery in a simulated setting.
Collapse
Affiliation(s)
- Peter Hertz
- Department of Surgery, Hospital Lillebaelt, University of Southern Denmark, Sygehusvej 24, Kolding, 6000, Denmark.
- Department of Regional Health Research, University of Southern Denmark, Odense, Denmark.
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR and Education, The Capital Region of Denmark, Copenhagen, Denmark.
| | - Claus Anders Bertelsen
- Department of Surgery, Copenhagen University Hospital - North Zealand, Hillerød, Denmark
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Kim Houlind
- Department of Regional Health Research, University of Southern Denmark, Odense, Denmark
- Department of Vascular Surgery, Hospital Lillebaelt, University of Southern Denmark, Kolding, Denmark
| | - Lars Bundgaard
- Department of Surgery, Hospital Lillebaelt Vejle, Colorectal Cancer Center South, University of Southern Denmark, Odense, Denmark
| | - Lars Konge
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR and Education, The Capital Region of Denmark, Copenhagen, Denmark
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Flemming Bjerrum
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR and Education, The Capital Region of Denmark, Copenhagen, Denmark
- Gastrounit, Surgical section, Copenhagen University Hospital - Amager and Hvidovre, Hvidovre, Denmark
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Morten Bo Søndergaard Svendsen
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR and Education, The Capital Region of Denmark, Copenhagen, Denmark
- Department of Computer Science, Faculty of Science, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
10
|
Goldenberg MG. Surgical Artificial Intelligence in Urology: Educational Applications. Urol Clin North Am 2024; 51:105-115. [PMID: 37945096 DOI: 10.1016/j.ucl.2023.06.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2023]
Abstract
Surgical education has seen immense change recently. Increased demand for iterative evaluation of trainees from medical school to independent practice has led to the generation of an overwhelming amount of data related to an individual's competency. Artificial intelligence has been proposed as a solution to automate and standardize the ability of stakeholders to assess the technical and nontechnical abilities of a surgical trainee. In both the simulation and clinical environments, evidence supports the use of machine learning algorithms to both evaluate trainee skill and provide real-time and automated feedback, enabling a shortened learning curve for many key procedural skills and ensuring patient safety.
Collapse
Affiliation(s)
- Mitchell G Goldenberg
- Catherine & Joseph Aresty Department of Urology, USC Institute of Urology, University of Southern California, 1441 Eastlake Avenue, Suite 7416, Los Angeles, CA 90033, USA.
| |
Collapse
|
11
|
Zhang H, Wang J, Liu C, Du B, Xiao J, Qian L, Zhang Q, Li J. Development of a continuously perfused ex vivo kidney training model for laparoscopic partial nephrectomy: validity and efficiency. Int J Surg 2023; 109:3919-3928. [PMID: 37737884 PMCID: PMC10720863 DOI: 10.1097/js9.0000000000000753] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Accepted: 09/04/2023] [Indexed: 09/23/2023]
Abstract
BACKGROUND Suture hemostasis is essential for laparoscopic partial nephrectomy (LPN). This study aimed to develop, validate, and test the efficacy of a continuously perfused training model (CPTM) in LPN with high-level simulated bleeding. MATERIALS AND METHODS The CPTM was constructed using fresh porcine kidneys with renal arteries continuously perfused with red-dyed liquid gelatin. Twenty-nine participants with expert, intermediate, or novice laparoscopic experience levels were recruited. The expert and intermediate participants evaluated the CPTM, and the novice participants were randomly assigned to one of two groups to complete training on a CPTM or dry box training model (DBTM). Messick's framework criteria were utilized to assess the validity and training efficacy of the model. The data were analyzed using the Mann-Whitney U , Kruskal-Wallis, and Friedman tests. A value of P< 0.05 was considered statistically significant. RESULTS Positive comments were provided by all experts and intermediates for the Content . The Relationships with other variables demonstrated significant differences among novices, intermediates, and experts in all metrics ( P< 0.05). The Consequences showed that the CPTM helped novices acquire LPN skills. The training efficacy was significantly better than that of the DBTM ( P< 0.05). There were no significant differences between the final performances of the novices and the initial performances of the experts ( P >0.05). Synthesizing all metrics, the LPN skills learned using CPTMs were significantly improved in the 12th round of training. CONCLUSION The CPTM offered a high-level simulation of bleeding with realistic tissue texture for acquiring LPN skills. Training of no fewer than 12 rounds is recommended for a novice's LPN training on the CPTM.
Collapse
Affiliation(s)
| | - Jun Wang
- Department of Anatomy, Guangdong Provincial Key Laboratory of Digital Medicine and Biomechanics, Guangdong Engineering Research Center for Translation of Medical 3D Printing Application, National Virtual & Reality Experimental Education Center for Medical Morphology, School of Basic Medical Sciences
| | - Chundong Liu
- Department of Urology, The Third Affiliated Hospital, Southern Medical University, Guangzhou
| | - Bingran Du
- Department of Stomatology, Shunde Hospital, Southern Medical University (The First People's Hospital of Shunde), Foshan
| | - Jujiao Xiao
- Center of Clinical Skills, Affiliated Jining First People's Hospital, Shandong First Medical University, Jining
| | - Lei Qian
- Department of Anatomy, Guangdong Provincial Key Laboratory of Digital Medicine and Biomechanics, Guangdong Engineering Research Center for Translation of Medical 3D Printing Application, National Virtual & Reality Experimental Education Center for Medical Morphology, School of Basic Medical Sciences
| | - Qun Zhang
- The Third Affiliated Hospital, Southern Medical University, Guangzhou, China
| | - Jianyi Li
- Department of Anatomy, Guangdong Provincial Key Laboratory of Digital Medicine and Biomechanics, Guangdong Engineering Research Center for Translation of Medical 3D Printing Application, National Virtual & Reality Experimental Education Center for Medical Morphology, School of Basic Medical Sciences
| |
Collapse
|
12
|
Pietersen PI, Hertz P, Olsen RG, Møller LB, Konge L, Bjerrum F. Transfer of skills between laparoscopic and robot-assisted surgery: a systematic review. Surg Endosc 2023; 37:9030-9042. [PMID: 37875694 DOI: 10.1007/s00464-023-10472-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Accepted: 09/17/2023] [Indexed: 10/26/2023]
Abstract
BACKGROUND Robot-assisted surgery is today well-implemented in many surgical specialties, but requires another skill set than laparoscopy. Most often, robot-assisted surgery is considered add-on to laparoscopic skills but very little is known about the transfer of skills. The aim of the study was to examine to what extent surgical skills are transferable between laparoscopic and robot-assisted surgery. METHODS A systematic search was conducted in three databases (Ovid Medline, Embase, and Web of Science). Studies investigating transfer of skills between laparoscopy and robot-assisted surgery in either a phantom-based, simulation-based, animal model, or clinical setting were eligible for inclusion. Quality assessment was done using the Medical education research study quality instrument and educational New Ottawa Scale. RESULTS Of 15,610 studies identified, 89 studies continued to full-text reading, and 37 studies were included. Four studies were found non-comparable and were left out of the results for the primary outcome. All 33 studies explored transfer from laparoscopy to robot-assisted surgery and 17 found a positive transfer whereas 15 did not. Only 11 studies explored transfer from robot-assisted surgery to laparoscopy, of which only three found a positive transfer. CONCLUSION An almost equal number of publications found a positive transfer and no transfer from laparoscopic to robot-assisted surgery. Fewer studies explored the transfer from robot-assisted surgery to laparoscopy. Very little evidence supports that surgeons trained solely in robot-assisted surgery can perform laparoscopy. This must be considered in future training programs as robot-assisted surgery is expected to become the first-in-line modality for many future surgeons.
Collapse
Affiliation(s)
- Pia Iben Pietersen
- Department of Radiology, Odense University Hospital, Kløvervænget 10, Entrance 112, 2nd floor, 5000, Odense C, Denmark.
- Simulation Center (SimC), Odense University Hospital, Odense, Denmark.
| | - Peter Hertz
- Department of Surgery, Hospital Lillebaelt, University of Southern Denmark, Kolding, Denmark
| | - Rikke Groth Olsen
- Copenhagen Prostate Cancer Center, Rigshospitalet, Copenhagen, Denmark
- Center for HR & Education, The Capital Region of Denmark, Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark
| | - Louise Birch Møller
- Center for HR & Education, The Capital Region of Denmark, Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark
| | - Lars Konge
- Center for HR & Education, The Capital Region of Denmark, Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark
- University of Copenhagen, Copenhagen, Denmark
| | - Flemming Bjerrum
- Center for HR & Education, The Capital Region of Denmark, Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark
- Department of Gastrointestinal and Hepatic Diseases, Copenhagen University Hospital - Herlev and Gentofte, Herlev, Denmark
| |
Collapse
|
13
|
Whittaker G, Ghita IA, Taylor M, Salmasi MY, Granato F, Athanasiou T. Current Status of Simulation in Thoracic Surgical Training. Ann Thorac Surg 2023; 116:1107-1115. [PMID: 37201622 DOI: 10.1016/j.athoracsur.2023.05.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Revised: 03/21/2023] [Accepted: 05/01/2023] [Indexed: 05/20/2023]
Abstract
BACKGROUND Simulation is playing an increasingly important role in surgical training but is not yet a mandatory part of most surgical curricula. A simulator must undergo rigorous validation to verify it as a reliable tool. The aim of this study was to review the literature to identify simulators that are currently available to augment thoracic surgical training and to analyze any evidence supporting or validating them. METHODS A literature search of the MEDLINE (1946 to November 2022) and Embase (1947 to November 2022) databases was performed to identify simulators for basic skills and procedures in thoracic surgery. A selection of keywords were used to perform the literature search. After identification of appropriate articles, data were extracted and analyzed. RESULTS Thirty-three simulators were found in 31 articles. Simulators for basic skills (n = 13) and thoracic lobectomy (n = 13) were most commonly described, followed by miscellaneous (n = 7). Most models were of a hybrid modality (n = 18). Evidence of validity was established in 48.5% (n = 16) of simulators. In total, 15.2% (n = 5) of simulators had 3 or more elements of validity demonstrated, and only 3.0% (n = 1) accomplished full validation. CONCLUSIONS Numerous simulators of varying modality and fidelity exist for a variety of thoracic surgical skills and procedures, although validation evidence is frequently inadequate. Simulation models may be able to provide training in basic surgical and procedural skills; however, further assessment of validity needs to be undertaken before consideration of their integration into training programs.
Collapse
Affiliation(s)
- George Whittaker
- Department of Surgery and Cancer, Imperial College London, London, United Kingdom; Department of Cardiothoracic Surgery, Manchester University NHS Foundation Trust, Manchester, United Kingdom.
| | - Ioana-Alexandra Ghita
- Faculty of Medicine, University of Medicine and Pharmacy of Craiova, Craiova, Romania
| | - Marcus Taylor
- Department of Cardiothoracic Surgery, Manchester University NHS Foundation Trust, Manchester, United Kingdom
| | - M Yousuf Salmasi
- Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | - Felice Granato
- Department of Cardiothoracic Surgery, Manchester University NHS Foundation Trust, Manchester, United Kingdom
| | - Thanos Athanasiou
- Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| |
Collapse
|
14
|
Williams SC, Ahmed R, Davids JD, Funnell JP, Hanrahan JG, Layard Horsfall H, Muirhead W, Nicolosi F, Thorne L, Marcus HJ, Grover P. Benchtop simulation of the retrosigmoid approach: Validation of a surgical simulator and development of a task-specific outcome measure score. World Neurosurg X 2023; 20:100230. [PMID: 37456690 PMCID: PMC10344945 DOI: 10.1016/j.wnsx.2023.100230] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2022] [Revised: 05/11/2023] [Accepted: 06/12/2023] [Indexed: 07/18/2023] Open
Abstract
Background Neurosurgical training is changing globally. Reduced working hours and training opportunities, increased patient safety expectations, and the impact of COVID-19 have reduced operative exposure. Benchtop simulators enable trainees to develop surgical skills in a controlled environment. We aim to validate a high-fidelity simulator model (RetrosigmoidBox, UpSurgeOn) for the retrosigmoid approach to the cerebellopontine angle (CPA). Methods Novice and expert Neurosurgeons and Ear, Nose, and Throat surgeons performed a surgical task using the model - identification of the trigeminal nerve. Experts completed a post-task questionnaire examining face and content validity. Construct validity was assessed through scoring of operative videos employing Objective Structured Assessment of Technical Skills (OSATS) and a novel Task-Specific Outcome Measure score. Results Fifteen novice and five expert participants were recruited. Forty percent of experts agreed or strongly agreed that the brain tissue looked real. Experts unanimously agreed that the RetrosigmoidBox was appropriate for teaching. Statistically significant differences were noted in task performance between novices and experts, demonstrating construct validity. Median total OSATS score was 14/25 (IQR 10-19) for novices and 22/25 (IQR 20-22) for experts (p < 0.05). Median Task-Specific Outcome Measure score was 10/20 (IQR 7-17) for novices compared to 19/20 (IQR 18.5-19.5) for experts (p < 0.05). Conclusion The RetrosigmoidBox benchtop simulator has a high degree of content and construct validity and moderate face validity. The changing landscape of neurosurgical training mean that simulators are likely to become increasingly important in the delivery of high-quality education. We demonstrate the validity of a Task-Specific Outcome Measure score for performance assessment of a simulated approach to the CPA.
Collapse
Affiliation(s)
- Simon C. Williams
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, UK
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), London, UK
| | - Razna Ahmed
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), London, UK
- Queen Square Institute of Neurology, University College London, London, UK
| | - Joseph Darlington Davids
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, UK
- Institute of Global Health Innovation and Hamlyn Centre for Robotics Surgery, Imperial College London, London, UK
| | - Jonathan P. Funnell
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, UK
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), London, UK
| | - John Gerrard Hanrahan
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, UK
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), London, UK
| | - Hugo Layard Horsfall
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, UK
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), London, UK
| | - William Muirhead
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, UK
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), London, UK
| | - Federico Nicolosi
- School of Medicine and Surgery, University of Milano-Bicocca, Monza, Italy
| | - Lewis Thorne
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, UK
| | - Hani J. Marcus
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, UK
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), London, UK
| | - Patrick Grover
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, London, UK
| |
Collapse
|
15
|
Frithioff A, Frendø M, Foghsgaard S, Sørensen MS, Andersen SAW. Are Video Recordings Reliable for Assessing Surgical Performance? A Prospective Reliability Study Using Generalizability Theory. Simul Healthc 2023; 18:219-225. [PMID: 36260767 DOI: 10.1097/sih.0000000000000672] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
INTRODUCTION Reliability is pivotal in surgical skills assessment. Video-based assessment can be used for objective assessment without physical presence of assessors. However, its reliability for surgical assessments remains largely unexplored. In this study, we evaluated the reliability of video-based versus physical assessments of novices' surgical performances on human cadavers and 3D-printed models-an emerging simulation modality. METHODS Eighteen otorhinolaryngology residents performed 2 to 3 mastoidectomies on a 3D-printed model and 1 procedure on a human cadaver. Performances were rated by 3 experts evaluating the final surgical result using a well-known assessment tool. Performances were rated both hands-on/physically and by video recordings. Interrater reliability and intrarater reliability were explored using κ statistics and the optimal number of raters and performances required in either assessment modality was determined using generalizability theory. RESULTS Interrater reliability was moderate with a mean κ score of 0.58 (range 0.53-0.62) for video-based assessment and 0.60 (range, 0.55-0.69) for physical assessment. Video-based and physical assessments were equally reliable (G coefficient 0.85 vs. 0.80 for 3D-printed models and 0.86 vs 0.87 for cadaver dissections). The interaction between rater and assessment modality contributed to 8.1% to 9.1% of the estimated variance. For the 3D-printed models, 2 raters evaluating 2 video-recorded performances or 3 raters physically assessing 2 performances yielded sufficient reliability for high-stakes assessment (G coefficient >0.8). CONCLUSIONS Video-based and physical assessments were equally reliable. Some raters were affected by changing from physical to video-based assessment; consequently, assessment should be either physical or video based, not a combination.
Collapse
Affiliation(s)
- Andreas Frithioff
- From the Copenhagen Hearing and Balance Center, Department of Otorhinolaryngology-Head & Neck Surgery and Audiology (A.F., M.F., S.F., M.S., S.A.W.A.), Rigshospitalet, Copenhagen; and Copenhagen Academy for Medical Education and Simulation (CAMES; A.F., M.F., S.A.W.A.), Center for HR & Education, Copenhagen, Denmark
| | | | | | | | | |
Collapse
|
16
|
Frithioff A, Frendø M, Weiss K, Foghsgaard S, Mikkelsen PT, Frederiksen TW, Pedersen DB, Sørensen MS, Andersen SAW. 3-D-Printed Models for Temporal Bone Training: A Validity Study. Otol Neurotol 2023; 44:e497-e503. [PMID: 37442608 DOI: 10.1097/mao.0000000000003936] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/15/2023]
Abstract
OBJECTIVE 3-D printing offers convenient and low-cost mastoidectomy training; nonetheless, training benefits using 3-D-printed temporal bones remain largely unexplored. In this study, we have collected validity evidence for a low-cost, 3-D-printed temporal bone for mastoidectomy training and established a credible pass/fail score for performance on the model. STUDY DESIGN A prospective educational study gathering validity evidence using Messick's validity framework. SETTING Seven Danish otorhinolaryngology training institutions. PARTICIPANTS Eighteen otorhinolaryngology residents (novices) and 11 experienced otosurgeons (experts). INTERVENTION Residents and experienced otosurgeons each performed two to three anatomical mastoidectomies on a low-cost, 3-D-printed temporal bone model produced in-house. After drilling, mastoidectomy performances were rated by three blinded experts using a 25-item modified Welling scale (WS). MAIN OUTCOME MEASURE Validity evidence using Messick's framework including reliability assessment applying both classical test theory and Generalizability theory. RESULTS Novices achieved a mean score of 13.9 points; experienced otosurgeons achieved 23.2 points. Using the contrasting groups method, we established a 21/25-point pass/fail level. The Generalizability coefficient was 0.91, and 75% of the score variance was attributable to participant performance, indicating a high level of assessment reliability. Subsequent D studies revealed that two raters rating one performance or one rater rating two performances were sufficiently reliable for high-stakes assessment. CONCLUSION Validity evidence supports using a low-cost, 3-D-printed model for mastoidectomy training. The model can be printed in-house using consumer-grade 3-D printers and serves as an additional training tool in the temporal bone curriculum. For competency-based training, we established a cut-off score of 21 of 25 WS points using the contrasting groups method.
Collapse
Affiliation(s)
| | | | - Kenneth Weiss
- Department of Civil and Mechanical Engineering, Technical University of Denmark, Kgs. Lyngby
| | - Søren Foghsgaard
- Copenhagen Hearing and Balance Center, Dept. of Otorhinolaryngology-Head & Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark
| | - Peter Trier Mikkelsen
- Copenhagen Hearing and Balance Center, Dept. of Otorhinolaryngology-Head & Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark
| | | | - David Bue Pedersen
- Department of Civil and Mechanical Engineering, Technical University of Denmark, Kgs. Lyngby
| | - Mads Sølvsten Sørensen
- Copenhagen Hearing and Balance Center, Dept. of Otorhinolaryngology-Head & Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark
| | | |
Collapse
|
17
|
Mizunuma K, Kurashima Y, Poudel S, Watanabe Y, Noji T, Nakamura T, Okamura K, Shichinohe T, Hirano S. Surgical skills assessment of pancreaticojejunostomy using a simulator may predict patient outcomes: A multicenter prospective observational study. Surgery 2023; 173:1374-1380. [PMID: 37003952 DOI: 10.1016/j.surg.2023.02.027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2022] [Revised: 12/30/2022] [Accepted: 02/23/2023] [Indexed: 04/03/2023]
Abstract
BACKGROUND Pancreatoduodenectomy, an advanced surgical procedure with a high complication rate, requires surgical skill in performing pancreaticojejunostomy, which correlates with operative outcomes. We aimed to analyze the correlation between pancreaticojejunostomy assessment conducted in a simulator environment and the operating room and patient clinical outcomes. METHODS We recruited 30 surgeons (with different experience levels in pancreatoduodenectomy) from 11 institutes. Three trained blinded raters assessed the videos of the pancreaticojejunostomy procedure performed in the operating room using a simulator according to an objective structured assessment of technical skill and a newly developed pancreaticojejunostomy assessment scale. The correlations between the assessment score of the pancreaticojejunostomy performed in the operating room and using the simulator and between each assessment score and patient outcomes were calculated. The participants were also surveyed regarding various aspects of the simulator as a training tool. RESULTS There was no correlation between the average score of the pancreaticojejunostomy performed in the operating room and that in the simulator environment (r = 0.047). Pancreaticojejunostomy scores using the simulator were significantly lower in patients with postoperative pancreatic fistula than in those without postoperative pancreatic fistula (P = .05). Multivariate analysis showed that pancreaticojejunostomy assessment scores were independent factors in postoperative pancreatic fistula (P = .09). The participants highly rated the simulator and considered that it had the potential to be used for training. CONCLUSION There was no correlation between pancreaticojejunostomy surgical performance in the operating room and the simulation environment. Surgical skills evaluated in the simulation setting could predict patient surgical outcomes.
Collapse
Affiliation(s)
- Kenichi Mizunuma
- Department of Gastroenterologial Surgery II, Hokkaido University Faculty of Medicine, Sapporo, Japan
| | - Yo Kurashima
- Department of Gastroenterologial Surgery II, Hokkaido University Faculty of Medicine, Sapporo, Japan; Clinical Simulation Center, Hokkaido University Graduate School of Medicine, Sapporo, Japan
| | - Saseem Poudel
- Department of Gastroenterologial Surgery II, Hokkaido University Faculty of Medicine, Sapporo, Japan.
| | - Yusuke Watanabe
- Clinical Research and Medical Innovation Center, Institute of Health Science Innovation for Medical Care, Hokkaido University Hospital, Sapporo, Japan
| | - Takehiro Noji
- Department of Gastroenterologial Surgery II, Hokkaido University Faculty of Medicine, Sapporo, Japan
| | - Toru Nakamura
- Department of Gastroenterologial Surgery II, Hokkaido University Faculty of Medicine, Sapporo, Japan
| | - Keisuke Okamura
- Department of Gastroenterologial Surgery II, Hokkaido University Faculty of Medicine, Sapporo, Japan
| | - Toshiaki Shichinohe
- Department of Gastroenterologial Surgery II, Hokkaido University Faculty of Medicine, Sapporo, Japan
| | - Satoshi Hirano
- Department of Gastroenterologial Surgery II, Hokkaido University Faculty of Medicine, Sapporo, Japan
| |
Collapse
|
18
|
Ahmed R, Muirhead W, Williams SC, Bagchi B, Datta P, Gupta P, Salvadores Fernandez C, Funnell JP, Hanrahan JG, Davids JD, Grover P, Tiwari MK, Murphy M, Marcus HJ. A synthetic model simulator for intracranial aneurysm clipping: validation of the UpSurgeOn AneurysmBox. Front Surg 2023; 10:1185516. [PMID: 37325417 PMCID: PMC10264641 DOI: 10.3389/fsurg.2023.1185516] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2023] [Accepted: 05/17/2023] [Indexed: 06/17/2023] Open
Abstract
Background and objectives In recent decades, the rise of endovascular management of aneurysms has led to a significant decline in operative training for surgical aneurysm clipping. Simulation has the potential to bridge this gap and benchtop synthetic simulators aim to combine the best of both anatomical realism and haptic feedback. The aim of this study was to validate a synthetic benchtop simulator for aneurysm clipping (AneurysmBox, UpSurgeOn). Methods Expert and novice surgeons from multiple neurosurgical centres were asked to clip a terminal internal carotid artery aneurysm using the AneurysmBox. Face and content validity were evaluated using Likert scales by asking experts to complete a post-task questionnaire. Construct validity was evaluated by comparing expert and novice performance using the modified Objective Structured Assessment of Technical Skills (mOSATS), developing a curriculum-derived assessment of Specific Technical Skills (STS), and measuring the forces exerted using a force-sensitive glove. Results Ten experts and eighteen novices completed the task. Most experts agreed that the brain looked realistic (8/10), but far fewer agreed that the brain felt realistic (2/10). Half the expert participants (5/10) agreed that the aneurysm clip application task was realistic. When compared to novices, experts had a significantly higher median mOSATS (27 vs. 14.5; p < 0.01) and STS score (18 vs. 9; p < 0.01); the STS score was strongly correlated with the previously validated mOSATS score (p < 0.01). Overall, there was a trend towards experts exerting a lower median force than novices, however, these differences were not statistically significant (3.8 N vs. 4.0 N; p = 0.77). Suggested improvements for the model included reduced stiffness and the addition of cerebrospinal fluid (CSF) and arachnoid mater. Conclusion At present, the AneurysmBox has equivocal face and content validity, and future versions may benefit from materials that allow for improved haptic feedback. Nonetheless, it has good construct validity, suggesting it is a promising adjunct to training.
Collapse
Affiliation(s)
- Razna Ahmed
- Queen Square Institute of Neurology, University College London, London, United Kingdom
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, United Kingdom
| | - William Muirhead
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, United Kingdom
- Victor Horsley Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, United Kingdom
| | - Simon C. Williams
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, United Kingdom
- Victor Horsley Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, United Kingdom
| | - Biswajoy Bagchi
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, United Kingdom
- Nanoengineered Systems Laboratory, Department of Mechanical Engineering, University College London, London, United Kingdom
| | - Priyankan Datta
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, United Kingdom
- Nanoengineered Systems Laboratory, Department of Mechanical Engineering, University College London, London, United Kingdom
| | - Priya Gupta
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, United Kingdom
- Nanoengineered Systems Laboratory, Department of Mechanical Engineering, University College London, London, United Kingdom
| | - Carmen Salvadores Fernandez
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, United Kingdom
- Nanoengineered Systems Laboratory, Department of Mechanical Engineering, University College London, London, United Kingdom
| | - Jonathan P. Funnell
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, United Kingdom
- Victor Horsley Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, United Kingdom
| | - John G. Hanrahan
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, United Kingdom
- Victor Horsley Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, United Kingdom
| | - Joseph D. Davids
- Victor Horsley Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, United Kingdom
- Institute of Global Health Innovation and Hamlyn Centre for Robotics Surgery, Imperial College London, London, United Kingdom
| | - Patrick Grover
- Victor Horsley Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, United Kingdom
| | - Manish K. Tiwari
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, United Kingdom
- Nanoengineered Systems Laboratory, Department of Mechanical Engineering, University College London, London, United Kingdom
| | - Mary Murphy
- Victor Horsley Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, United Kingdom
| | - Hani J. Marcus
- Queen Square Institute of Neurology, University College London, London, United Kingdom
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, United Kingdom
- Victor Horsley Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, London, United Kingdom
| |
Collapse
|
19
|
Stackhouse AA, Rafi D, Walls R, Dodd RV, Badger K, Davies DJ, Brown CA, Cowell A, Meeran K, Halse O, Kinross J, Lupton M, Hughes EA, Sam AH. Knowledge Attainment and Engagement Among Medical Students: A Comparison of Three Forms of Online Learning. ADVANCES IN MEDICAL EDUCATION AND PRACTICE 2023; 14:373-380. [PMID: 37101695 PMCID: PMC10124742 DOI: 10.2147/amep.s391816] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 10/10/2022] [Accepted: 02/28/2023] [Indexed: 06/19/2023]
Abstract
OBJECTIVE This study compared knowledge attainment and student enjoyment and engagement between clinical case vignette, patient-testimony videos and mixed reality (MR) teaching via the Microsoft HoloLens 2, all delivered remotely to third year medical students. The feasibility of conducting MR teaching on a large scale was also assessed. SETTING & PARTICIPANTS Medical students in Year 3 at Imperial College London participated in three online teaching sessions, one in each format. All students were expected to attend these scheduled teaching sessions and to complete the formative assessment. Inclusion of their data used as part of the research trial was optional. PRIMARY AND SECONDARY OUTCOME MEASURES The primary outcome measure was performance on a formative assessment, which served to compare knowledge attainment between three forms of online learning. Moreover, we aimed to explore student engagement with each form of learning via a questionnaire, and also feasibility of applying MR as a teaching tool on a large scale. Comparisons between performances on the formative assessment between the three groups were investigated using a repeated measures two-way ANOVA. Engagement and enjoyment were also analysed in the same manner. RESULTS A total of 252 students participated in the study. Knowledge attainment of students using MR was comparable with the other two methods. Participants reported higher enjoyment and engagement (p<0.001) for the case vignette method, compared with MR and video-based teaching. There was no difference in enjoyment or engagement ratings between MR and the video-based methods. CONCLUSION This study demonstrated that the implementation of MR is an effective, acceptable, and feasible way of teaching clinical medicine to undergraduate students on a large scale. However, case-based tutorials were found to be favoured most by students. Future work could further explore the best uses for MR teaching within the medical curriculum.
Collapse
Affiliation(s)
| | - Damir Rafi
- Imperial College School of Medicine, Imperial College London, London, UK
| | - Risheka Walls
- Imperial College School of Medicine, Imperial College London, London, UK
- Imperial College Healthcare NHS Trust, London, UK
| | - Rebecca V Dodd
- Imperial College School of Medicine, Imperial College London, London, UK
| | - Kerry Badger
- Imperial College School of Medicine, Imperial College London, London, UK
| | - Daniel J Davies
- Imperial College School of Medicine, Imperial College London, London, UK
| | - Celia A Brown
- Warwick Medical School, Department of Medicine, Warwick, UK
| | - Adrian Cowell
- Imperial College School of Medicine, Imperial College London, London, UK
| | - Karim Meeran
- Imperial College School of Medicine, Imperial College London, London, UK
| | - Omid Halse
- Imperial College School of Medicine, Imperial College London, London, UK
| | - James Kinross
- Department of Surgery and Cancer, Imperial College London, London, UK
| | - Martin Lupton
- Imperial College School of Medicine, Imperial College London, London, UK
| | | | - Amir H Sam
- Imperial College School of Medicine, Imperial College London, London, UK
| |
Collapse
|
20
|
Frithioff A, Weiss K, Frendø M, Senn P, Mikkelsen PT, Sieber D, Sørensen MS, Pedersen DB, Andersen SAW. 3D-printing a cost-effective model for mastoidectomy training. 3D Print Med 2023; 9:12. [PMID: 37062800 PMCID: PMC10108487 DOI: 10.1186/s41205-023-00174-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2022] [Accepted: 03/24/2023] [Indexed: 04/18/2023] Open
Abstract
BACKGROUND 3D-printed temporal bone models can potentially provide a cost-effective alternative to cadaver surgery that can be manufactured locally at the training department. The objective of this study was to create a cost-effective 3D-printed model suitable for mastoidectomy training using entry level and commercially available print technologies, enabling individuals, without prior experience on 3D-printing, to manufacture their own models for basic temporal bone training. METHODS Expert technical professionals and an experienced otosurgeon identified the best material for replicating the temporal bone and created a cost-effective printing routine for the model using entry-level print technologies. Eleven participants at a temporal bone dissection course evaluated the model using a questionnaire. RESULTS The 3D-printed temporal bone model was printed using a material extrusion 3D-printer with a heat resistant filament, reducing melting during drilling. After printing, a few simple post-processing steps were designed to replicate the dura, sigmoid sinus and facial nerve. Modifying the 3D-printer by installing a direct-drive and ruby nozzle resulted in more successful prints and less need for maintenance. Upon evaluation by otorhinolaryngology trainees, unanimous feedback was that the model provided a good introduction to the mastoidectomy procedure, and supplementing practice to cadaveric temporal bones. CONCLUSION In-house production of a cost-effective 3D-printed model for temporal bone training is feasible and enables training institutions to manufacture their own models. Further, this work demonstrates the feasibility of creating new temporal bone models with anatomical variation to provide ample training opportunity.
Collapse
Affiliation(s)
- Andreas Frithioff
- Copenhagen Hearing and Balance Center, Dept. of Otorhinolaryngology-Head & Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark.
| | - Kenneth Weiss
- Department of Mechanical Engineering, Technical University of Denmark, Kgs. Lyngby, Denmark
| | - Martin Frendø
- Copenhagen Hearing and Balance Center, Dept. of Otorhinolaryngology-Head & Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR & Education, Region H, Copenhagen, Denmark
- Department of Plastic Surgery, Herlev & Gentofte Hospital, Copenhagen, Denmark
| | - Pascal Senn
- Department of Clinical Neurosciences, Service of ORL & Head and Neck Surgery, University Hospital of Geneva, Geneva, Switzerland
| | - Peter Trier Mikkelsen
- Copenhagen Hearing and Balance Center, Dept. of Otorhinolaryngology-Head & Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark
| | - Daniel Sieber
- Department of Medical & Health Technologies, MCI | The Entrepreneurial School, Innsbruck, Austria
| | - Mads Sølvsten Sørensen
- Copenhagen Hearing and Balance Center, Dept. of Otorhinolaryngology-Head & Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark
- Institute for Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | - David Bue Pedersen
- Department of Mechanical Engineering, Technical University of Denmark, Kgs. Lyngby, Denmark
| | - Steven Arild Wuyts Andersen
- Copenhagen Hearing and Balance Center, Dept. of Otorhinolaryngology-Head & Neck Surgery and Audiology, Rigshospitalet, Copenhagen, Denmark
| |
Collapse
|
21
|
Richardson CM, Zopf DA, Ikeda AK, van Horn A, Cohen K, Nourmohammadi Z, Nassar M, Park JS, Johnson KE. A Validated 3D Printed Laryngeal Suturing Simulator for Endoscopic Laryngeal Cleft Repair. Laryngoscope 2023; 133:785-791. [PMID: 35932231 DOI: 10.1002/lary.30320] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2022] [Revised: 07/01/2022] [Accepted: 07/05/2022] [Indexed: 11/08/2022]
Abstract
OBJECTIVES Endoscopic laryngeal cleft repair (ELCR) with endolaryngeal suturing is an advanced surgical skill. This study objective was to assess the validity of 3-dimensionally (3D) printed laryngeal suturing simulator for ELCR. STUDY DESIGN Development and validation of a simulator for ELCR. METHODS An ELCR model was developed using 3D printed and readily available materials. Participants were surveyed before and after a simulation session using five-point Likert scale questions. Performance data was assessed using blinded expert video review and rated using a novel objective structured assessment of technical skills (OSATS) for endoscopic laryngeal suturing. RESULTS Twenty-one participants ranging from residents to attendings completed the simulation session. Survey respondents reported on a five-point Likert scale that the model was "easy to use" and "quite realistic" (both mean of 4). Confidence improved significantly in 86% of participants (p < 0.01). Overall OSATS scores (out of a total of 55) showed a median improvement in technical skills of 11.7 points (p = 0.004). OSATS demonstrated good intra-rater (κ = 0.689 and 0.677) and moderate inter-rater (κ = 0.573) reliability. Completion times improved from the first to the last suture by a median time of 512 to 350 s (decrease of 202 s, p = 0.002). Participants with no prior ELCR experience improved more than those with in vivo experience. CONCLUSION This study demonstrates the validity of a simulator utilizing 3D printed larynges for ELCR. A novel OSATS for endoscopic laryngeal suturing was successfully implemented. Confidence, technical skills, and completion times improved with the use of the model across a variety of participants. Laryngoscope, 133:785-791, 2023.
Collapse
Affiliation(s)
- Clare M Richardson
- Division of Pediatric of Otolaryngology - Head and Neck Surgery, Seattle Children's Hospital, Seattle, Washington, USA.,Department of Otolaryngology - Head and Neck Surgery, University of Washington, Seattle, Washington, USA
| | - David A Zopf
- Division of Pediatric of Otolaryngology - Head and Neck Surgery, CS Mott Children's Hospital, Ann Arbor, Michigan, USA.,Department of Otolaryngology - Head and Neck Surgery, University of Michigan, Ann Arbor, Michigan, USA
| | - Allison K Ikeda
- Department of Otolaryngology - Head and Neck Surgery, University of Washington, Seattle, Washington, USA
| | - Adam van Horn
- Division of Otolaryngology - Head and Neck Surgery, Marshall University, Huntington, West Virginia, USA
| | - Katheryn Cohen
- Department of Otolaryngology - Head and Neck Surgery, University of Michigan, Ann Arbor, Michigan, USA
| | - Zahra Nourmohammadi
- Department of Otolaryngology - Head and Neck Surgery, University of Michigan, Ann Arbor, Michigan, USA
| | - Michel Nassar
- Department of Otolaryngology - Head and Neck Surgery, Montefiore Medical Center, New York City, New York, USA
| | - Jason S Park
- Department of Otolaryngology - Head and Neck Surgery, Monroe Carell Jr. Children's Hospital at Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Kaalan E Johnson
- Division of Pediatric of Otolaryngology - Head and Neck Surgery, Seattle Children's Hospital, Seattle, Washington, USA.,Department of Otolaryngology - Head and Neck Surgery, University of Washington, Seattle, Washington, USA
| |
Collapse
|
22
|
Gomes SH, Trindade M, Petrisor C, Costa D, Correia-Pinto J, Costa PS, Pêgo JM. Objective structured assessment ultrasound skill scale for hyomental distance competence - psychometric study. BMC MEDICAL EDUCATION 2023; 23:177. [PMID: 36949512 PMCID: PMC10035246 DOI: 10.1186/s12909-023-04146-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/09/2022] [Accepted: 03/09/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND Ultrasound assessment of the airway recently integrates the point-of-care approach to patient evaluation since ultrasound measurements can predict a difficult laryngoscopy and tracheal intubation. Because ultrasonography is performer-dependent, a proper training and assessment tool is needed to increase diagnostic accuracy. An objective, structured assessment ultrasound skill (OSAUS) scale was recently developed to guide training and assess competence. This work aims to study the psychometric properties of OSAUS Scale when used to evaluate competence in ultrasound hyomental distance (HMD) measurement. METHODS Prospective and experimental study. Volunteers were recruited and enrolled in groups with different expertise. Each participant performed three ultrasonographic HMD evaluation. The performance was videorecorded and anonymized. Five assessors blindly rated participants' performance using OSAUS scale and a Global Rating Scale (GRS). A psychometric study of OSAUS scale as assessment tool for ultrasound HMD competence was done. RESULTS Fifteen voluntaries participated on the study. Psychometric analysis of OSAUS showed strong internal consistency (Cronbach's alpha 0.916) and inter-rater reliability (ICC 0.720; p < 0.001). The novice group scored 15.4±0.18 (mean±SD), the intermediate 14.3±0.75 and expert 13.6±0.1.25, with a significant difference between novice and expert groups (p = 0.036). The time in seconds to complete the task was evaluated: novice (90±34) (mean±SD), intermediate (84±23) and experts (83±15), with no significant differences between groups. A strong correlation was observed between OSAUS and global rating scale (r = 0.970, p < 0.001). CONCLUSION The study demonstrated evidence of validity and reliability. Further studies are needed to implement OSAUS scale in the clinical setting for training and assessment of airway ultrasound competence.
Collapse
Affiliation(s)
- Sara Hora Gomes
- Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Campus Gualtar, Braga, 4710-057, Portugal.
- ICVS/3B's-PT Government Associate Laboratory, Braga/Guimarães, 4710-057, Portugal.
| | - Marta Trindade
- Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Campus Gualtar, Braga, 4710-057, Portugal
| | - Cristina Petrisor
- Anesthesia and Intensive Care II Department, Pharmacy Cluj-Napoca and Anesthesia and Intensive Care Department, "Iuliu Hatieganu" University of Medicine, Clinical Emergency County Hospital, Cluj-Napoca, 400347, Romania
| | - Dinis Costa
- Department of Anesthesia, Hospital de Braga, Braga, 4710-243, Portugal
| | - Jorge Correia-Pinto
- Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Campus Gualtar, Braga, 4710-057, Portugal
- ICVS/3B's-PT Government Associate Laboratory, Braga/Guimarães, 4710-057, Portugal
- Department of Pediatric Surgery, Hospital de Braga, Braga, 4710-243, Portugal
| | - Patrício S Costa
- Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Campus Gualtar, Braga, 4710-057, Portugal
- ICVS/3B's-PT Government Associate Laboratory, Braga/Guimarães, 4710-057, Portugal
| | - José M Pêgo
- Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Campus Gualtar, Braga, 4710-057, Portugal
- ICVS/3B's-PT Government Associate Laboratory, Braga/Guimarães, 4710-057, Portugal
- iCognitus4ALL - IT Solutions, Braga, 4470-057, Portugal
| |
Collapse
|
23
|
El Hussein MT, Hakkola J. Valid and Reliable Tools to Measure Safety of Nursing Students During Simulated Learning Experiences: A Scoping Review. TEACHING AND LEARNING IN NURSING 2023. [DOI: 10.1016/j.teln.2022.12.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/09/2023]
|
24
|
Jørgensen RJ, Olsen RG, Svendsen MBS, Stadeager M, Konge L, Bjerrum F. Comparing Simulator Metrics and Rater Assessment of Laparoscopic Suturing Skills. JOURNAL OF SURGICAL EDUCATION 2023; 80:302-310. [PMID: 37683093 DOI: 10.1016/j.jsurg.2022.09.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Revised: 08/17/2022] [Accepted: 09/25/2022] [Indexed: 09/10/2023]
Abstract
BACKGROUND Laparoscopic intracorporeal suturing is important to master and competence should be ensured using an optimal method in a simulated environment before proceeding to real operations. The objectives of this study were to gather validity evidence for two tools for assessing laparoscopic intracorporeal knot tying and compare the rater-based assessment of laparoscopic intracorporeal suturing with the assessment based on simulator metrics. METHODS Twenty-eight novices and 19 experienced surgeons performed four laparoscopic sutures on a Simball Box simulator twice. Two surgeons used the Intracorporeal Suturing Assessment Tool (ISAT) for blinded video rating. RESULTS Composite Simulator Score (CSS) had higher test-retest reliability than the ISAT. The correlation between the number performed procedures including suturing and ISAT score was 0.51, p<0.001, and 0.59 p<0.001 for CSS. We found an inter-rater reliability (0.72, p<0.001 for test 1 and 0.53 p<0.001 for test 2). The pass/fail rates for ISAT and CSS were similar. CONCLUSION CSS and ISAT provide similar results for assessing laparoscopic suturing but assess different aspects of performance. Using simulator metrics and raters' assessments in combination should be considered for a more comprehensive evaluation of laparoscopic knot-tying competency.
Collapse
Affiliation(s)
- Rikke Jeong Jørgensen
- Copenhagen Academy for Medical Education and Simulation, Centre for HR and Education, Capital Region, Copenhagen, Denmark.
| | - Rikke Groth Olsen
- Copenhagen Academy for Medical Education and Simulation, Centre for HR and Education, Capital Region, Copenhagen, Denmark
| | - Morten Bo Søndergaard Svendsen
- Copenhagen Academy for Medical Education and Simulation, Centre for HR and Education, Capital Region, Copenhagen, Denmark
| | - Morten Stadeager
- Copenhagen Academy for Medical Education and Simulation, Centre for HR and Education, Capital Region, Copenhagen, Denmark; Department of Surgery, Hvidovre Hospital, Copenhagen University Hospital, Copenhagen, Denmark
| | - Lars Konge
- Copenhagen Academy for Medical Education and Simulation, Centre for HR and Education, Capital Region, Copenhagen, Denmark; University of Copenhagen, Copenhagen, Denmark
| | - Flemming Bjerrum
- Copenhagen Academy for Medical Education and Simulation, Centre for HR and Education, Capital Region, Copenhagen, Denmark; Department of Surgery, Herlev-Gentofte Hospital, Herlev, Denmark
| |
Collapse
|
25
|
Stefan P, Pfandler M, Kullmann A, Eck U, Koch A, Mehren C, von der Heide A, Weidert S, Fürmetz J, Euler E, Lazarovici M, Navab N, Weigl M. Computer-assisted simulated workplace-based assessment in surgery: application of the universal framework of intraoperative performance within a mixed-reality simulation. BMJ SURGERY, INTERVENTIONS, & HEALTH TECHNOLOGIES 2023; 5:e000135. [PMID: 36687799 PMCID: PMC9853221 DOI: 10.1136/bmjsit-2022-000135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2022] [Accepted: 08/24/2022] [Indexed: 01/20/2023] Open
Abstract
Objectives Workplace-based assessment (WBA) is a key requirement of competency-based medical education in postgraduate surgical education. Although simulated workplace-based assessment (SWBA) has been proposed to complement WBA, it is insufficiently adopted in surgical education. In particular, approaches to criterion-referenced and automated assessment of intraoperative surgical competency in contextualized SWBA settings are missing.Main objectives were (1) application of the universal framework of intraoperative performance and exemplary adaptation to spine surgery (vertebroplasty); (2) development of computer-assisted assessment based on criterion-referenced metrics; and (3) implementation in contextualized, team-based operating room (OR) simulation, and evaluation of validity. Design Multistage development and assessment study: (1) expert-based definition of performance indicators based on framework's performance domains; (2) development of respective assessment metrics based on preoperative planning and intraoperative performance data; (3) implementation in mixed-reality OR simulation and assessment of surgeons operating in a confederate team. Statistical analyses included internal consistency and interdomain associations, correlations with experience, and technical and non-technical performances. Setting Surgical simulation center. Full surgical team set-up within mixed-reality OR simulation. Participants Eleven surgeons were recruited from two teaching hospitals. Eligibility criteria included surgical specialists in orthopedic, trauma, or neurosurgery with prior VP or kyphoplasty experience. Main outcome measures Computer-assisted assessment of surgeons' intraoperative performance. Results Performance scores were associated with surgeons' experience, observational assessment (Objective Structured Assessment of Technical Skill) scores and overall pass/fail ratings. Results provide strong evidence for validity of our computer-assisted SWBA approach. Diverse indicators of surgeons' technical and non-technical performances could be quantified and captured. Conclusions This study is the first to investigate computer-assisted assessment based on a competency framework in authentic, contextualized team-based OR simulation. Our approach discriminates surgical competency across the domains of intraoperative performance. It advances previous automated assessment based on the use of current surgical simulators in decontextualized settings. Our findings inform future use of computer-assisted multidomain competency assessments of surgeons using SWBA approaches.
Collapse
Affiliation(s)
- Philipp Stefan
- Chair for Computer Aided Medical Procedures and Augmented Reality, Department of Informatics, Technical University of Munich, München, Germany
| | - Michael Pfandler
- Institute and Outpatient Clinic for Occupational, Social, and Environmental Medicine, University Hospital, Ludwig Maximilians University Munich, München, Germany
| | - Aljoscha Kullmann
- Chair for Computer Aided Medical Procedures and Augmented Reality, Department of Informatics, Technical University of Munich, München, Germany
| | - Ulrich Eck
- Chair for Computer Aided Medical Procedures and Augmented Reality, Department of Informatics, Technical University of Munich, München, Germany
| | - Amelie Koch
- Institute and Outpatient Clinic for Occupational, Social, and Environmental Medicine, University Hospital, Ludwig Maximilians University Munich, München, Germany
| | - Christoph Mehren
- Spine Center, Schön Klinik München Harlaching, München, Germany,Academic Teaching Hospital and Spine Research Institute, Paracelsus Medical University, Salzburg, Austria
| | - Anna von der Heide
- Department of General, Trauma and Reconstructive Surgery, University Hospital, Campus Grosshadern, Ludwig Maximilians University Munich, München, Germany
| | - Simon Weidert
- Department of General, Trauma and Reconstructive Surgery, University Hospital, Campus Grosshadern, Ludwig Maximilians University Munich, München, Germany
| | - Julian Fürmetz
- Department of General, Trauma and Reconstructive Surgery, University Hospital, Campus Innenstadt, Ludwig Maximilians University Munich, München, Germany
| | - Ekkehard Euler
- Department of General, Trauma and Reconstructive Surgery, University Hospital, Campus Innenstadt, Ludwig Maximilians University Munich, München, Germany
| | - Marc Lazarovici
- Institute for Emergency Medicine and Management in Medicine (INM), University Hospital, Ludwig Maximilians University Munich, München, Germany
| | - Nassir Navab
- Chair for Computer Aided Medical Procedures and Augmented Reality, Department of Informatics, Technical University of Munich, München, Germany
| | - Matthias Weigl
- Institute and Outpatient Clinic for Occupational, Social, and Environmental Medicine, University Hospital, Ludwig Maximilians University Munich, München, Germany,Institute for Patient Safety, University of Bonn, Bonn, Germany
| |
Collapse
|
26
|
Galvez-Yanjari V, de la Fuente R, Munoz-Gama J, Sepúlveda M. The Sequence of Steps: A Key Concept Missing in Surgical Training-A Systematic Review and Recommendations to Include It. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2023; 20:1436. [PMID: 36674190 PMCID: PMC9859547 DOI: 10.3390/ijerph20021436] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Revised: 12/30/2022] [Accepted: 01/04/2023] [Indexed: 06/17/2023]
Abstract
BACKGROUND Surgical procedures have an inherent feature, which is the sequence of steps. Moreover, studies have shown variability in surgeons' performances, which is valuable to expose residents to different ways to perform a procedure. However, it is unclear how to include the sequence of steps in training programs. METHODS We conducted a systematic review, including studies reporting explicit teaching of a standard sequence of steps, where assessment considered adherence to a standard sequence, and where faculty or students at any level participated. We searched for articles on PubMed, EMBASE, CINAHL, Web of Science, and Google Scholar databases. RESULTS We selected nine articles that met the inclusion criteria. The main strategy to teach the sequence was to use videos to demonstrate the procedure. The simulation was the main strategy to assess the learning of the sequence of steps. Non-standardized scoring protocols and written tests with variable validity evidence were the instruments used to assess the learning, and were focused on adherence to a standard sequence and the omission of steps. CONCLUSIONS Teaching and learning assessment of a standard sequence of steps is scarcely reported in procedural skills training literature. More research is needed to evaluate whether the new strategies to teach and assess the order of steps work. We recommend the use of Surgical Process Models and Surgical Data Science to incorporate the sequence of steps when teaching and assessing procedural skills.
Collapse
Affiliation(s)
- Victor Galvez-Yanjari
- Department of Computer Science, School of Engineering, Pontificia Universidad Católica de Chile, Santiago 8331150, Chile
| | - Rene de la Fuente
- Division of Anesthesiology, School of Medicine, Pontificia Universidad Católica de Chile, Santiago 8330024, Chile
| | - Jorge Munoz-Gama
- Department of Computer Science, School of Engineering, Pontificia Universidad Católica de Chile, Santiago 8331150, Chile
| | - Marcos Sepúlveda
- Department of Computer Science, School of Engineering, Pontificia Universidad Católica de Chile, Santiago 8331150, Chile
| |
Collapse
|
27
|
Buléon C, Mattatia L, Minehart RD, Rudolph JW, Lois FJ, Guillouet E, Philippon AL, Brissaud O, Lefevre-Scelles A, Benhamou D, Lecomte F, group TSAWS, Bellot A, Crublé I, Philippot G, Vanderlinden T, Batrancourt S, Boithias-Guerot C, Bréaud J, de Vries P, Sibert L, Sécheresse T, Boulant V, Delamarre L, Grillet L, Jund M, Mathurin C, Berthod J, Debien B, Gacia O, Der Sahakian G, Boet S, Oriot D, Chabot JM. Simulation-based summative assessment in healthcare: an overview of key principles for practice. ADVANCES IN SIMULATION (LONDON, ENGLAND) 2022; 7:42. [PMID: 36578052 PMCID: PMC9795938 DOI: 10.1186/s41077-022-00238-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/02/2022] [Accepted: 11/30/2022] [Indexed: 12/29/2022]
Abstract
BACKGROUND Healthcare curricula need summative assessments relevant to and representative of clinical situations to best select and train learners. Simulation provides multiple benefits with a growing literature base proving its utility for training in a formative context. Advancing to the next step, "the use of simulation for summative assessment" requires rigorous and evidence-based development because any summative assessment is high stakes for participants, trainers, and programs. The first step of this process is to identify the baseline from which we can start. METHODS First, using a modified nominal group technique, a task force of 34 panelists defined topics to clarify the why, how, what, when, and who for using simulation-based summative assessment (SBSA). Second, each topic was explored by a group of panelists based on state-of-the-art literature reviews technique with a snowball method to identify further references. Our goal was to identify current knowledge and potential recommendations for future directions. Results were cross-checked among groups and reviewed by an independent expert committee. RESULTS Seven topics were selected by the task force: "What can be assessed in simulation?", "Assessment tools for SBSA", "Consequences of undergoing the SBSA process", "Scenarios for SBSA", "Debriefing, video, and research for SBSA", "Trainers for SBSA", and "Implementation of SBSA in healthcare". Together, these seven explorations provide an overview of what is known and can be done with relative certainty, and what is unknown and probably needs further investigation. Based on this work, we highlighted the trustworthiness of different summative assessment-related conclusions, the remaining important problems and questions, and their consequences for participants and institutions of how SBSA is conducted. CONCLUSION Our results identified among the seven topics one area with robust evidence in the literature ("What can be assessed in simulation?"), three areas with evidence that require guidance by expert opinion ("Assessment tools for SBSA", "Scenarios for SBSA", "Implementation of SBSA in healthcare"), and three areas with weak or emerging evidence ("Consequences of undergoing the SBSA process", "Debriefing for SBSA", "Trainers for SBSA"). Using SBSA holds much promise, with increasing demand for this application. Due to the important stakes involved, it must be rigorously conducted and supervised. Guidelines for good practice should be formalized to help with conduct and implementation. We believe this baseline can direct future investigation and the development of guidelines.
Collapse
Affiliation(s)
- Clément Buléon
- grid.460771.30000 0004 1785 9671Department of Anesthesiology, Intensive Care and Perioperative Medicine, Caen Normandy University Hospital, 6th Floor, Caen, France ,grid.412043.00000 0001 2186 4076Medical School, University of Caen Normandy, Caen, France ,grid.419998.40000 0004 0452 5971Center for Medical Simulation, Boston, MA USA
| | - Laurent Mattatia
- grid.411165.60000 0004 0593 8241Department of Anesthesiology, Intensive Care and Perioperative Medicine, Nîmes University Hospital, Nîmes, France
| | - Rebecca D. Minehart
- grid.419998.40000 0004 0452 5971Center for Medical Simulation, Boston, MA USA ,grid.32224.350000 0004 0386 9924Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, MA USA ,grid.38142.3c000000041936754XHarvard Medical School, Boston, MA USA
| | - Jenny W. Rudolph
- grid.419998.40000 0004 0452 5971Center for Medical Simulation, Boston, MA USA ,grid.32224.350000 0004 0386 9924Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, MA USA ,grid.38142.3c000000041936754XHarvard Medical School, Boston, MA USA
| | - Fernande J. Lois
- grid.4861.b0000 0001 0805 7253Department of Anesthesiology, Intensive Care and Perioperative Medicine, Liège University Hospital, Liège, Belgique
| | - Erwan Guillouet
- grid.460771.30000 0004 1785 9671Department of Anesthesiology, Intensive Care and Perioperative Medicine, Caen Normandy University Hospital, 6th Floor, Caen, France ,grid.412043.00000 0001 2186 4076Medical School, University of Caen Normandy, Caen, France
| | - Anne-Laure Philippon
- grid.411439.a0000 0001 2150 9058Department of Emergency Medicine, Pitié Salpêtrière University Hospital, APHP, Paris, France
| | - Olivier Brissaud
- grid.42399.350000 0004 0593 7118Department of Pediatric Intensive Care, Pellegrin University Hospital, Bordeaux, France
| | - Antoine Lefevre-Scelles
- grid.41724.340000 0001 2296 5231Department of Emergency Medicine, Rouen University Hospital, Rouen, France
| | - Dan Benhamou
- grid.413784.d0000 0001 2181 7253Department of Anesthesiology, Intensive Care and Perioperative Medicine, Kremlin Bicêtre University Hospital, APHP, Paris, France
| | - François Lecomte
- grid.411784.f0000 0001 0274 3893Department of Emergency Medicine, Cochin University Hospital, APHP, Paris, France
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
28
|
Tjønnås MS, Guzmán-García C, Sánchez-González P, Gómez EJ, Oropesa I, Våpenstad C. Stress in surgical educational environments: a systematic review. BMC MEDICAL EDUCATION 2022; 22:791. [PMID: 36380334 PMCID: PMC9667591 DOI: 10.1186/s12909-022-03841-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Accepted: 10/29/2022] [Indexed: 06/16/2023]
Abstract
BACKGROUND The effects of stress on surgical residents and how stress management training can prepare residents to effectively manage stressful situations is a relevant topic. This systematic review aimed to analyze the literature regarding (1) the current stress monitoring tools and their use in surgical environments, (2) the current methods in surgical stress management training, and (3) how stress affects surgical performance. METHODS A search strategy was implemented to retrieve relevant articles from Web of Science, Scopus, and PubMed. The 787 initially retrieved articles were reviewed for further evaluation according to the inclusion/exclusion criteria (Prospero registration number CRD42021252682). RESULTS Sixty-one articles were included in the review. The stress monitoring methods found in the articles showed heart rate analysis as the most used monitoring tool for physiological parameters while the STAI-6 scale was preferred for psychological parameters. The stress management methods found in the articles were mental-, simulation- and feedback-based training, with the mental-based training showing clear positive effects on participants. The studies analyzing the effects of stress on surgical performance showed both negative and positive effects on technical and non-technical performance. CONCLUSIONS The impact of stress responses presents an important factor in surgical environments, affecting residents' training and performance. This study identified the main methods used for monitoring stress parameters in surgical educational environments. The applied surgical stress management training methods were diverse and demonstrated positive effects on surgeons' stress levels and performance. There were negative and positive effects of stress on surgical performance, although a collective pattern on their effects was not clear.
Collapse
Affiliation(s)
- Maria Suong Tjønnås
- Department of Neuromedicine and Movement Science (INB), Faculty of Medicine and Health Sciences, NTNU, Norwegian University of Science and Technology, N-7491, Trondheim, Norway.
- SINTEF Digital, Health Department, Trondheim, Norway.
| | - Carmen Guzmán-García
- Biomedical Engineering and Telemedicine Centre (GBT), ETSI Telecomunicación, Center for Biomedical Technology, Universidad Politécnica de Madrid (UPM), Madrid, Spain
| | - Patricia Sánchez-González
- Biomedical Engineering and Telemedicine Centre (GBT), ETSI Telecomunicación, Center for Biomedical Technology, Universidad Politécnica de Madrid (UPM), Madrid, Spain
- Networking Research Center on Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN), Madrid, Spain
| | - Enrique Javier Gómez
- Biomedical Engineering and Telemedicine Centre (GBT), ETSI Telecomunicación, Center for Biomedical Technology, Universidad Politécnica de Madrid (UPM), Madrid, Spain
- Networking Research Center on Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN), Madrid, Spain
| | - Ignacio Oropesa
- Biomedical Engineering and Telemedicine Centre (GBT), ETSI Telecomunicación, Center for Biomedical Technology, Universidad Politécnica de Madrid (UPM), Madrid, Spain
| | - Cecilie Våpenstad
- SINTEF Digital, Health Department, Trondheim, Norway
- Department of Clinical and Molecular Medicine (IKOM), Faculty of Medicine and Health Sciences, NTNU, Norwegian University of Science and Technology, Trondheim, Norway
| |
Collapse
|
29
|
Assessing VATS competence based on simulated lobectomies of all five lung lobes. Surg Endosc 2022; 36:8067-8075. [PMID: 35467146 DOI: 10.1007/s00464-022-09235-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2021] [Accepted: 04/02/2022] [Indexed: 01/06/2023]
Abstract
OBJECTIVES To determine the number of procedures and expert raters necessary to provide a reliable assessment of competence in Video-Assisted Thoracoscopic Surgery (VATS) lobectomy. METHODS Three randomly selected VATS lobectomies were performed on a virtual reality simulator by participants with varying experience in VATS. Video recordings of the procedures were independently rated by three blinded VATS experts using a modified VATS lobectomy assessment tool (VATSAT). The unitary framework of validity was used to describe validity evidence, and generalizability theory was used to explore the reliability of different assessment options. RESULTS Forty-one participants (22 novices, 10 intermediates, and 9 experienced) performed a total of 123 lobectomies. Internal consistency reliability, inter-rater reliability, and test-retest reliability were 0.94, 0.85, and 0.90, respectively. Generalizability theory found that a minimum of two procedures and four raters or three procedures and three raters were needed to ensure the overall reliability of 0.8. ANOVA showed significant differences in test scores between the three groups (P < 0.001). A pass/fail level of 19 out of 25 points was established using the contrasting groups' standard setting method, leaving one false positive (one novice passed) and zero false negatives (all experienced passed). CONCLUSION We demonstrated validity evidence for a VR simulator test with different lung lobes, and a credible pass/fail level was identified. Our results can be used to implement a standardized mastery learning training program for trainees in VATS lobectomies that ensures that everyone reaches basic competency before performing supervised operations on patients.
Collapse
|
30
|
Ortiz C, Vela J, Contreras C, Belmar F, Paul I, Zinco A, Ramos JP, Ottolino P, Achurra P, Jarufe N, Alseidi A, Varas J. A new approach for the acquisition of trauma surgical skills: an OSCE type of simulation training program. Surg Endosc 2022; 36:8441-8450. [PMID: 35237901 PMCID: PMC8890468 DOI: 10.1007/s00464-022-09098-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Accepted: 02/07/2022] [Indexed: 01/06/2023]
Abstract
BACKGROUND Worldwide, trauma-related deaths are one of the main causes of mortality. Appropriate surgical treatment is crucial to prevent mortality, however, in the past decade, general surgery residents' exposure to trauma cases has decreased, particularly since the COVID-19 pandemic. In this context, accessible simulation-based training scenarios are essential. METHODS A low-cost, previously tested OSCE scenario for the evaluation of surgical skills in trauma was implemented as part of a short training boot camp for residents and recently graduated surgeons. The following stations were included bowel anastomosis, vascular anastomosis, penetrating lung injury, penetrating cardiac injury, and gastric perforation (laparoscopic suturing). A total of 75 participants from 15 different programs were recruited. Each station was videotaped in high definition and assessed in a remote and asynchronous manner. The level of competency was assessed through global and specific rating scales alongside procedural times. Self-confidence to perform the procedure as the leading surgeon was evaluated before and after training. RESULTS Statistically significant differences were found in pre-training scores between groups for all stations. The lowest scores were obtained in the cardiac and lung injury stations. After training, participants significantly increased their level of competence in both grading systems. Procedural times for the pulmonary tractotomy, bowel anastomosis, and vascular anastomosis stations increased after training. A significant improvement in self-confidence was shown in all stations. CONCLUSION An OSCE scenario for training surgical skills in trauma was effective in improving proficiency level and self-confidence. Low pre-training scores and level of confidence in the cardiac and lung injury stations represent a deficit in residency programs that should be addressed. The incorporation of simulation-based teaching tools at early stages in residency would be beneficial when future surgeons face extremely severe trauma scenarios.
Collapse
Affiliation(s)
- Catalina Ortiz
- Department of Digestive Surgery, Pontificia Universidad Católica de Chile, Santiago, Chile
- Experimental Surgery and Simulation Center, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Javier Vela
- Department of Digestive Surgery, Pontificia Universidad Católica de Chile, Santiago, Chile
- Experimental Surgery and Simulation Center, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Caterina Contreras
- Department of Digestive Surgery, Pontificia Universidad Católica de Chile, Santiago, Chile
- Experimental Surgery and Simulation Center, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Francisca Belmar
- Department of Digestive Surgery, Pontificia Universidad Católica de Chile, Santiago, Chile
- Experimental Surgery and Simulation Center, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Ivan Paul
- Department of Digestive Surgery, Pontificia Universidad Católica de Chile, Santiago, Chile
- Experimental Surgery and Simulation Center, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Analia Zinco
- Trauma Surgery Department, Hospital Sótero del Río, Santiago, Chile
| | - Juan Pablo Ramos
- Trauma Surgery Department, Hospital Sótero del Río, Santiago, Chile
| | - Pablo Ottolino
- Experimental Surgery and Simulation Center, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
- Trauma Surgery Department, Hospital Sótero del Río, Santiago, Chile
| | - Pablo Achurra
- Department of Digestive Surgery, Pontificia Universidad Católica de Chile, Santiago, Chile
- Experimental Surgery and Simulation Center, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Nicolas Jarufe
- Department of Surgery, Clínica Las Condes, Santiago, Chile
| | - Adnan Alseidi
- Department of Surgery, University of California San Francisco, San Francisco, USA
| | - Julian Varas
- Department of Digestive Surgery, Pontificia Universidad Católica de Chile, Santiago, Chile.
- Experimental Surgery and Simulation Center, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile.
| |
Collapse
|
31
|
Oviedo-Peñata CA, Giraldo Mejía GE, Riaño-Benavides CH, Maldonado-Estrada JG, Lemos Duque JD. Development and validation of a composed canine simulator for advanced veterinary laparoscopic training. Front Vet Sci 2022; 9:936144. [PMID: 36325095 PMCID: PMC9621388 DOI: 10.3389/fvets.2022.936144] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2022] [Accepted: 09/05/2022] [Indexed: 11/04/2022] Open
Abstract
The development of innovative simulation models for veterinary laparoscopic surgery training is a priority today. This study aimed to describe a didactic simulation tool for the training of total laparoscopic gastropexy (TLG) with intracorporeal sutures in dogs. CALMA Veterinary Lap-trainer composite simulator (CLVTS) was developed from a plaster cast of 2 Great Dane canines mimicking the space and the correct position to carry out a TLG. After video instruction, 16 veterinarians with different degrees of experience in minimally invasive surgery (Experts, n = 6 and intermediates, n = 10) evaluated four sequential simulating TLG with intracorporeal suturing in the CLVTS. Subsequently, they completed an anonymous questionnaire analyzing the realism, usefulness, and educational quality of the simulator. The CLVTS showed a good preliminary acceptance (4.7/5) in terms of the usefulness and adequacy of the exercises that, in the participants' opinion, are appropriate and are related to the difficulty of the TLG. In addition, both experienced and intermediate surgeons gave high marks (4.5/5) to the feeling of realism, design, and practicality. There were no significant differences between the responses of the two groups. The results suggest that the CVLTS has both face and content validity. Where it can be practiced in a structured environment for the development of a total laparoscopic gastropexy with intracorporeal suture and without compromising patient safety, but still has some limitations of the scope of the study. Further studies are needed to establish the ability to assess or measure technical skills, including the degree of transferability to the actual surgical environment.
Collapse
Affiliation(s)
- Carlos A. Oviedo-Peñata
- Tropical Animal Production Research Group, Faculty of Veterinary Medicine and Zootechny, University of Cordoba, Monteria, Colombia,OHVRI-Research Group, Faculty of Agrarian Sciences, College of Veterinary Medicine, University of Antioquia, Medellín, Colombia,*Correspondence: Carlos A. Oviedo-Peñata
| | - Gloria E. Giraldo Mejía
- OHVRI-Research Group, Faculty of Agrarian Sciences, College of Veterinary Medicine, University of Antioquia, Medellín, Colombia
| | - Carlos Humberto Riaño-Benavides
- OHVRI-Research Group, Faculty of Agrarian Sciences, College of Veterinary Medicine, University of Antioquia, Medellín, Colombia
| | - Juan G. Maldonado-Estrada
- OHVRI-Research Group, Faculty of Agrarian Sciences, College of Veterinary Medicine, University of Antioquia, Medellín, Colombia
| | - Juan D. Lemos Duque
- Bioinstrumentation and Clinical Engineering Research Group-GIBIC, Department of Bioengineering, Faculty of Engineering, Universidad de Antioquia, Medellín, Colombia
| |
Collapse
|
32
|
Training outside of the operating room improves intern resident psychomotor skills on a validated ASSH tool. SURGERY IN PRACTICE AND SCIENCE 2022. [DOI: 10.1016/j.sipas.2022.100099] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
|
33
|
Lowater SJ, Grauslund J, Vergmann AS. Modern Educational Simulation-Based Tools Among Residents of Ophthalmology: A Narrative Review. Ophthalmol Ther 2022; 11:1961-1974. [PMID: 36001249 DOI: 10.1007/s40123-022-00559-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2022] [Accepted: 08/03/2022] [Indexed: 11/29/2022] Open
Abstract
INTRODUCTION With no specific overview on simulation-based training for educational purposes among residents in ophthalmology, this narrative review attempts to highlight the current literature on modern educational simulation-based tools used to educate residents in ophthalmology. METHODS We searched databases Web of Science and PubMed between March 15 and July 21, 2022. Relevant and accessible articles and abstracts published after 2006 and in English only were included. RESULTS Simulation-based cataract surgery is associated with better outcomes in the operating room and faster surgeries. Construct validity has been established across different procedures and levels in simulation-based cataract surgery and simulation-based vitreoretinal surgery. Other simulation-based procedures indicate promising results but in general lack evidence-based validity. DISCUSSION This narrative review highlights and evaluates the current and relevant literature of modern educational simulation-based tools to train ophthalmology residents in different fundamental skills like simulation-based ophthalmoscopy and complex surgical procedures like simulation-based cataract surgery and vitreoretinal surgery. Some studies attempt to develop simulators for the use in education of ophthalmology residents. Other studies strive to establish validity of the respective procedures or modern education tools and some studies investigate the effect of simulation-based training. The most validated modern educational simulation-based tool is the Eyesi Surgical Simulator (VRmagic, Germany). However, other modern educational simulation-based tools have also been evaluated, including the HelpMeSee Eye Surgery Simulator (HelpMeSee Inc., New York, USA) and the MicroVisTouch Surgical Simulator (ImmersiveTouch, USA). CONCLUSION Simulation-based training has already been established for residents in ophthalmology to benefit the most from skill-demanding procedures resulting in better learning and better patient handling. Future studies should aim to validate more simulation-based procedures for the teaching of ophthalmology residents so that the evidence is kept at a high standard.
Collapse
Affiliation(s)
- Simon J Lowater
- Research Unit of Ophthalmology, Department of Ophthalmology, Odense University Hospital, J. B. Winsløws Vej 4, 5000, Odense C, Denmark.
| | - Jakob Grauslund
- Research Unit of Ophthalmology, Department of Ophthalmology, Odense University Hospital, J. B. Winsløws Vej 4, 5000, Odense C, Denmark.,Department of Clinical Research, University of Southern Denmark, Odense, Denmark
| | - Anna S Vergmann
- Research Unit of Ophthalmology, Department of Ophthalmology, Odense University Hospital, J. B. Winsløws Vej 4, 5000, Odense C, Denmark
| |
Collapse
|
34
|
Planas De Lathawer V. An exploration of Third-Year student midwives’ experiences of High-Risk Module Assessment in preparation for practice and real-world emergencies. Midwifery 2022; 114:103450. [DOI: 10.1016/j.midw.2022.103450] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Revised: 07/14/2022] [Accepted: 08/05/2022] [Indexed: 11/29/2022]
|
35
|
Haiser A, Aydin A, Kunduzi B, Ahmed K, Dasgupta P. A Systematic Review of Simulation-Based Training in Vascular Surgery. J Surg Res 2022; 279:409-419. [PMID: 35839575 PMCID: PMC9483723 DOI: 10.1016/j.jss.2022.05.009] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Revised: 04/20/2022] [Accepted: 05/22/2022] [Indexed: 11/25/2022]
Abstract
Introduction Recent advancements in surgical technology, reduced working hours, and training opportunities exacerbated by the COVID-19 pandemic have led to an increase in simulation-based training. Furthermore, a rise in endovascular procedures has led to a requirement for high-fidelity simulators that offer comprehensive feedback. This review aims to identify vascular surgery simulation models and assess their validity and levels of effectiveness (LoE) for each model in order to successfully implement them into current training curricula. Methods PubMed and EMBASE were searched on January 1, 2021, for full-text English studies on vascular surgery simulators. Eligible articles were given validity ratings based on Messick’s modern concept of validity alongside an LoE score according to McGaghie’s translational outcomes. Results Overall 76 eligible articles validated 34 vascular surgery simulators and training courses for open and endovascular procedures. High validity ratings were achieved across studies for: content (35), response processes (12), the internal structure (5), relations to other variables (57), and consequences (2). Only seven studies achieved an LoE greater than 3/5. Overall, ANGIO Mentor was the most highly validated and effective simulator and was the only simulator to achieve an LoE of 5/5. Conclusions Simulation-based training in vascular surgery is a continuously developing field with exciting future prospects, demonstrated by the vast number of models and training courses. To effectively integrate simulation models into current vascular surgery curricula and assessments, there is a need for studies to look at trainee skill retention over a longer period of time. A more detailed discussion on cost-effectiveness is also needed.
Collapse
Affiliation(s)
- Alexander Haiser
- Guy's, King's and St Thomas' School of Medical Education, King's College London, London, UK
| | - Abdullatif Aydin
- MRC Centre for Transplantation, Guy's Hospital, King's College London, London, UK.
| | - Basir Kunduzi
- Department of Transplant Surgery, Guy's and St. Thomas' NHS Foundation Trust, London, London, UK
| | - Kamran Ahmed
- MRC Centre for Transplantation, Guy's Hospital, King's College London, London, UK
| | - Prokar Dasgupta
- MRC Centre for Transplantation, Guy's Hospital, King's College London, London, UK
| |
Collapse
|
36
|
Evidence-based assessment of acupuncture skills using a newly developed simulation-based test. WORLD JOURNAL OF ACUPUNCTURE-MOXIBUSTION 2022. [DOI: 10.1016/j.wjam.2022.07.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
37
|
De Mol L, Desender L, Van Herzeele I, Van de Voorde P, Konge L, Willaert W. Assessing competence in Chest Tube Insertion with the ACTION-tool: A Delphi study. Int J Surg 2022; 104:106791. [DOI: 10.1016/j.ijsu.2022.106791] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Revised: 07/06/2022] [Accepted: 07/08/2022] [Indexed: 10/16/2022]
|
38
|
Bube SH, Kingo PS, Madsen MG, Vásquez JL, Norus T, Olsen RG, Dahl C, Hansen RB, Konge L, Azawi N. National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients. EUR UROL SUPPL 2022; 39:29-35. [PMID: 35528788 PMCID: PMC9068726 DOI: 10.1016/j.euros.2022.03.003] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/09/2022] [Indexed: 11/02/2022] Open
Abstract
Background Objective Design, setting, and participants Outcome measurements and statistical analyses Results and limitations Conclusions Patient summary
Collapse
|
39
|
Jacobsen N, Larsen JD, Falster C, Nolsøe CP, Konge L, Graumann O, Laursen CB. Using Immersive Virtual Reality Simulation to Ensure Competence in Contrast-Enhanced Ultrasound. ULTRASOUND IN MEDICINE & BIOLOGY 2022; 48:912-923. [PMID: 35227531 DOI: 10.1016/j.ultrasmedbio.2022.01.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/19/2021] [Revised: 01/11/2022] [Accepted: 01/24/2022] [Indexed: 06/14/2023]
Abstract
Contrast-enhanced ultrasound (CEUS) is used in various medical specialties as a diagnostic imaging tool and for procedural guidance. Experience in the procedure is currently attained via supervised clinical practice that is challenged by patient availability and risks. Prior simulation-based training and subsequent assessment could improve and ensure competence before performance on patients, but no simulator currently exists. Immersive virtual reality (IVR) is a new promising simulation tool that can replicate complex interactions and environments that are unfeasible to achieve by traditional simulators. This study was aimed at developing an IVR simulation-based test for core CEUS competencies and gathering validity evidence for the test in accordance with Messick's framework. The test was developed by IVR software specialists and clinical experts in CEUS and medical education and imitated a CEUS examination of a patient with a focal liver lesion with emphasis on the pre-contrast preparations. Twenty-five medical doctors with varying CEUS experience were recruited as test participants, and their results were used to analyze test quality and to establish a pass/fail standard. The final test of 23 test items had good internal reliability (Cronbach's α = 0.85) and discriminatory abilities. The risks of false positives and negatives (9.1% and 23.6%, respectively) were acceptable for the test to be used as a certification tool prior to supervised clinical training in CEUS.
Collapse
Affiliation(s)
- Niels Jacobsen
- Department of Respiratory Medicine, Odense University Hospital, Odense, Denmark; Odense Respiratory Research Unit (ODIN), Department of Clinical Research, University of Southern Denmark, Odense, Demark; Regional Center for Technical Simulation (TechSim), Odense University Hospital, Odense, Denmark.
| | - Jonas D Larsen
- Odense Respiratory Research Unit (ODIN), Department of Clinical Research, University of Southern Denmark, Odense, Demark; Department of Radiology, Odense University Hospital, Odense, Denmark; Research and Innovation Unit of Radiology, University of Southern Denmark, Odense, Denmark
| | - Casper Falster
- Department of Respiratory Medicine, Odense University Hospital, Odense, Denmark; Odense Respiratory Research Unit (ODIN), Department of Clinical Research, University of Southern Denmark, Odense, Demark
| | - Christian P Nolsøe
- Center for Surgical Ultrasound, Department of Surgery, Zealand University Hospital, Køge, Denmark; Copenhagen Academy for Medical Education and Simulation (CAMES), Center for Human Resources and Education, The Capital Region of Denmark, Copenhagen, Denmark
| | - Lars Konge
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for Human Resources and Education, The Capital Region of Denmark, Copenhagen, Denmark
| | - Ole Graumann
- Department of Radiology, Odense University Hospital, Odense, Denmark; Research and Innovation Unit of Radiology, University of Southern Denmark, Odense, Denmark
| | - Christian B Laursen
- Department of Respiratory Medicine, Odense University Hospital, Odense, Denmark; Odense Respiratory Research Unit (ODIN), Department of Clinical Research, University of Southern Denmark, Odense, Demark
| |
Collapse
|
40
|
Fjørtoft K, Konge L, Christensen J, Thinggard E. Overcoming Gender Bias in Assessment of Surgical Skills. JOURNAL OF SURGICAL EDUCATION 2022; 79:753-760. [PMID: 35115269 DOI: 10.1016/j.jsurg.2022.01.006] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Revised: 12/20/2021] [Accepted: 01/08/2022] [Indexed: 06/14/2023]
Abstract
OBJECTIVE Female and male surgical residents are treated differently based on their gender. The use of assessment tools can help obtain objectivity in surgical skills assessment, avoid gender bias, and promote equal learning opportunities. The objective of the study was to explore whether knowledge of gender causes bias in the rating of surgical skills, and whether the raters' gender, surgical specialty, or experience affect ratings. DESIGN The study is designed as a cross-sectional study, where an anonymous video showing surgical performance was rated by surgeons from different surgical specialties and different levels of surgical experience. The same video was presented to two randomized groups as either a male or female surgical trainee performing the procedure. SETTING The participants used Objective Structured Assessment of Technical Skills (OSATS) to rate a video-recorded exam from a surgical skills course at Copenhagen Academy for Medical Education and Simulation (CAMES). The video rating was done through an internet-based platform and could thus be performed at home or at another location of choice. PARTICIPANTS Novice and experienced surgeons from departments of gastrointestinal surgery, gynaecology, and urology in Denmark. RESULTS There was no difference in OSATS rating score of the perceived female and male trainee (female trainee (n = 50):12.26 (SD = 3.08), male trainee (n = 57): 12.00 (SD = 3.28), p-value = 0.67). Rater characteristics: gender, age, surgical experience, and surgical specialty did not affect ratings. CONCLUSIONS When OSATS a systematic assessment tool was used in rating surgical trainees we did not find any significant implicit gender bias. The rating was unaffected by the raters' gender, age, surgical specialty, or experience.
Collapse
Affiliation(s)
- Karn Fjørtoft
- Department of Surgical Gastroenterology, Bispebjerg Hospital, Copenhagen, Denmark; Capital Region, Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark.
| | - Lars Konge
- Capital Region, Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark.
| | - John Christensen
- Capital Region, Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark.
| | - Ebbe Thinggard
- Capital Region, Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark; Department of Gynaecology and Obstetrics, Hvidovre Hospital, Hvidovre, Denmark.
| |
Collapse
|
41
|
Carstensen SMD, Velander MJ, Konge L, Østergaard M, Pfeiffer Jensen M, Just SA, Terslev L. Training and assessment of musculoskeletal ultrasound and injection skills-a systematic review. Rheumatology (Oxford) 2022; 61:3889-3901. [PMID: 35218339 DOI: 10.1093/rheumatology/keac119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Revised: 02/17/2022] [Indexed: 11/12/2022] Open
Abstract
OBJECTIVES To examine how residents are trained and assessed in musculoskeletal ultrasound (MSUS), MSUS-guided and landmark-guided joint aspiration and injection. Additionally, to present the available assessment tools and examine their supporting validity evidence. METHODS A systematic search of PubMed, Cochrane Library, and Embase was conducted in accordance with the PRISMA guidelines and studies published from January 1, 2000 to May 31, 2021 were included. Two independent reviewers performed the search and data extraction. The studies were evaluated using the Medical Education Research Quality Instrument (MERSQI). RESULTS 9,884 articles were screened, and 43 were included; 3 were randomized studies, 21 pre- and post-test studies, 16 descriptive studies, and 3 studies developing assessment tools. The studies used various theoretical training modalities e.g. lectures, anatomical quizzes and e-learning. The practical training models varied from mannequins and cadavers to healthy volunteers and patients. The majority of studies used subjective "comfort level" as assessment, others used practical examination and/or theoretical examination. All training programs increased trainees' self-confidence, theoretical knowledge, and/or practical performance, however few used validated assessment tools to measure the effect. Only one study met the MERSQI high methodical quality cut-off score of 14. CONCLUSION The included studies were heterogeneous, and most were of poor methodological quality and not based on contemporary educational theories. This review highlights the need for educational studies using validated theoretical and practical assessment tools to ensure optimal MSUS training and assessment in rheumatology.
Collapse
Affiliation(s)
- Stine M D Carstensen
- Copenhagen Center for Arthritis Research, Center for Rheumatology and Spine Diseases, Centre for Head and Orthopaedics, Rigshospitalet Glostrup Valdemar Hansens Vej 17, Glostrup, Denmark.,Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.,Copenhagen Academy for Medical Education and Simulation, The Capital Region of Denmark, Copenhagen, Denmark
| | - Marie Juul Velander
- Section of Rheumatology, Department of Medicine, Svendborg Hospital-OUH Baagøes Allé 15, Svendborg, Denmark
| | - Lars Konge
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.,Copenhagen Academy for Medical Education and Simulation, The Capital Region of Denmark, Copenhagen, Denmark
| | - Mikkel Østergaard
- Copenhagen Center for Arthritis Research, Center for Rheumatology and Spine Diseases, Centre for Head and Orthopaedics, Rigshospitalet Glostrup Valdemar Hansens Vej 17, Glostrup, Denmark.,Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Mogens Pfeiffer Jensen
- Copenhagen Center for Arthritis Research, Center for Rheumatology and Spine Diseases, Centre for Head and Orthopaedics, Rigshospitalet Glostrup Valdemar Hansens Vej 17, Glostrup, Denmark.,Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Søren Andreas Just
- Section of Rheumatology, Department of Medicine, Svendborg Hospital-OUH Baagøes Allé 15, Svendborg, Denmark
| | - Lene Terslev
- Copenhagen Center for Arthritis Research, Center for Rheumatology and Spine Diseases, Centre for Head and Orthopaedics, Rigshospitalet Glostrup Valdemar Hansens Vej 17, Glostrup, Denmark.,Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
42
|
Ensuring competence in ultrasound-guided procedures-a validity study of a newly developed assessment tool. Eur Radiol 2022; 32:4954-4966. [PMID: 35195745 DOI: 10.1007/s00330-022-08542-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Revised: 11/17/2021] [Accepted: 12/16/2021] [Indexed: 11/04/2022]
Abstract
OBJECTIVES To investigate the validity of the Interventional Ultrasound Skills Evaluation (IUSE) tool for assessment of procedural competence in ultrasound-guided procedures in a clinical environment, including a pass/fail score. METHODS Novices and experienced radiologists were recruited from four hospitals and were observed and assessed while performing ultrasound-guided procedures. Performances were assessed using the IUSE tool by two independent raters. Validity evidence was gathered in accordance with Messick's framework: response process was ensured by standardisation of written rater instructions. Internal structure was explored using Cronbach's alpha for internal consistency reliability; inter-rater reliability was calculated as Pearson's r independently across all ratings, and test-retest reliability was reported using Cronbach's alpha. Relationship to other variables was investigated by comparing performances of the participants in each group. Consequences evidence was explored by calculating a pass/fail standard using the contrasting groups method. RESULTS Six novices and twelve experienced radiologists were enrolled. The IUSE tool had high internal consistency (Cronbach's alpha = 0.96, high inter-rater reliability (Pearson's r = 0.95), and high test-retest reliability (Cronbach's alpha = 0.98), and the mean score was 33.28 for novices and 59.25 for experienced with a highly significant difference (p value < 0.001). The pass/fail score was set at 55 resulting in no false positives or false negatives. CONCLUSIONS Validity evidence from multiple sources supports the use of the IUSE tool for assessment of competence in ultrasound-guided procedures in a clinical environment and its use in high-stakes assessment such as certification. A credible pass/fail criterion was established to inform decision-making. KEY POINTS • A multi-site validity investigation established that the Interventional Ultrasound Skills Evaluation (IUSE) tool can be used to assess procedural competence in ultrasound-guided procedures. • Validity evidence was gathered according to Messick's framework validity from the following sources: response process, internal structure, relationship to other variables, and consequences evidence. • The IUSE tool can be used for both formative and summative assessment, and a credible pass/fail score was established to help inform decision-making such as certification.
Collapse
|
43
|
Chebib E, Lemarteleur V, Azalé M, Deneufbourg L, Ceccaldi PF, Teissier N. Step-by-step development and evaluation of a 3D printed home-made low-cost pediatric tracheobronchial tree for foreign body aspiration extractions. Int J Pediatr Otorhinolaryngol 2022; 153:111040. [PMID: 35026720 DOI: 10.1016/j.ijporl.2022.111040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/08/2021] [Revised: 11/12/2021] [Accepted: 01/05/2022] [Indexed: 10/19/2022]
Abstract
OBJECTIVES The management of foreign body aspirations (FBA) is dreaded by pediatric physicians due to the high risk of respiratory distress and a potential fatal outcome, favored by a lack of experience of young specialists. Furthermore, there has been an increasing requirement for low-cost simulation. The aim was to describe the step-by-step manufacturing process and to validate a low-cost, easily home-made training model of pediatric tracheo-bronchial tree (pTBT) for simulation-based training in order to teach young physicians to practice foreign body (FBA) extractions. METHODS A simulator was designed in order to reproduce the physical and esthetic properties of a pTBT. The production cost of a single simulator was estimated. The simulator was then tested by experienced physicians using a rigid bronchoscope. A manufacturing manual of the simulator is hereby presented. A group of 7 experienced pediatric otolaryngologists performed a FBA extraction in the conditions of installation of an operating room. RESULTS The result of the survey showed a high fidelity of the simulator in mimicking the biological esthetics and physical properties of a pTBT during a FBA extraction (mean 4.3 ± 0.8). The total cost of the custom-made simulator is about 20.5 € ($23.4) for the production of the first simulator. CONCLUSIONS A highly realistic and easily reproducible pediatric tracheo-bronchial tree simulator is presented and can therefore be used during simulation-based training.
Collapse
Affiliation(s)
- Emilien Chebib
- Department of Otolaryngology, Robert Debre Hospital, Assistance Publique Hôpitaux de Paris (APHP) and Faculty of Medecine, University of Paris, Paris, France; Simulation Department of University of Paris, Paris, France.
| | | | - Mehdi Azalé
- Department of Anaesthesia and Intensive Care, Robert Debre Hospital, Assistance Publique Hôpitaux de Paris (APHP) and Faculty of Medecine, University of Paris, Paris, France
| | | | | | - Natacha Teissier
- Department of Otolaryngology, Robert Debre Hospital, Assistance Publique Hôpitaux de Paris (APHP) and Faculty of Medecine, University of Paris, Paris, France
| |
Collapse
|
44
|
Heskin L, Galvin R, Traynor O, Simms C. Simulation in Upper and Lower Limb Trauma Skill Acquisition: A Review. Simul Healthc 2022; 17:54-65. [PMID: 34009908 DOI: 10.1097/sih.0000000000000570] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
SUMMARY STATEMENT This review aimed to explore the published evidence with regard to the types and composition of both full- and part-task trainers to teach surgeons extremity exploration procedures in limb trauma management. Studies were included if they reported the development and/or validation of synthetic or virtual task trainers. Studies were evaluated to determine their derivation, usability, and clinical utility.A total of 638 citations were identified and 63 satisfied the inclusion criteria. Twenty-five articles addressed simulator validation and 36 addressed level of learning achieved with simulator engagement. Two studies described a dedicated limb simulator. Simulators were developed to repair limb structures including skin (n = 15), tendon (n = 7), nerve (n = 1), fascia (n = 1), muscle (n = 1), vascular (n = 24), and bone (n = 11). Considerations such as material fidelity, learning outcomes, cost or reusability, validity, and effectiveness are inconsistently reported. Future studies should address design standards for the effective production of synthetic or virtual simulators for limb trauma management.
Collapse
Affiliation(s)
- Leonie Heskin
- From the Department of Surgical Affairs, Royal College of Surgeons in Ireland (L.H., O.T.); University of Limerick (R.G.); and Trinity College Dublin (C.S.), Dublin, Ireland
| | | | | | | |
Collapse
|
45
|
Noyes JA, Carbonneau KJ, Matthew SM. Comparative Effectiveness of Training with Simulators Versus Traditional Instruction in Veterinary Education: Meta-Analysis and Systematic Review. JOURNAL OF VETERINARY MEDICAL EDUCATION 2022; 49:25-38. [PMID: 33891532 DOI: 10.3138/jvme-2020-0026] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
One of the greatest challenges in veterinary education is adequately preparing students with the clinical skills they need to be successful health care providers. Integration of simulators, devices used to represent real world phenomena under test conditions, into the veterinary curriculum can help address challenges to clinical instruction. The use of simulators in veterinary education is increasing; however, their effectiveness remains unclear. This meta-analysis seeks to synthesize the evidence for simulator training in veterinary education to provide a consensus of effect, guide the integration of simulators into the curriculum, and direct the development of future simulation-based research in veterinary medicine. A systematic search identified 416 potential manuscripts from which 60 articles were included after application of inclusion criteria. Information was extracted from 71 independent experiments. The overall weighted mean effect size for simulator training was g = 0.49 for the random-effects model. The outcome measures of knowledge, time, process, and product all produced statistically significant mean effect sizes favoring simulation (d = 0.41, 0.35, 0.70, 0.53, respectively). A moderator analysis revealed that study characteristics and instructional design features moderated the effectiveness of simulator training. Overall, the results indicate that simulator training in veterinary education can be effective for knowledge and clinical skill outcomes.
Collapse
|
46
|
Merged virtual reality teaching of the fundamentals of laparoscopic surgery: a randomized controlled trial. Surg Endosc 2022; 36:6368-6376. [PMID: 34981231 PMCID: PMC8722746 DOI: 10.1007/s00464-021-08939-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2021] [Accepted: 12/06/2021] [Indexed: 11/25/2022]
Abstract
Background The COVID-19 pandemic challenges our ability to provide surgical education, as our ability to gather and train together has been restricted due to safety concerns. However, the importance of quality surgical education has remained. High-fidelity simulation platforms have been developed that merge virtual reality video streams to allow for remote instruction and collaboration. This study sought to validate the use of a merged virtual reality (MVR) platform for the instruction and assessment of the fundamentals of laparoscopic surgery (FLS) skills. Methods This was a prospective randomized controlled non-inferiority study. Thirty participants were randomized between three groups: The standard group received in-person instruction and expert feedback, the experimental group received identical training via the MVR platform, and the control group practiced on their own, but received no feedback. All participants were pre-tested for baseline performance at the beginning of the study. Change in performance was evaluated immediately after training and one month later for retention. Ordinary one-way analysis of variance was used to evaluate the effects of time, group, and time-on-group. Results The pre-test confirmed baseline homogeneity between the groups. MVR was non-inferior to standard in-person training for total FLS times on either the post-test (p = 0.632) or the retention test (p = 0.829). Performance was also identical between MVR and standard training groups for each of the individual FLS tasks. Each group improved significantly in nearly all tasks after practice; however, the standard and MVR training groups both improved significantly more than controls for the ligating loop, extracorporeal suturing, intracorporeal suturing, and total FLS task training but did not reach statistical significance for peg transfer and pattern cut tasks. Conclusion This randomized, controlled trial has demonstrated the use of an MVR platform as non-inferior to in-person instruction for the FLS program, forming the foundation for future work on remote instruction and collaboration. Supplementary Information The online version contains supplementary material available at 10.1007/s00464-021-08939-4.
Collapse
|
47
|
Mitsakos AT, Xanthoudakis E, Irish W, Robey WC, Gilbird RM, Cringan J, Haisch CE. The Resource Costs of Maintaining Learner Utilization of a Simulation Center During the COVID-19 Pandemic. Am Surg 2021:31348211058637. [PMID: 34851174 DOI: 10.1177/00031348211058637] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
BACKGROUND Despite advances in online education during the COVID-19 pandemic, its impact on surgical simulation remains unclear. The aim of this study was to compare the costs and resources required to maintain simulation training in the pandemic and to evaluate how it affected exposure of medical students to simulation during their surgical clerkship. METHODS The number of learners, contact hours, staff hours, and costs were collected from a multi-departmental simulation center of a single academic institution in a retrospective fashion. Utilization and expenditure metrics were compared between the first quarter of academic years 2018-2020. Statistical analysis was performed to evaluate potential differences between overall resource utilization before and during the pandemic, and subgroup analysis was performed for the resources required for the training of the third-year medical students. RESULTS The overall number of learners and contact hours decreased during the first quarter of the academic year 2020 in comparison with 2019 and 2018. However, the staff hours increased. In addition, the costs for PPE increased for the same periods of time. In the subgroup analysis of the third-year medical students, there was an increase in the number of learners, as well as in the staff hours and in the space required to perform the simulation training. DISCUSSION Despite an increase in costs and resources spent on surgical simulation during the pandemic, the utilization by academic entities has remained unaffected. Further studies are required to identify potential solutions to lower simulation resources without a negative impact on the quality of surgical simulation.
Collapse
Affiliation(s)
- Anastasios T Mitsakos
- Department of Surgery, Division of Surgical Education, 12278Brody School of Medicine at East Carolina University, Greenville, NC, USA
| | - Eftechios Xanthoudakis
- Department of Surgery, Division of Surgical Education, 12278Brody School of Medicine at East Carolina University, Greenville, NC, USA
| | - William Irish
- Department of Surgery, Division of Surgical Research, 12278Brody School of Medicine at East Carolina University, Greenville, NC, USA
| | - Walter C Robey
- Department of Emergency Medicine, 12278Brody School of Medicine at East Carolina University, Greenville, NC, USA.,Department of Academic Affairs, Division of Health Sciences, 12278Brody School of Medicine at East Carolina University, Greenville, NC, USA
| | - Rebecca M Gilbird
- Department of Academic Affairs, Division of Health Sciences, 12278Brody School of Medicine at East Carolina University, Greenville, NC, USA
| | - Jessica Cringan
- Department of Academic Affairs, Division of Health Sciences, 12278Brody School of Medicine at East Carolina University, Greenville, NC, USA
| | - Carl E Haisch
- Department of Surgery, Division of Surgical Education, 12278Brody School of Medicine at East Carolina University, Greenville, NC, USA
| |
Collapse
|
48
|
Bube SH, Kingo PS, Madsen MG, Vásquez JL, Norus TP, Olsen RG, Dahl C, Hansen RB, Konge L, Azawi NH. Validation of a novel assessment tool identifying proficiency in Transurethral Bladder Tumour Resection: The OSATURBS assessment tool. J Endourol 2021; 36:572-579. [PMID: 34731011 DOI: 10.1089/end.2021.0768] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Competence in transurethral bladder tumour resection (TURB) is critical in bladder cancer management and should be ensured before independent practice. OBJECTIVE Develop an assessment tool for TURB and explore validity evidence in a clinical context. DESIGN, SETTING, AND PARTICIPANTS July 2019-March 2021, a total of 33 volunteer doctors from three hospitals were included. Participants performed two TURB procedures on patients with bladder tumours. A newly developed assessment tool (OSATURBS) was used for direct observation assessment, self-assessment, and blinded video-assessment. Outcome measurements and statistical analysis: Cronbach's alpha and Pearson's r were calculated for across items internal consistency reliability, inter-rater reliability, and test-retest reliability. Correlation between OSATURBS scores and the operative experience was calculated with Pearson's r and a pass/fail score was established. Differences in assessment scores were explored with paired t-test and independent samples t-test. RESULTS AND LIMITATIONS The internal consistency reliability across items Cronbach's alpha was 0.94 (n = 260, p < 0.001). Inter-rater reliability = 0.80 (n = 64, p < 0.001). Test-retest correlation was high, r = 0.71 (n = 32, p < 0.001). Relation to TURB experience was high, r = 0.71 (n = 32, p < 0.001). Pass/fail score = 19 points. Direct observation assessments were strongly correlated with video ratings (r = 0.85, p < 0.001) but with a significant social bias with lower scores for inexperienced and higher scores for experienced participants. Participants tended to overestimate their own performances. CONCLUSIONS OSATURBS assessment tool for TURB can be used for assessment of surgical proficiency in the clinical setting. Direct observation assessment and self-assessment are biased, and blinded video-assessment of TURB performances is advised.
Collapse
Affiliation(s)
- Sarah Hjartbro Bube
- Zealand University Hospital Roskilde, 53140, Department of Urology, Roskilde, Zealand, Denmark.,University of Copenhagen, 4321, Faculty of Health and Medical Science, Copenhagen, Denmark;
| | | | - Mia Gebauer Madsen
- Aarhus Universitetshospital, 11297, Department of Urology, Aarhus, Denmark;
| | - Juan Luis Vásquez
- Zealand University Hospital Roskilde, 53140, Department of Urology, Roskilde, Zealand, Denmark.,University of Copenhagen, 4321, Faculty of Health and Medical Science, Copenhagen, Zealand, Denmark;
| | - Thomas Peter Norus
- Zealand University Hospital Roskilde, 53140, Department of Urology, Roskilde, Sjaelland, Denmark;
| | - Rikke Groth Olsen
- National Hospital of the Faroe Islands, 112892, Surgical Department, Torshavn, Faroe Islands.,Rigshospitalet, 53146, CAMES - Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark;
| | - Claus Dahl
- Capio Ramsay Santé, Department of Urology, Hellerup, Denmark;
| | - Rikke Bølling Hansen
- Herlev Hospital, 53176, Department of Urology, Gentofte, Denmark.,Rigshospitalet, 53146, CAMES - Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark;
| | - Lars Konge
- Rigshospitalet, 53146, CAMES - Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark.,University of Copenhagen, 4321, Faculty of Health and Medical Science, Copenhagen, Denmark;
| | - Nessn H Azawi
- Zealand University Hospital Roskilde, 53140, Department of Urology, Roskilde, Zealand, Denmark.,University of Copenhagen, 4321, Faculty of Health and Medical Science, Copenhagen, Denmark;
| |
Collapse
|
49
|
Johnson G, Vergis A, Unger B, Park J, Gillman L. Design and Validity Evidence for a Unique Endoscopy Simulator Using a Commercial Video Game. Cureus 2021; 13:e18379. [PMID: 34725623 PMCID: PMC8553393 DOI: 10.7759/cureus.18379] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/29/2021] [Indexed: 11/05/2022] Open
Abstract
Background Procedural simulation enhances early endoscopy training. Multiple commercial simulators are available; however, their application is limited by cost and poor user compliance. First-person "shooter" (FPS) video games are popular. In this study, we aimed to show that a novel in-house designed colonoscope controller used to play an FPS video game shares similar constructs with real-life endoscopy. Methodology Participants completed the first three levels on an FPS video game, Portal (Valve Corporation, Bellevue, WA), first using a conventional controller and then the modified endoscope controller. A total of 12 expert endoscopists and 12 surgical residents with minimal endoscopy experience were evaluated based on completion time, button presses, and hand motion analyses. Results Experts outperformed novices for completion time (expert: 944 seconds; novice: 1,515 seconds; p = 0.006) and hand movements (expert: 1,263.1; novice: 2,052.6; p = 0.004) in using the novel colonoscope controller. There was no difference in button presses or total path length traveled. Furthermore, performance did not differ using conventional game controls. Conclusions Experts outperformed novices using the endoscope but not the conventional controller with respect to the economy of movement and completion time. This result confirms that our endoscope-controlled video game shares similar paradigms with real-life endoscopy and serves as a first step toward creating a more enjoyable and cheaper alternative to commercially available endoscopy simulators.
Collapse
Affiliation(s)
| | | | - Bertram Unger
- Internal Medicine, University of Manitoba, Winnipeg, CAN
| | - Jason Park
- Surgery, University of Manitoba, Winnipeg, CAN
| | | |
Collapse
|
50
|
Abstract
OBJECTIVE This systematic review aims to examine the use of standard-setting methods in the context of simulation-based training of surgical procedures. SUMMARY OF BACKGROUND Simulation-based training is increasingly used in surgical education. However, it is important to determine which level of competency trainees must reach during simulation-based training before operating on patients. Therefore, pass/fail standards must be established using systematic, transparent, and valid methods. METHODS Systematic literature search was done in four databases (Ovid MEDLINE, Embase, Web of Science, and Cochrane Library). Original studies investigating simulation-based assessment of surgical procedures with application of a standard setting were included. Quality of evidence was appraised using GRADE. RESULTS Of 24,299 studies identified by searches, 232 studies met the inclusion criteria. Publications using already established standard settings were excluded (N = 70), resulting in 162 original studies included in the final analyses. Most studies described how the standard setting was determined (N = 147, 91%) and most used the mean or median performance score of experienced surgeons (n = 65, 40%) for standard setting. We found considerable differences across most of the studies regarding study design, set-up, and expert level classification. The studies were appraised as having low and moderate evidence. CONCLUSION Surgical education is shifting towards competency-based education, and simulation-based training is increasingly used for acquiring skills and assessment. Most studies consider and describe how standard settings are established using more or less structured methods but for current and future educational programs, a critical approach is needed so that the learners receive a fair, valid and reliable assessment.
Collapse
|