1
|
Hidding J, Bucher J, Heiliger C, Andrade D, Trupka L, Halmen M, Werner J, Karcz K, Frank A. Laparoscopy training of novices with complex curved instruments using 2D- and 3D-visualization. Langenbecks Arch Surg 2024; 409:109. [PMID: 38570339 PMCID: PMC10990991 DOI: 10.1007/s00423-024-03297-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Accepted: 03/22/2024] [Indexed: 04/05/2024]
Abstract
PURPOSE Beside many advantages, disadvantages such as reduced degrees of freedom and poorer depth perception are still apparent in laparoscopic surgery. 3D visualization and the development of complex instruments are intended to counteract the disadvantages. We want to find out whether the use of complex instruments and 3D visualization has an influence on the performance of novices. METHODS 48 medical students with no experience in laparoscopic surgery or simulator-based laparoscopy training were included. They were randomized in four groups according to a stratification assessment. During a structured training period they completed the FLS-Tasks "PEG Transfer", "Pattern Cut" and "Intracorporeal Suture" and a transfer task based on these three. Two groups used conventional laparoscopic instruments with 3D or 2D visualization, two groups used complex curved instruments. The groups were compared in terms of their performance. RESULTS In 2D laparoscopy there was a better performance with straight instruments vs. curved instruments in PEG Transfer and Intracorporeal Suture. In the transfer task, fewer errors were made with straight instruments. In 2D vs. 3D laparoscopy when using complex curved instruments there was an advantage in Intracorporeal Suture and PEG Transfer for 3D visualization. Regarding the transfer exercise, a better performance was observed and fewer errors were made in 3D group. CONCLUSION We could show that learning laparoscopic techniques with complex curved instruments is more difficult with standard 2D visualization and can be overcome using 3D optics. The use of curved instruments under 3D vision seems to be advantageous when working on more difficult tasks.
Collapse
Affiliation(s)
- Johanna Hidding
- Department of Oral and Maxillofacial Plastic Surgery and Interdisciplinary Department of Oral Surgery and Implantology, University of Cologne, Cologne, Germany
| | - Julian Bucher
- Department of General, Visceral and Transplantation Surgery, LMU University Hospital, Marchioninistr. 15, 81377, Munich, Germany
| | - Christian Heiliger
- Department of General, Visceral and Transplantation Surgery, LMU University Hospital, Marchioninistr. 15, 81377, Munich, Germany
| | - Dorian Andrade
- Department of General, Visceral and Transplantation Surgery, LMU University Hospital, Marchioninistr. 15, 81377, Munich, Germany
| | - Lukas Trupka
- Department of General, Visceral and Transplantation Surgery, LMU University Hospital, Marchioninistr. 15, 81377, Munich, Germany
| | - Matthias Halmen
- Department of General, Visceral and Transplantation Surgery, LMU University Hospital, Marchioninistr. 15, 81377, Munich, Germany
| | - Jens Werner
- Department of General, Visceral and Transplantation Surgery, LMU University Hospital, Marchioninistr. 15, 81377, Munich, Germany
| | - Konrad Karcz
- Department of General, Visceral and Transplantation Surgery, LMU University Hospital, Marchioninistr. 15, 81377, Munich, Germany
| | - Alexander Frank
- Department of General, Visceral and Transplantation Surgery, LMU University Hospital, Marchioninistr. 15, 81377, Munich, Germany.
| |
Collapse
|
2
|
Thinggaard E, Fjørtoft K, Gögenur I, Konge L. Using Self-Rated Examinations to Ensure Competence in Laparoscopy. Simul Healthc 2022; 17:242-248. [PMID: 34652325 DOI: 10.1097/sih.0000000000000618] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
INTRODUCTION Asking participants to rate their own performance during unsupervised training in laparoscopy is reliable and may be cost-effective. The objective of the study was to explore the reliability of self-rated examinations where participants rate their own performance and decide themselves when they have passed tasks in basic laparoscopic skills. METHODS This prospective observational study was conducted at the Copenhagen Academy for Medical Education and Simulation where simulation-based laparoscopic skill training is offered. Here, participants taking part in a basic laparoscopic skills course were asked to rate their own performance and decide when they had passed the Training and Assessment of Basic Laparoscopic Techniques test. To explore reliability, all examinations were video recorded and rated by a blinded rater after the end of the course. RESULTS Thirty-two surgical trainees participated in the course, and 28 completed the study. We found a high reliability when comparing self-rated scores and blinded ratings with an intraclass correlation coefficient of 0.89 ( P < 0.001); self-rated scores compared with blinded ratings were not significantly different (mean = 451 vs. 455, P = 0.28), and the participants did not underestimate nor overestimate their performance. CONCLUSIONS Ratings from self-rated examinations in a basic laparoscopic skills course are reliable, and participants neither underestimate nor overestimate their performance. Self-rated examinations may also be beneficial because they also can offer a cost-effective approach to assessment of surgical trainees.
Collapse
Affiliation(s)
- Ebbe Thinggaard
- From the Department of Gynaecology and Obstetrics (E.T.), Hvidovre Hospital, Hvidovre; Copenhagen Academy for Medical Education and Simulation (E.T., K.F., L.K.), Rigshospitalet; Department of Surgical Gastroenterology (K.F.), Bispebjerg Hospital, Copenhagen; and Department of Surgery (I.G.), Zealand University Hospital, Koege, Denmark
| | | | | | | |
Collapse
|
3
|
Abstract
OBJECTIVE This systematic review aims to examine the use of standard-setting methods in the context of simulation-based training of surgical procedures. SUMMARY OF BACKGROUND Simulation-based training is increasingly used in surgical education. However, it is important to determine which level of competency trainees must reach during simulation-based training before operating on patients. Therefore, pass/fail standards must be established using systematic, transparent, and valid methods. METHODS Systematic literature search was done in four databases (Ovid MEDLINE, Embase, Web of Science, and Cochrane Library). Original studies investigating simulation-based assessment of surgical procedures with application of a standard setting were included. Quality of evidence was appraised using GRADE. RESULTS Of 24,299 studies identified by searches, 232 studies met the inclusion criteria. Publications using already established standard settings were excluded (N = 70), resulting in 162 original studies included in the final analyses. Most studies described how the standard setting was determined (N = 147, 91%) and most used the mean or median performance score of experienced surgeons (n = 65, 40%) for standard setting. We found considerable differences across most of the studies regarding study design, set-up, and expert level classification. The studies were appraised as having low and moderate evidence. CONCLUSION Surgical education is shifting towards competency-based education, and simulation-based training is increasingly used for acquiring skills and assessment. Most studies consider and describe how standard settings are established using more or less structured methods but for current and future educational programs, a critical approach is needed so that the learners receive a fair, valid and reliable assessment.
Collapse
|
4
|
Trujillo Loli Y, D'Carlo Trejo Huamán M, Campos Medina S. Telementoring of in-home real-time laparoscopy using whatsapp messenger: An innovative teaching tool during the COVID-19 pandemic. A cohort study. Ann Med Surg (Lond) 2021; 62:481-484. [PMID: 33552507 PMCID: PMC7846469 DOI: 10.1016/j.amsu.2021.01.085] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2020] [Revised: 01/23/2021] [Accepted: 01/23/2021] [Indexed: 12/18/2022] Open
Abstract
Background COVID-19 pandemic has decreased worldwide surgical activity, thus impairing the training of surgical residents. As a coping strategy, we opted for laparoscopic simulation with endotrainer; however, due to social distancing rules, a face-to-face tutoring training method is unfeasible. In this context telementoring using WhatsApp® Messenger (owned by Facebook Inc., Menlo Park, CA) with a Smartphone aims to train surgical residents in laparoscopy from their homes during the pandemic. Methods Trainers and surgical residents who were at home participated in this study. They used endotrainers with Smartphone cameras and were trained by video calls using WhatsApp. The baseline and final score of intracorporeal knotting was measured, according to the Fundamentals of Laparoscopic Surgery (FLS) methodology, measuring the difference in means with the paired t-test. Recommendations for virtual surgical tutoring were followed and user satisfaction was measured using the Likert scale. Results 40 surgical residents participated. The average distance between the mentor and the trainee was 12.87 km. Average bandwidth was 32.5 Megabits per second (Mbits/s), latency was 424.5 ms (ms), good quality image with immediate feedback was achieved. The statistically significant difference between basal and final measurement was 23.7 points (p < 0.001). All participants found telementoring a good educational tool. Conclusion Smartphone telementoring using WhatsApp® Messenger is an effective educational tool for acquiring laparoscopic skills with a statistically significant difference. This type of training represents a new teaching resource in the light of the recommendations of social distancing during a pandemic.
Collapse
Affiliation(s)
- Yeray Trujillo Loli
- Universidad Nacional Mayor de San Marcos, Av. Grau Block 7, Lima, 15001, Lima, Peru
- Lap Gym Perú Training Center, Jr Zeus 833, Lima, 15719, Lima, Peru
- Corresponding author. Universidad Nacional Mayor de San Marcos, Av. Grau block 7, Lima, 15001, Lima, Peru.
| | - Mario D'Carlo Trejo Huamán
- Universidad Nacional Mayor de San Marcos, Av. Grau Block 7, Lima, 15001, Lima, Peru
- Lap Gym Perú Training Center, Jr Zeus 833, Lima, 15719, Lima, Peru
| | - Stefanie Campos Medina
- Universidad de San Martín de Porres, Alameda del Corregidor 1531, La Molina, 15024, Lima, Peru
| |
Collapse
|
5
|
Soriero D, Atzori G, Barra F, Pertile D, Massobrio A, Conti L, Gusmini D, Epis L, Gallo M, Banchini F, Capelli P, Penza V, Scabini S. Development and Validation of a Homemade, Low-Cost Laparoscopic Simulator for Resident Surgeons (LABOT). INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:ijerph17010323. [PMID: 31906532 PMCID: PMC6981870 DOI: 10.3390/ijerph17010323] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/12/2019] [Revised: 12/20/2019] [Accepted: 12/29/2019] [Indexed: 12/13/2022]
Abstract
Several studies have demonstrated that training with a laparoscopic simulator improves laparoscopic technical skills. We describe how to build a homemade, low-cost laparoscopic training simulator (LABOT) and its validation as a training instrument. First, sixty surgeons filled out a survey characterized by 12 closed-answer questions about realism, ergonomics, and usefulness for surgical training (global scores ranged from 1—very insufficient to 5—very good). The results of the questionnaires showed a mean (±SD) rating score of 4.18 ± 0.65 for all users. Then, 15 students (group S) and 15 residents (group R) completed 3 different tasks (T1, T2, T3), which were repeated twice to evaluate the execution time and the number of users’ procedural errors. For T1, the R group had a lower mean execution time and a lower rate of procedural errors than the S group; for T2, the R and S groups had a similar mean execution time, but the R group had a lower rate of errors; and for T3, the R and S groups had a similar mean execution time and rate of errors. On a second attempt, all the participants tended to improve their results in doing these surgical tasks; nevertheless, after subgroup analysis of the T1 results, the S group had a better improvement of both parameters. Our laparoscopic simulator is simple to build, low-cost, easy to use, and seems to be a suitable resource for improving laparoscopic skills. In the future, further studies should evaluate the potential of this laparoscopic box on long-term surgical training with more complex tasks and simulation attempts.
Collapse
Affiliation(s)
- Domenico Soriero
- OU Oncological Surgery, IRCCS Ospedale Policlinico San Martino, 16132 Genoa, Italy; (D.S.); (D.P.); (A.M.); (L.E.); (S.S.)
| | - Giulia Atzori
- Department of Surgical Sciences and Integrated Methodologies, University of Genoa, 16132 Genoa, Italy;
| | - Fabio Barra
- Academic Unit of Obstetrics and Gynecology, IRCCS Ospedale Policlinico San Martino, 16132 Genoa, Italy
- Correspondence: ; Tel.: +39-3349437959
| | - Davide Pertile
- OU Oncological Surgery, IRCCS Ospedale Policlinico San Martino, 16132 Genoa, Italy; (D.S.); (D.P.); (A.M.); (L.E.); (S.S.)
| | - Andrea Massobrio
- OU Oncological Surgery, IRCCS Ospedale Policlinico San Martino, 16132 Genoa, Italy; (D.S.); (D.P.); (A.M.); (L.E.); (S.S.)
| | - Luigi Conti
- UOC General, Vascular and Thoracic Surgery, G. Da Saliceto Hospital, AUSL, 29121 Piacenza, Italy; (L.C.); (F.B.); (P.C.)
| | - Dario Gusmini
- Association of Architects of Bergamo, 24100 Bergamo, Italy
| | - Lorenzo Epis
- OU Oncological Surgery, IRCCS Ospedale Policlinico San Martino, 16132 Genoa, Italy; (D.S.); (D.P.); (A.M.); (L.E.); (S.S.)
| | - Maurizio Gallo
- Department of Internal Medicine (Di.M.I.), University of Genoa, 16132 Genoa, Italy;
| | - Filippo Banchini
- UOC General, Vascular and Thoracic Surgery, G. Da Saliceto Hospital, AUSL, 29121 Piacenza, Italy; (L.C.); (F.B.); (P.C.)
| | - Patrizio Capelli
- UOC General, Vascular and Thoracic Surgery, G. Da Saliceto Hospital, AUSL, 29121 Piacenza, Italy; (L.C.); (F.B.); (P.C.)
| | - Veronica Penza
- Biomedical Robotics Lab, Advanced Robotics Department, Istituto Italiano di Tecnologia, 16152 Genoa, Italy;
| | - Stefano Scabini
- OU Oncological Surgery, IRCCS Ospedale Policlinico San Martino, 16132 Genoa, Italy; (D.S.); (D.P.); (A.M.); (L.E.); (S.S.)
| |
Collapse
|
6
|
Fjørtoft K, Konge L, Gögenur I, Thinggaard E. The Implementation Gap in Laparoscopic Simulation Training. Scand J Surg 2018; 108:109-116. [PMID: 30207205 DOI: 10.1177/1457496918798201] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
BACKGROUND AND AIMS Simulation-based training in laparoscopy can improve patient safety and efficiency of care, but it depends on how it is used. Research in medical education has moved from demonstrating transferability of simulation training to the operating room to how to best implement it. This study aims to investigate how simulation-based training in laparoscopy has been implemented Scandinavia. MATERIAL AND METHODS An online survey was sent out to medical doctors at surgical, gynecological, and urological departments at 138 hospitals in Denmark, Norway, and Sweden. The questionnaire included questions on respondents' baseline characteristics, opinions, access, and actual use of simulation-based training in laparoscopy. RESULTS In total, 738 respondents completed the survey. Of these, 636 (86.2%) of respondents agreed or strongly agreed that simulation-based training in laparoscopy should be mandatory. A total of 602 (81.6%) had access to simulation-based training in laparoscopy. Of the total 738 respondents, 141 (19.1%) were offered structured training courses, 129 (17.5%) were required to reach a predefined level of competency, and 66 (8.9%) had mandatory courses in laparoscopy. In all, 72 (9.8%) had never used simulation-based training in laparoscopy. CONCLUSION An implementation gap in laparoscopic simulation-based training still exists in Scandinavia. Simulation equipment is generally available, but there is a lack of structured simulation-based training.
Collapse
Affiliation(s)
- K Fjørtoft
- 1 Copenhagen Academy for Medical Education and Simulation-CAMES Rigshospitalet, Copenhagen, Denmark
| | - L Konge
- 1 Copenhagen Academy for Medical Education and Simulation-CAMES Rigshospitalet, Copenhagen, Denmark
| | - I Gögenur
- 2 Center for Surgical Science, Zealand University Hospital, Køge, Denmark
| | - E Thinggaard
- 1 Copenhagen Academy for Medical Education and Simulation-CAMES Rigshospitalet, Copenhagen, Denmark.,2 Center for Surgical Science, Zealand University Hospital, Køge, Denmark
| |
Collapse
|
7
|
Stefan P, Habert S, Winkler A, Lazarovici M, Fürmetz J, Eck U, Navab N. A radiation-free mixed-reality training environment and assessment concept for C-arm-based surgery. Int J Comput Assist Radiol Surg 2018; 13:1335-1344. [PMID: 29943226 DOI: 10.1007/s11548-018-1807-6] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2018] [Accepted: 06/04/2018] [Indexed: 11/29/2022]
Abstract
PURPOSE The discrepancy of continuously decreasing opportunities for clinical training and assessment and the increasing complexity of interventions in surgery has led to the development of different training and assessment options like anatomical models, computer-based simulators or cadaver trainings. However, trainees, following training, assessment and ultimately performing patient treatment, still face a steep learning curve. METHODS To address this problem for C-arm-based surgery, we introduce a realistic radiation-free simulation system that combines patient-based 3D printed anatomy and simulated X-ray imaging using a physical C-arm. To explore the fidelity and usefulness of the proposed mixed-reality system for training and assessment, we conducted a user study with six surgical experts performing a facet joint injection on the simulator. RESULTS In a technical evaluation, we show that our system simulates X-ray images accurately with an RMSE of 1.85 mm compared to real X-ray imaging. The participants expressed agreement with the overall realism of the simulation, the usefulness of the system for assessment and strong agreement with the usefulness of such a mixed-reality system for training of novices and experts. In a quantitative analysis, we furthermore evaluated the suitability of the system for the assessment of surgical skills and gather preliminary evidence for validity. CONCLUSION The proposed mixed-reality simulation system facilitates a transition to C-arm-based surgery and has the potential to complement or even replace large parts of cadaver training, to provide a safe assessment environment and to reduce the risk for errors when proceeding to patient treatment. We propose an assessment concept and outline the steps necessary to expand the system into a test instrument that provides reliable and justified assessments scores indicative of surgical proficiency with sufficient evidence for validity.
Collapse
Affiliation(s)
- Philipp Stefan
- Computer Aided Medical Procedures (CAMP), Technische Universität München, Munich, Germany.
| | - Séverine Habert
- Computer Aided Medical Procedures (CAMP), Technische Universität München, Munich, Germany.
| | - Alexander Winkler
- Computer Aided Medical Procedures (CAMP), Technische Universität München, Munich, Germany
| | - Marc Lazarovici
- Klinikum der Universität München, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Julian Fürmetz
- Klinikum der Universität München, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Ulrich Eck
- Computer Aided Medical Procedures (CAMP), Technische Universität München, Munich, Germany
| | - Nassir Navab
- Computer Aided Medical Procedures (CAMP), Technische Universität München, Munich, Germany.,Computer Aided Medical Procedures, Johns Hopkins University, Baltimore, USA
| |
Collapse
|
8
|
Jørgensen M, Konge L, Subhi Y. Contrasting groups' standard setting for consequences analysis in validity studies: reporting considerations. Adv Simul (Lond) 2018; 3:5. [PMID: 29556423 PMCID: PMC5845294 DOI: 10.1186/s41077-018-0064-7] [Citation(s) in RCA: 55] [Impact Index Per Article: 9.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2017] [Accepted: 02/15/2018] [Indexed: 12/14/2022] Open
Abstract
Background The contrasting groups’ standard setting method is commonly used for consequences analysis in validity studies for performance in medicine and surgery. The method identifies a pass/fail cut-off score, from which it is possible to determine false positives and false negatives based on observed numbers in each group. Since groups in validity studies are often small, e.g., due to a limited number of experts, these analyses are sensitive to outliers on the normal distribution curve. Methods We propose that these shortcomings can be addressed in a simple manner using the cumulative distribution function. Results We demonstrate considerable absolute differences between the observed false positives/negatives and the theoretical false positives/negatives. In addition, several important examples are given. Conclusions We propose that a better reporting strategy is to report theoretical false positives and false negatives together with the observed false positives and negatives, and we have developed an Excel sheet to facilitate such calculations. Trial registration Not relevant. Electronic supplementary material The online version of this article (10.1186/s41077-018-0064-7) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Morten Jørgensen
- 1Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.,2Copenhagen Academy for Medical Education and Simulation, Capital Region of Denmark, Copenhagen, Denmark.,3Department of Ophthalmology, Zealand University Hospital, Vestermarksvej 23, DK-4000 Roskilde, Denmark
| | - Lars Konge
- 1Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.,2Copenhagen Academy for Medical Education and Simulation, Capital Region of Denmark, Copenhagen, Denmark
| | - Yousif Subhi
- 1Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.,3Department of Ophthalmology, Zealand University Hospital, Vestermarksvej 23, DK-4000 Roskilde, Denmark
| |
Collapse
|
9
|
Development of a reliable simulation-based test for diagnostic abdominal ultrasound with a pass/fail standard usable for mastery learning. Eur Radiol 2017; 28:51-57. [PMID: 28677051 DOI: 10.1007/s00330-017-4913-x] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2016] [Revised: 03/13/2017] [Accepted: 05/24/2017] [Indexed: 01/14/2023]
Abstract
BACKGROUND This study aimed to develop a test with validity evidence for abdominal diagnostic ultrasound with a pass/fail-standard to facilitate mastery learning. METHOD The simulator had 150 real-life patient abdominal scans of which 15 cases with 44 findings were selected, representing level 1 from The European Federation of Societies for Ultrasound in Medicine and Biology. Four groups of experience levels were constructed: Novices (medical students), trainees (first-year radiology residents), intermediates (third- to fourth-year radiology residents) and advanced (physicians with ultrasound fellowship). Participants were tested in a standardized setup and scored by two blinded reviewers prior to an item analysis. RESULTS The item analysis excluded 14 diagnoses. Both internal consistency (Cronbach's alpha 0.96) and inter-rater reliability (0.99) were good and there were statistically significant differences (p < 0.001) between all four groups, except the intermediate and advanced groups (p = 1.0). There was a statistically significant correlation between experience and test scores (Pearson's r = 0.82, p < 0.001). The pass/fail-standard failed all novices (no false positives) and passed all advanced (no false negatives). All intermediate participants and six out of 14 trainees passed. CONCLUSION We developed a test for diagnostic abdominal ultrasound with solid validity evidence and a pass/fail-standard without any false-positive or false-negative scores. KEY POINTS • Ultrasound training can benefit from competency-based education based on reliable tests. • This simulation-based test can differentiate between competency levels of ultrasound examiners. • This test is suitable for competency-based education, e.g. mastery learning. • We provide a pass/fail standard without false-negative or false-positive scores.
Collapse
|
10
|
Thinggaard E, Bjerrum F, Strandbygaard J, Gögenur I, Konge L. Ensuring Competency of Novice Laparoscopic Surgeons-Exploring Standard Setting Methods and their Consequences. JOURNAL OF SURGICAL EDUCATION 2016; 73:986-991. [PMID: 27324697 DOI: 10.1016/j.jsurg.2016.05.008] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/12/2016] [Revised: 04/14/2016] [Accepted: 05/16/2016] [Indexed: 06/06/2023]
Abstract
OBJECTIVE Simulation-based assessment tools have been developed to allow for proficiency-based simulator training in laparoscopy. However, few studies have examined the consequences of different standard setting methods or examined what level of proficiency is considered adequate for trainees. The objectives of the present study were to explore the consequences of different standard setting methods and to examine the proficiency level that surgical trainees are expected to reach, before performing supervised surgery on patients. DESIGN Study participants undertook the Training and Assessment of Basic Laparoscopic Techniques test. The tests were video-recorded and rated using a simple scoring system based on number of errors and time. Participants were then asked to assess how high a score a novice should reach before performing supervised surgery on a patient. We then compared 3 methods of standard setting: expert performance level, contrasting groups method, and a modified Angoff method. SETTING The study was conducted at the Copenhagen Academy for Medical Education and Simulation. The academy provides surgical simulation training in laparoscopy for trainees at the hospitals in the Capital Region and the Zealand Region of Denmark. PARTICIPANTS Participants were recruited among surgical trainees in their first year of specialty training from surgery, gynecology, and urology departments. A total of 40 participants were included and completed the trial. RESULTS The different standard setting methods resulted in different pass/fail levels. At the expert performance level, the pass/fail level was 474 points-the contrasting groups method resulted in 358 points and the modified Angoff method resulted in 311 points among experienced surgeons, and 386 points among trainees. The different proficiency levels resulted in a failure rate of 0% to 50% of experienced surgeons and a pass rate of 0% to 25% of novices. Novice laparoscopic surgeons set a higher pass/fail level than experienced surgeons did (p = 0.008). CONCLUSION Required proficiency levels varies depending on the standard setting method used, which highlights the importance of using an established standard setting method to set the pass/fail level.
Collapse
Affiliation(s)
- Ebbe Thinggaard
- Department of Surgery, Roskilde and Koege Hospital, Koege, Denmark; Copenhagen Academy for Medical Education and Simulation, Rigshospitalet, Copenhagen, Denmark.
| | - Flemming Bjerrum
- Department of Surgery, Roskilde and Koege Hospital, Koege, Denmark
| | | | - Ismail Gögenur
- Department of Surgery, Roskilde and Koege Hospital, Koege, Denmark
| | - Lars Konge
- Copenhagen Academy for Medical Education and Simulation, Rigshospitalet, Copenhagen, Denmark
| |
Collapse
|
11
|
Goldenberg MG, Garbens A, Szasz P, Hauer T, Grantcharov TP. Systematic review to establish absolute standards for technical performance in surgery. Br J Surg 2016; 104:13-21. [PMID: 27686465 DOI: 10.1002/bjs.10313] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2016] [Revised: 05/12/2016] [Accepted: 08/10/2016] [Indexed: 01/01/2023]
Abstract
BACKGROUND Standard setting allows educators to create benchmarks that distinguish between those who pass and those who fail an assessment. It can also be used to create standards in clinical and simulated procedural skill. The objective of this review was to perform a systematic review of the literature using absolute standard-setting methodology to create benchmarks in technical performance. METHODS A systematic review was conducted by searching MEDLINE, Embase, PsycINFO and the Cochrane Database of Systematic Reviews. Abstracts of retrieved studies were reviewed and those meeting the inclusion criteria were selected for full-text review. The quality of evidence presented in the included studies was assessed using the Medical Education Research Study Quality Instrument (MERSQI), where a score of 14 or more of 18 indicates high-quality evidence. RESULTS Of 1809 studies identified, 37 used standard-setting methodology for assessment of procedural skill. Of these, 24 used participant-centred and 13 employed item-centred methods. Thirty studies took place in a simulated environment, and seven in a clinical setting. The included studies assessed residents (26 of 37), fellows (6 of 37) and staff physicians (17 of 37). Seventeen articles achieved a MERSQI score of 14 or more of 18, whereas 20 did not meet this mark. CONCLUSION Absolute standard-setting methodologies can be used to establish cut-offs for procedural skill assessments.
Collapse
Affiliation(s)
- M G Goldenberg
- Department of Surgery, University of Toronto, Toronto, Canada
| | - A Garbens
- Department of Surgery, University of Toronto, Toronto, Canada
| | - P Szasz
- Department of Surgery, University of Toronto, Toronto, Canada
| | - T Hauer
- Department of Surgery, University of Toronto, Toronto, Canada
| | - T P Grantcharov
- Department of Surgery, University of Toronto, Toronto, Canada
| |
Collapse
|
12
|
Thinggaard E, Konge L, Bjerrum F, Strandbygaard J, Gögenur I, Spanager L. Take-home training in a simulation-based laparoscopy course. Surg Endosc 2016; 31:1738-1745. [PMID: 27515838 DOI: 10.1007/s00464-016-5166-5] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2016] [Accepted: 07/30/2016] [Indexed: 01/11/2023]
Abstract
BACKGROUND Simulation training can prepare trainees for clinical practice in laparoscopic surgery. Training on box trainers allows for simulation training at home, which studies have shown to be a feasible method of training. However, little research has been conducted into how to make it a more efficient method of training. Our aim was to investigate how box trainers are used in take-home training to help guide the design of take-home training courses. METHODS This study was designed using a mixed methods approach. Junior doctors participating in a laparoscopy curriculum, which included practising at home on box trainers, were invited. Quantitative data on training patterns was collected from logbooks. Qualitative data on the use of box trainers was retrieved from focus groups and individual interviews. RESULTS From logbooks, we found that 14 out of 18 junior doctors mixed their training modalities, and four practised first on box trainers then on virtual reality simulators. Twelve practised only at home, while five practised at both places and one practised solely at the simulation centre. After a delayed start, most practised for some time, then had a period without training and then started training again towards the end of the course. We found that the themes of the interviews were: training method, training pattern, feedback and self-regulation. Participants identified the lack of feedback as challenging but described how self-rating provided direction during unsupervised training. Mandatory training elements affected when and how much participants practised. CONCLUSIONS When participants practised at home, they took an individualised approach to training. They mixed their training at home with training at the simulation centre. Participants practised at the beginning and towards the end of the course. Self-rating helped to guide unsupervised training where feedback was not accessible. Curricular requirements and testing determined when and how much participants practised.
Collapse
Affiliation(s)
- Ebbe Thinggaard
- Copenhagen Academy for Medical Education and Simulation, Capital Region Denmark, Copenhagen, Denmark.
- Department of Surgery, Zealand University Hospital, Lykkebaekvej 1, 4600, Koege, Denmark.
| | - Lars Konge
- Copenhagen Academy for Medical Education and Simulation, Capital Region Denmark, Copenhagen, Denmark
| | - Flemming Bjerrum
- Department of Surgery, Zealand University Hospital, Lykkebaekvej 1, 4600, Koege, Denmark
| | | | - Ismail Gögenur
- Department of Surgery, Zealand University Hospital, Lykkebaekvej 1, 4600, Koege, Denmark
| | - Lene Spanager
- Department of Surgery, Nordsjaellands Hospital, Hilleroed, Denmark
| |
Collapse
|
13
|
Sørensen SMD, Mahmood O, Konge L, Thinggaard E, Bjerrum F. Laser visual guidance versus two-dimensional vision in laparoscopy: a randomized trial. Surg Endosc 2016; 31:112-118. [PMID: 27317036 PMCID: PMC5216078 DOI: 10.1007/s00464-016-4937-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2015] [Accepted: 04/09/2016] [Indexed: 12/04/2022]
Abstract
Background During laparoscopy, the surgeon’s loss of depth perception and spatial orientation is problematic. Laser visual guidance (LVG) is an innovative technology that improves depth perception to enhance the visual field. In this trial, we examined the effect of LVG on surgical novices’ motor skills, quality of task performance, and cognitive workload. Methods We designed a randomized controlled trial following the CONSORT statement. Thirty-two surgical novices completed the Training and Assessment of Basic Laparoscopic Techniques (TABLT) test. The first attempt allowed participants to familiarize themselves with the exercises. We then randomized the participants, and they completed a test session using either LVG or conventional two-dimensional vision. Results We found no significant difference between using the LVG tool and conventional 2D vision; however, both the mean completion time and movements used were less in the LVG group: Mean time used in the LVG group was 1288 s (95 % CI 1188–1388) versus 1354 s (95 % CI 1190–1518) (p = 0.45); mean angular path length used in the LVG group was 24,049° (95 % CI 20,761–27,336) versus 26,014° (95 % CI 22,059–29,970) (p = 0.42); mean path length in the LVG group was 4560 cm (95 % CI 3971–5,149 cm) versus 5062 cm (95 % CI 4328–5797), (p = 0.26). Moreover, the mean TABLT performance score was higher in the LVG group compared with the 2D group, although not significant: 379 (95 % CI 352–405) versus 338 (95 % CI 288–387) (p = 0.14). No significant difference was found between the groups’ cognitive workloads. Conclusion We found no significant improvement of laparoscopic motor skills when using LVG, although a tendency toward improved performance was seen. LVG could have the potential to help novice surgeons acquire basic laparoscopic; however, further development of the concept and validation is needed to confirm this.
Collapse
Affiliation(s)
- Stine Maya Dreier Sørensen
- Copenhagen Academy for Medical Education and Simulation, University of Copenhagen, Blegdamsvej 9, 2100, Copenhagen, Capital Region of Denmark, Denmark. .,University of Copenhagen, Copenhagen, Denmark.
| | - Oria Mahmood
- Copenhagen Academy for Medical Education and Simulation, University of Copenhagen, Blegdamsvej 9, 2100, Copenhagen, Capital Region of Denmark, Denmark.,University of Copenhagen, Copenhagen, Denmark
| | - Lars Konge
- Copenhagen Academy for Medical Education and Simulation, University of Copenhagen, Blegdamsvej 9, 2100, Copenhagen, Capital Region of Denmark, Denmark.,University of Copenhagen, Copenhagen, Denmark
| | - Ebbe Thinggaard
- Copenhagen Academy for Medical Education and Simulation, University of Copenhagen, Blegdamsvej 9, 2100, Copenhagen, Capital Region of Denmark, Denmark.,University of Copenhagen, Copenhagen, Denmark.,Department of Surgery, Roskilde and Koege Hospital, Koege, Denmark
| | - Flemming Bjerrum
- University of Copenhagen, Copenhagen, Denmark.,Department of Surgery, Roskilde and Koege Hospital, Koege, Denmark
| |
Collapse
|
14
|
Thinggaard E, Kleif J, Bjerrum F, Strandbygaard J, Gögenur I, Matthew Ritter E, Konge L. Off-site training of laparoscopic skills, a scoping review using a thematic analysis. Surg Endosc 2016; 30:4733-4741. [DOI: 10.1007/s00464-016-4834-9] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2015] [Accepted: 02/15/2016] [Indexed: 01/22/2023]
|