1
|
Ingram I, Hughes MA. The Effect of External Distractions on Simulated Laparoscopic Performance. J Laparoendosc Adv Surg Tech A 2024. [PMID: 39263976 DOI: 10.1089/lap.2024.0280] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/13/2024] Open
Abstract
Background: Surgeons are exposed to unavoidable distractions while operating. Distractions can cause stress, and stress can impair cognition and motor skills to the detriment of surgeon performance. This study assessed the impact of distractions on surgeon stress levels and surgeon performance during simulated laparoscopic tasks. Methods: Twelve surgical trainees completed a peg-thread transfer task three times on a laparoscopic simulator in a neutral environment. Six trainees then completed the task three more times in an environment overlain with distractions commonly found in the operating theatre. Six others completed three more tasks in a neutral environment. Stress was measured by recording heart rate and by assessing responses to the NASA Task Load Index. Performance was measured using instrument tracking metrics and time to complete the task. Results: The distracted participants reported significantly higher effort, frustration, and physical demand than the neutral group, as measured by the NASA Task Load Index, (P = .001, .031, and .044, respectively). The neutral group completed their final task significantly faster compared with baseline (P = .049), while the distracted group failed to show this improvement. The distracted group showed higher hand discordance, suggesting reduced ambidexterity. Conclusion: Distraction negatively impacted some aspects of performance and resulted in subjective increased stress. In future, simulation may have a role in "stress inoculation," enabling surgeons to maintain performance, despite distractions.
Collapse
Affiliation(s)
- Isabel Ingram
- Edinburgh Medical School, University of Edinburgh, Edinburgh, United Kingdom
| | - Mark A Hughes
- Department of Clinical Neurosciences, Royal Infirmary of Edinburgh, Edinburgh, United Kingdom
| |
Collapse
|
2
|
Murakami M, Nishida N, Nagano A, Sugita K, Yano K, Harumatsu T, Onishi S, Yamada K, Yamada W, Kawano T, Muto M, Ieiri S. Evaluation of skill acquisition characteristics depending on the size of a dry box. MINIM INVASIV THER 2024; 33:224-231. [PMID: 38400835 DOI: 10.1080/13645706.2024.2321950] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Accepted: 01/31/2024] [Indexed: 02/26/2024]
Abstract
BACKGROUND Few studies have analyzed the effect of the size of the working space in training on the acquisition of endoscopic skills. In this study, adult- and infant-sized dry boxes (DBs) were used to verify how the size of the working space in training affects forceps manipulation and learning curve. MATERIAL AND METHODS Seventy-two medical students were enrolled. The task was peg transfer. The training environment was divided into adult- and infant-sized DBs. Skill evaluations were also divided into adult- and infant-sized DBs (four groups in total). The forceps manipulation characteristics and task completion time were compared before and after training. RESULTS Regarding skill evaluations using adult-sized DBs, there were no significant differences between the infant- and adult-sized DB-trained groups. Regarding skill evaluations using infant-sized DBs, there were no significant differences between the groups before training. After training, there was no significant difference in the total path length or average acceleration of the forceps between the groups. However, the infant-sized DB-trained group had a significantly faster average forceps velocity and faster task completion time than the adult-sized DB-trained group. CONCLUSION Training with a small DB is more efficient in acquiring smoother and faster forceps manipulation in a small working space.
Collapse
Affiliation(s)
- Masakazu Murakami
- Department of Pediatric Surgery, Research Field in Medical and Health Sciences, Medical and Dental Area, Research and Education Assembly, Kagoshima University, Kagoshima, Japan
| | - Nanako Nishida
- Department of Pediatric Surgery, Research Field in Medical and Health Sciences, Medical and Dental Area, Research and Education Assembly, Kagoshima University, Kagoshima, Japan
| | - Ayaka Nagano
- Department of Pediatric Surgery, Research Field in Medical and Health Sciences, Medical and Dental Area, Research and Education Assembly, Kagoshima University, Kagoshima, Japan
| | - Koshiro Sugita
- Department of Pediatric Surgery, Research Field in Medical and Health Sciences, Medical and Dental Area, Research and Education Assembly, Kagoshima University, Kagoshima, Japan
| | - Keisuke Yano
- Department of Pediatric Surgery, Research Field in Medical and Health Sciences, Medical and Dental Area, Research and Education Assembly, Kagoshima University, Kagoshima, Japan
| | - Toshio Harumatsu
- Department of Pediatric Surgery, Research Field in Medical and Health Sciences, Medical and Dental Area, Research and Education Assembly, Kagoshima University, Kagoshima, Japan
| | - Shun Onishi
- Department of Pediatric Surgery, Research Field in Medical and Health Sciences, Medical and Dental Area, Research and Education Assembly, Kagoshima University, Kagoshima, Japan
| | - Koji Yamada
- Department of Pediatric Surgery, Research Field in Medical and Health Sciences, Medical and Dental Area, Research and Education Assembly, Kagoshima University, Kagoshima, Japan
| | - Waka Yamada
- Department of Pediatric Surgery, Research Field in Medical and Health Sciences, Medical and Dental Area, Research and Education Assembly, Kagoshima University, Kagoshima, Japan
| | - Takafumi Kawano
- Department of Pediatric Surgery, Research Field in Medical and Health Sciences, Medical and Dental Area, Research and Education Assembly, Kagoshima University, Kagoshima, Japan
| | - Mitsuru Muto
- Department of Pediatric Surgery, Research Field in Medical and Health Sciences, Medical and Dental Area, Research and Education Assembly, Kagoshima University, Kagoshima, Japan
| | - Satoshi Ieiri
- Department of Pediatric Surgery, Research Field in Medical and Health Sciences, Medical and Dental Area, Research and Education Assembly, Kagoshima University, Kagoshima, Japan
| |
Collapse
|
3
|
Simons MC, Hunt JA, Anderson SL. Trained to cut? A literature review of veterinary surgical resident training. Vet Surg 2024; 53:791-799. [PMID: 38816998 DOI: 10.1111/vsu.14110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Revised: 04/04/2024] [Accepted: 04/30/2024] [Indexed: 06/01/2024]
Abstract
A surgical residency trains veterinary graduates to a higher level of expertise in surgical procedures than is possible during veterinary school and prepares a resident to pursue board certification in surgery. The education of veterinary surgical residents has changed minimally since its inception in the twentieth century, and there are insufficient studies to determine if residency programs are producing surgeons with competence in each of the necessary procedural categories. The aims of this review were to report the current theory and methods used to provide surgical education to residents, to discuss the training most likely to create a competent, board-certified surgeon and to review assessment methods used during training. Several literature searches using broad terms such as "veterinary surgery residency," "veterinary surgery resident," and "veterinary surgical training" were performed using PubMed, CAB abstracts, and Google Scholar. Literature pertinent to theory, methods, training, and assessment of veterinary surgical residents was included. The reviewed literature demonstrated the need for research-based learning curves for specific procedures. Simulation training is known to facilitate deliberate practice and should be leveraged where possible to reach competency. The creation of validated assessment methods should be pursued as it enables assessment of competency instead of inferring its development from case logs. Understanding and supporting learner cognition and providing sufficient feedback remain important issues in the field. Surgical educators are urged to continue to search for innovative and evidence-based ways to train competent surgical residents.
Collapse
Affiliation(s)
- Micha C Simons
- Department of Small Animal Clinical Sciences, Virginia Maryland Regional College of Veterinary Medicine, Blacksburg, Virginia, USA
| | - Julie A Hunt
- Department of Clinical Sciences, Lincoln Memorial University Richard A. Gillespie College of Veterinary Medicine, Harrogate, Tennessee, USA
| | - Stacy L Anderson
- Department of Clinical Sciences, Lincoln Memorial University Richard A. Gillespie College of Veterinary Medicine, Harrogate, Tennessee, USA
| |
Collapse
|
4
|
de Laat JM, van der Horst-Schrivers AN, Appelman-Dijkstra NM, Bisschop PH, Dreijerink KM, Drent ML, van de Klauw MM, de Ranitz WL, Stades AM, Stikkelbroeck NM, Timmers HJ, ten Cate O. Assessment of Entrustable Professional Activities Among Dutch Endocrine Supervisors. JOURNAL OF CME 2024; 13:2360137. [PMID: 38831939 PMCID: PMC11146265 DOI: 10.1080/28338073.2024.2360137] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/04/2024] [Accepted: 05/19/2024] [Indexed: 06/05/2024]
Abstract
Entrustable Professional Activities (EPAs) are an important tool to support individualisation of medical training in a competency-based setting and are increasingly implemented in the clinical speciality training for endocrinologist. This study aims to assess interrater agreement and factors that potentially impact EPA scores. Five known factors that affect entrustment decisions in health profesions training (capability, integrity, reliability, humility, agency) were used in this study. A case-vignette study using standardised written cases. Case vignettes (n = 6) on the topics thyroid disease, pituitary disease, adrenal disease, calcium and bone disorders, diabetes mellitus, and gonadal disorders were written by two endocrinologists and a medical education expert and assessed by endocrinologists experienced in the supervision of residents in training. Primary outcome is the inter-rater agreement of entrustment decisions for endocrine EPAs among raters. Secondary outcomes included the dichotomous interrater agreement (entrusted vs. non-entrusted), and an exploration of factors that impact decision-making. The study protocol was registered and approved by the Ethical Review Board of the Netherlands Association for Medical Education (NVMO-ERB # 2020.2.5). Nine endocrinologists from six different academic regions participated. Overall, the Fleiss Kappa measure of agreement for the EPA level was 0.11 (95% CI: 0.03-0.22) and for the entrustment decision 0.24 (95% CI 0.11-0.37). Of the five features that impacted the entrustment decision, capability was ranked as the most important by a majority of raters (56%-67%) in every case. There is a considerable discrepancy between the EPA levels assigned by different raters. These findings emphasise the need to base entrustment decisions on multiple observations, made by a team of supervisors and enriched with factors other than direct medical competence.
Collapse
Affiliation(s)
- Joanne M. de Laat
- Department of Internal Medicine, Division of Endocrinology, Radboud University Medical Center, Nijmegen, The Netherlands
- Utrecht Center for Research and Development of Health Professions Education, University Medical Center Utrecht, Utrecht, The Netherlands
| | | | | | - Peter H. Bisschop
- Department of Endocrinology and Metabolism, Amsterdam UMC, Location Academic Medical Center, Amsterdam, The Netherlands
| | - Koen M.A. Dreijerink
- Department of Internal Medicine, Amsterdam UMC, Location VU University Medical Center, Amsterdam, The Netherlands
| | - Madeleine L. Drent
- Department of Internal Medicine, Amsterdam UMC, Location VU University Medical Center, Amsterdam, The Netherlands
| | - Melanie M. van de Klauw
- Department of Endocrinology, University Medical Center Groningen, Groningen, The Netherlands
| | - Wendela L. de Ranitz
- Department of Endocrinology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Aline M.E. Stades
- Department of Endocrinology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Nike M.M.L. Stikkelbroeck
- Department of Internal Medicine, Division of Endocrinology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Henri J.L.M. Timmers
- Department of Internal Medicine, Division of Endocrinology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Olle ten Cate
- Utrecht Center for Research and Development of Health Professions Education, University Medical Center Utrecht, Utrecht, The Netherlands
| |
Collapse
|
5
|
Stan C, Ujvary PL, Blebea C, Tănase MI, Tănase M, Pop SS, Maniu AA, Cosgarea M, Rădeanu DG. Hand Motion Analysis Using Accelerometer-Based Sensors and Sheep's Head Model for Basic Training in Functional Endoscopic Sinus Surgery. Cureus 2024; 16:e59725. [PMID: 38841010 PMCID: PMC11151713 DOI: 10.7759/cureus.59725] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/06/2024] [Indexed: 06/07/2024] Open
Abstract
INTRODUCTION Motion analysis, the study of movement patterns to evaluate performance, plays a crucial role in surgical training. It provides objective data that can be used to assess and improve trainee's precision, efficiency, and overall surgical technique. The primary aim of this study is to employ accelerometer-based sensors placed on the wrist to analyze hand motions during endoscopic sinus surgery training using the sheep's head. By capturing detailed movement data, the study seeks to quantify the motion characteristics that distinguish different levels of surgical expertise. This approach seeks to quantify motion characteristics indicative of surgical expertise and enhance the objectivity and effectiveness of surgical training feedback mechanisms. MATERIALS AND METHODS Twenty-four participants were divided into three groups based on their experience with endoscopic endonasal surgery. Each participant was tasked with performing specified procedures on an individual sheep's head, concentrating on exploring both nasal passages. A single Bluetooth Accelerometer WitMotion sensor was mounted on the dorsal surface of each hand. This facilitates the evaluation of efficiency parameters such as time, path length, and acceleration during the training procedures. Accelerometer data were collected and imported in CSV format (comma-separated values) for each group of surgeons-senior, specialist, and resident-mean values and standard deviations were computed. The Shapiro-Wilk Test assessed the normality of the distribution. The Kruskal-Wallis test was employed to compare procedural time, acceleration, and path length differences across the three surgeon experience levels. RESULTS For the procedural time, statistical significance appears in all surgical steps (p<0.001), with the biggest difference in the septoplasty group in favor of the senior group. A clear difference can be observed between the resulting acceleration of the dominant hands (instrument hand) and the non-dominant hand (endoscopic hand) and between the study groups. The difference between groups reaches statistical significance with a p-value <0.001. A statistically significant difference can be seen between the paths covered by each hand of every participant (p<0.001). Also, senior doctors covered significantly less movement with both hands than the specialists and the resident doctors (p<0.001). CONCLUSIONS The data show a clear learning curve from resident to senior, with residents taking more time and using more hand movements to complete the same tasks. Specialists are in the intermediate phase, showing signs of honing their technique towards efficiency. This comprehensive data set can help tailor training programs to focus on both efficiency (quicker procedures) and economy of motion (reduced path length and acceleration), especially in more complex procedures where the difference in performance is more pronounced.
Collapse
Affiliation(s)
- Constantin Stan
- Otolaryngology, Iuliu Hațieganu University of Medicine and Pharmacy, Cluj-Napoca, ROU
- Surgical Clinical, Faculty of Medicine and Pharmacy, "Dunarea de Jos" University of Galati, Galati, ROU
| | - Peter L Ujvary
- Otolaryngology, Iuliu Hațieganu University of Medicine and Pharmacy, Cluj Napoca, ROU
| | - Cristina Blebea
- Otolaryngology, Iuliu Hațieganu University of Medicine and Pharmacy, Cluj-Napoca, ROU
| | - Mihai I Tănase
- Otolaryngology, Iuliu Hațieganu University of Medicine and Pharmacy, Cluj-Napoca, ROU
| | - Mara Tănase
- Otolaryngology, Iuliu Hațieganu University of Medicine and Pharmacy, Cluj-Napoca, ROU
| | - Septimiu Sever Pop
- Otolaryngology, Iuliu Hațieganu University of Medicine and Pharmacy, Cluj-Napoca, ROU
| | - Alma A Maniu
- Otolaryngology, Iuliu Hațieganu University of Medicine and Pharmacy, Cluj-Napoca, ROU
| | - Marcel Cosgarea
- Otolaryngology, Iuliu Hațieganu University of Medicine and Pharmacy, Cluj-Napoca, ROU
| | - Doinel G Rădeanu
- Otolaryngology, Iuliu Hațieganu University of Medicine and Pharmacy, Cluj-Napoca, ROU
| |
Collapse
|
6
|
Chen Z, Yang D, Li A, Sun L, Zhao J, Liu J, Liu L, Zhou X, Chen Y, Cai Y, Wu Z, Cheng K, Cai H, Tang M, Peng B, Wang X. Decoding surgical skill: an objective and efficient algorithm for surgical skill classification based on surgical gesture features -experimental studies. Int J Surg 2024; 110:1441-1449. [PMID: 38079605 PMCID: PMC10942222 DOI: 10.1097/js9.0000000000000975] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2023] [Accepted: 11/21/2023] [Indexed: 03/16/2024]
Abstract
BACKGROUND Various surgical skills lead to differences in patient outcomes and identifying poorly skilled surgeons with constructive feedback contributes to surgical quality improvement. The aim of the study was to develop an algorithm for evaluating surgical skills in laparoscopic cholecystectomy based on the features of elementary functional surgical gestures (Surgestures). MATERIALS AND METHODS Seventy-five laparoscopic cholecystectomy videos were collected from 33 surgeons in five hospitals. The phase of mobilization hepatocystic triangle and gallbladder dissection from the liver bed of each video were annotated with 14 Surgestures. The videos were grouped into competent and incompetent based on the quantiles of modified global operative assessment of laparoscopic skills (mGOALS). Surgeon-related information, clinical data, and intraoperative events were analyzed. Sixty-three Surgesture features were extracted to develop the surgical skill classification algorithm. The area under the receiver operating characteristic curve of the classification and the top features were evaluated. RESULTS Correlation analysis revealed that most perioperative factors had no significant correlation with mGOALS scores. The incompetent group has a higher probability of cholecystic vascular injury compared to the competent group (30.8 vs 6.1%, P =0.004). The competent group demonstrated fewer inefficient Surgestures, lower shift frequency, and a larger dissection-exposure ratio of Surgestures during the procedure. The area under the receiver operating characteristic curve of the classification algorithm achieved 0.866. Different Surgesture features contributed variably to overall performance and specific skill items. CONCLUSION The computer algorithm accurately classified surgeons with different skill levels using objective Surgesture features, adding insight into designing automatic laparoscopic surgical skill assessment tools with technical feedback.
Collapse
Affiliation(s)
- Zixin Chen
- Department of General Surgery, Division of Pancreatic Surgery
- West China School of Medicine, West China Hospital of Sichuan University
| | - Dewei Yang
- Chongqing University of Posts and Telecommunications, School of Advanced Manufacturing Engineering, Chongqing
| | - Ang Li
- Department of General Surgery, Division of Pancreatic Surgery
- Guang’an People’s Hospital, Guang’an
| | - Louzong Sun
- Department of Hepatobiliary Surgery, Zigong First People’s Hospital, Zigong
| | - Jifan Zhao
- Chengdu Withai Innovations Technology Company, Chengdu
| | - Jie Liu
- Chengdu Withai Innovations Technology Company, Chengdu
| | - Linxun Liu
- Department of General Surgery, Qinghai Provincial People’s Hospital, Xining, People’s Republic of China
| | - Xiaobo Zhou
- School of Biomedical Informatics, McGovern Medical School, University of Texas Health Science Center, Houston, USA
| | - Yonghua Chen
- Department of General Surgery, Division of Pancreatic Surgery
| | - Yunqiang Cai
- Department of General Surgery, Division of Pancreatic Surgery
| | - Zhong Wu
- Department of General Surgery, Division of Pancreatic Surgery
| | - Ke Cheng
- Department of General Surgery, Division of Pancreatic Surgery
| | - He Cai
- Department of General Surgery, Division of Pancreatic Surgery
| | - Ming Tang
- Department of General Surgery, Division of Pancreatic Surgery
- West China School of Medicine, West China Hospital of Sichuan University
| | - Bing Peng
- Department of General Surgery, Division of Pancreatic Surgery
| | - Xin Wang
- Department of General Surgery, Division of Pancreatic Surgery
| |
Collapse
|
7
|
Salame G, Holden M, Lucas BP, Portillo A. Change in economy of ultrasound probe motion among general medicine trainees. Ultrasound J 2024; 16:5. [PMID: 38289444 PMCID: PMC10828286 DOI: 10.1186/s13089-023-00345-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Accepted: 11/07/2023] [Indexed: 02/02/2024] Open
Abstract
OBJECTIVES To observe change in economy of 9 ultrasound probe movement metrics among internal medicine trainees during a 5-day training course in cardiac point of care ultrasound (POCUS). METHODS We used a novel probe tracking device to record nine features of ultrasound probe movement, while trainees and experts optimized ultrasound clips on the same volunteer patients. These features included translational movements, gyroscopic movements (titling, rocking, and rotation), smoothness, total path length, and scanning time. We determined the adjusted difference between each trainee's movements and the mean value of the experts' movements for each patient. We then used a mixed effects model to trend average the adjusted differences between trainees and experts throughout the 5 days of the course. RESULTS Fifteen trainees were enrolled. Three echocardiographer technicians and the course director served as experts. Across 16 unique patients, 294 ultrasound clips were acquired. For all 9 movements, the adjusted difference between trainees and experts narrowed day-to-day (p value < 0.05), suggesting ongoing improvement during training. By the last day of the course, there were no statistically significant differences between trainees and experts in translational movement, gyroscopic movement, smoothness, or total path length; yet on average trainees took 28 s (95% CI [14.7-40.3] seconds) more to acquire a clip. CONCLUSIONS We detected improved ultrasound probe motion economy among internal medicine trainees during a 5-day training course in cardiac POCUS using an inexpensive probe tracking device. Objectively quantifying probe motion economy may help assess a trainee's level of proficiency in this skill and individualize their POCUS training.
Collapse
Affiliation(s)
- Gerard Salame
- Department of Medicine, Saint Joseph Hospital/SCL Health, 1375 E 19th Ave, Denver, CO, 80218, USA.
| | - Matthew Holden
- School of Computer Science, Carleton University, Ottawa, ON, Canada
| | - Brian P Lucas
- Medicine Service, White River Junction VA Medical Center, White River Junction, Vermont, USA
- The Dartmouth Institute of Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth College, Hanover, NH, USA
- Geisel School of Medicine at Dartmouth College, Hanover, NH, USA
| | | |
Collapse
|
8
|
Buote NJ, Fransson B, Rishniw M. Comparison of Attempts Needed for Veterinary Students to Reach Proficiency in a Basic and Advanced Robotic Simulator Task. JOURNAL OF VETERINARY MEDICAL EDUCATION 2023:e20220130. [PMID: 36917585 DOI: 10.3138/jvme-2022-0130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
Surgical training of students is one of the most difficult aspects of veterinary medical education. Competing interests of house officers, owner's wishes, and ethical concerns have led to increasing use of inanimate models for basic surgical skills training. Robotic simulators have benefits for psychomotor training but have not been previously investigated in veterinary medicine. Understanding the necessary practice time for new training devices is important when determining their potential value. The aim of this study was to compare the number of attempts needed for veterinary students to reach proficiency in both a basic and advanced robotic simulator task, and to assess the predictive nature of performance variables. Each student performed a basic and advanced tasks on the Mimic dV-Trainer™ until proficiency was reached. Students required a median of eight attempts (95% CI = 7-8, range: 6-11) to reach proficiency for the basic task versus 22 attempts (95% CI = 20-26, range: 11-62) for the advanced task. The median time required to complete training for the basic and advanced task was 13.5 minutes (range: 8-24 minutes) and 26.5 minutes (range: 11-82 minutes) respectively. The difference in task attempts supports the training protocol and confirms proficiency can be attained in a short period of time. The number of attempts to reach proficiency correlated with specific performance variables which can be used by educators to aid in training students on a robotic simulator. Continued investigations on robotic simulators should be performed to investigate their use in improving psychomotor skills in veterinary students.
Collapse
Affiliation(s)
- Nicole J Buote
- ACVS Founding Fellow Minimally Invasive Surgery (Soft Tissue), Department of Clinical Sciences, College of Veterinary Medicine, Cornell University, Ithaca, NY 14853 USA
| | - Boel Fransson
- ACVS Founding Fellow Minimally Invasive Surgery (Soft Tissue), Department of Clinical Sciences, Washington State University, Pullman, WA 99164-6610
| | - Mark Rishniw
- Department of Clinical Sciences, College of Veterinary Medicine, Cornell University, Ithaca, NY 14853 USA
| |
Collapse
|
9
|
Kulkarni CS, Deng S, Wang T, Hartman-Kenzler J, Barnes LE, Parker SH, Safford SD, Lau N. Scene-dependent, feedforward eye gaze metrics can differentiate technical skill levels of trainees in laparoscopic surgery. Surg Endosc 2023; 37:1569-1580. [PMID: 36123548 PMCID: PMC11062149 DOI: 10.1007/s00464-022-09582-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2022] [Accepted: 08/25/2022] [Indexed: 10/14/2022]
Abstract
INTRODUCTION In laparoscopic surgery, looking in the target areas is an indicator of proficiency. However, gaze behaviors revealing feedforward control (i.e., looking ahead) and their importance have been under-investigated in surgery. This study aims to establish the sensitivity and relative importance of different scene-dependent gaze and motion metrics for estimating trainee proficiency levels in surgical skills. METHODS Medical students performed the Fundamentals of Laparoscopic Surgery peg transfer task while recording their gaze on the monitor and tool activities inside the trainer box. Using computer vision and fixation algorithms, five scene-dependent gaze metrics and one tool speed metric were computed for 499 practice trials. Cluster analysis on the six metrics was used to group the trials into different clusters/proficiency levels, and ANOVAs were conducted to test differences between proficiency levels. A Random Forest model was trained to study metric importance at predicting proficiency levels. RESULTS Three clusters were identified, corresponding to three proficiency levels. The correspondence between the clusters and proficiency levels was confirmed by differences between completion times (F2,488 = 38.94, p < .001). Further, ANOVAs revealed significant differences between the three levels for all six metrics. The Random Forest model predicted proficiency level with 99% out-of-bag accuracy and revealed that scene-dependent gaze metrics reflecting feedforward behaviors were more important for prediction than the ones reflecting feedback behaviors. CONCLUSION Scene-dependent gaze metrics revealed skill levels of trainees more precisely than between experts and novices as suggested in the literature. Further, feedforward gaze metrics appeared to be more important than feedback ones at predicting proficiency.
Collapse
Affiliation(s)
- Chaitanya S Kulkarni
- Grado Department of Industrial and Systems Engineering, Virginia Tech, 250 Durham Hall (0118), 1145 Perry Street, Blacksburg, VA, 24061, USA
| | - Shiyu Deng
- Grado Department of Industrial and Systems Engineering, Virginia Tech, 250 Durham Hall (0118), 1145 Perry Street, Blacksburg, VA, 24061, USA
| | - Tianzi Wang
- Grado Department of Industrial and Systems Engineering, Virginia Tech, 250 Durham Hall (0118), 1145 Perry Street, Blacksburg, VA, 24061, USA
| | | | - Laura E Barnes
- Environmental and Systems Engineering, University of Virginia, Charlottesville, VA, USA
| | | | - Shawn D Safford
- Division of Pediatric General and Thoracic Surgery, UPMC Children's Hospital of Pittsburgh, Harrisburg, PA, USA
| | - Nathan Lau
- Grado Department of Industrial and Systems Engineering, Virginia Tech, 250 Durham Hall (0118), 1145 Perry Street, Blacksburg, VA, 24061, USA.
| |
Collapse
|
10
|
Koskinen J, He W, Elomaa AP, Kaipainen A, Hussein A, Zheng B, Huotarinen A, Bednarik R. Utilizing Grasp Monitoring to Predict Microsurgical Expertise. J Surg Res 2023; 282:101-108. [PMID: 36265429 DOI: 10.1016/j.jss.2022.09.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Revised: 07/22/2022] [Accepted: 09/18/2022] [Indexed: 11/07/2022]
Abstract
INTRODUCTION Most microsurgical procedures require the surgeon to use tools to grasp and hold fragile objects in the surgical site. Prior research on grasping in surgery has mostly either been in other surgical techniques or used grasping as an auxiliary metric. We focus on microsurgery and investigate what grasping can tell about microsurgical skill and suturing performance. This study lays groundwork for using automatic detection of grasps to evaluate surgical skill. METHODS Five expert surgeons and six novices completed sutures on a microsurgical training board. Video recordings of the performance were annotated for the number of grasps, while an eye tracker recorded the participants' pupil dilations for cognitive workload assessment. Performance was measured with suturing duration and the University of Western Ontario Microsurgical Skills Assessment instrument (UWOMSA). Differences in skill, suturing performance and cognitive workload were compared with grasping behavior. RESULTS Novices needed significantly more grasps to complete sutures and failed to grasp more often than the experts. The number of grasps affected the suturing duration more in novices. Decreasing suturing efficiency as measured by UWOMSA instrument was associated with increase in grasps, even when we controlled for overall skill differences. Novices displayed larger pupil dilations when averaged over a sufficiently large sample, and the difference increased after the grasp. CONCLUSIONS Grasping action during microsurgical procedures can be used as a conceptually simple yet objective proxy in microsurgical performance assessment. If the grasps could be detected automatically, they could be used to aid in computational evaluation of surgical trainees' performance.
Collapse
Affiliation(s)
- Jani Koskinen
- School of Computing, University of Eastern Finland, Joensuu, Finland.
| | - Wenjing He
- Department of Surgery, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Manitoba, Canada
| | - Antti-Pekka Elomaa
- Department of Neurosurgery, Institute of Clinical Medicine, Kuopio University Hospital, Kuopio, Finland; Microsurgery Center, Kuopio University Hospital, Kuopio, Finland; School of Medicine, University of Eastern Finland, Kuopio, Finland
| | - Aku Kaipainen
- Department of Neurosurgery, Institute of Clinical Medicine, Kuopio University Hospital, Kuopio, Finland; Microsurgery Center, Kuopio University Hospital, Kuopio, Finland
| | - Ahmed Hussein
- Department of Neurosurgery, Institute of Clinical Medicine, Kuopio University Hospital, Kuopio, Finland; Microsurgery Center, Kuopio University Hospital, Kuopio, Finland
| | - Bin Zheng
- Surgical Simulation Research Lab, Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | - Antti Huotarinen
- Department of Neurosurgery, Institute of Clinical Medicine, Kuopio University Hospital, Kuopio, Finland; Microsurgery Center, Kuopio University Hospital, Kuopio, Finland
| | - Roman Bednarik
- School of Computing, University of Eastern Finland, Joensuu, Finland
| |
Collapse
|
11
|
Azargoshasb S, Boekestijn I, Roestenberg M, KleinJan GH, van der Hage JA, van der Poel HG, Rietbergen DDD, van Oosterom MN, van Leeuwen FWB. Quantifying the Impact of Signal-to-background Ratios on Surgical Discrimination of Fluorescent Lesions. Mol Imaging Biol 2023; 25:180-189. [PMID: 35711014 PMCID: PMC9971139 DOI: 10.1007/s11307-022-01736-y] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Revised: 03/28/2022] [Accepted: 04/21/2022] [Indexed: 12/14/2022]
Abstract
PURPOSE Surgical fluorescence guidance has gained popularity in various settings, e.g., minimally invasive robot-assisted laparoscopic surgery. In pursuit of novel receptor-targeted tracers, the field of fluorescence-guided surgery is currently moving toward increasingly lower signal intensities. This highlights the importance of understanding the impact of low fluorescence intensities on clinical decision making. This study uses kinematics to investigate the impact of signal-to-background ratios (SBR) on surgical performance. METHODS Using a custom grid exercise containing hidden fluorescent targets, a da Vinci Xi robot with Firefly fluorescence endoscope and ProGrasp and Maryland forceps instruments, we studied how the participants' (N = 16) actions were influenced by the fluorescent SBR. To monitor the surgeon's actions, the surgical instrument tip was tracked using a custom video-based tracking framework. The digitized instrument tracks were then subjected to multi-parametric kinematic analysis, allowing for the isolation of various metrics (e.g., velocity, jerkiness, tortuosity). These were incorporated in scores for dexterity (Dx), decision making (DM), overall performance (PS) and proficiency. All were related to the SBR values. RESULTS Multi-parametric analysis showed that task completion time, time spent in fluorescence-imaging mode and total pathlength are metrics that are directly related to the SBR. Below SBR 1.5, these values substantially increased, and handling errors became more frequent. The difference in Dx and DM between the targets that gave SBR < 1.50 and SBR > 1.50, indicates that the latter group generally yields a 2.5-fold higher Dx value and a threefold higher DM value. As these values provide the basis for the PS score, proficiency could only be achieved at SBR > 1.55. CONCLUSION By tracking the surgical instruments we were able to, for the first time, quantitatively and objectively assess how the instrument positioning is impacted by fluorescent SBR. Our findings suggest that in ideal situations a minimum SBR of 1.5 is required to discriminate fluorescent lesions, a substantially lower value than the SBR 2 often reported in literature.
Collapse
Affiliation(s)
- Samaneh Azargoshasb
- Interventional Molecular Imaging Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands.,Department of Urology, Netherlands Cancer Institute-Antoni Van Leeuwenhoek Hospital, Amsterdam, the Netherlands
| | - Imke Boekestijn
- Interventional Molecular Imaging Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands.,Section of Nuclear Medicine, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands
| | - Meta Roestenberg
- Department of Parasitology, Leiden University Medical Center, Leiden, the Netherlands.,Department of Infectious Diseases, Leiden University Medical Center, Leiden, the Netherlands
| | - Gijs H KleinJan
- Department of Urology, Leiden University Medical Center, Leiden, The Netherlands
| | - Jos A van der Hage
- Department of Surgery, Leiden University Medical Center, Leiden, the Netherlands
| | - Henk G van der Poel
- Department of Urology, Netherlands Cancer Institute-Antoni Van Leeuwenhoek Hospital, Amsterdam, the Netherlands
| | - Daphne D D Rietbergen
- Interventional Molecular Imaging Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands.,Section of Nuclear Medicine, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands
| | - Matthias N van Oosterom
- Interventional Molecular Imaging Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands.,Department of Urology, Netherlands Cancer Institute-Antoni Van Leeuwenhoek Hospital, Amsterdam, the Netherlands
| | - Fijs W B van Leeuwen
- Interventional Molecular Imaging Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands. .,Department of Urology, Netherlands Cancer Institute-Antoni Van Leeuwenhoek Hospital, Amsterdam, the Netherlands.
| |
Collapse
|
12
|
Valles J, Zhang T, McIntosh P, Pacilli M, Nataraja RM. Assessment of Core Surgical Skills Using a Mixed Reality Headset - The MoTOR Study. J Med Syst 2022; 46:102. [PMID: 36418760 DOI: 10.1007/s10916-022-01891-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Revised: 11/12/2022] [Accepted: 11/14/2022] [Indexed: 11/25/2022]
Abstract
INTRODUCTION Surgical skill assessment utilises direct observation and feedback by an expert which is potentially subjective, therefore obtaining objective data for hand and eye tracking is essential. Our aim was to evaluate a wearable mixed reality (MR) headset in these domains. METHODS Participants with differing levels of surgical expertise [novice (N), intermediate (I) & expert (E)] performed 4 simulated surgical tasks; 2 general dexterity (tasks 1&2) and 2 surgical skills (tasks 3&4) wearing the MR headset capturing their hand and eye movements (median & range). Metrics included hand path length and the speed of each index or thumb tip. Gaze data were also captured. Participant demographics, prior expertise and current experience were captured with an electronic survey. Data were analysed with a Shapiro-Wilk test or ANOVA as appropriate. A p-value of < 0.05 was significant. RESULTS Thirty-six participants were analysed (N = 18, I = 8, E = 8). Tasks 1&2 revealed 2 speed outcomes (left index and left-hand speed) which were significant. For tasks 3&4, various outcomes were significant: path length for left hand (N:45 cm vs. I:31 cm vs. E:27 cm, p = 0.03) and right hand (N:48 cm vs. I:29 cm vs. E:28 cm, p = 0.01) and total time (N:456s vs. I:292 vs. E: 245, p = 0.0002). With left-hand-tying, average path length (N:61 cm vs. I:39 vs. E:36, p = 0.04), average speed (N:11 cm/s vs. I:23 vs. E:24, p = 0.03), and total time (N:156s vs. I:43 vs. E:37, p = 0.003) were significant. The gaze-tracking was not statistically significant. CONCLUSION The MR headset can be utilised as a valid tool for surgical performance assessment. Outcomes including path length and speed can be valuable metrics captured by the MR Headset during the task completion for detecting surgical proficiency.
Collapse
Affiliation(s)
- John Valles
- Department of Paediatric Surgery & Monash Children's Simulation, Monash Children's Hospital, 246 Clayton Road, Clayton, 3168, Melbourne, Australia.,Departments of Paediatrics & Surgery, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Taiqing Zhang
- Virtual and Augmented Reality Services Unit, Monash eSolutions, Monash University, Melbourne, VIC, Australia
| | - Paul McIntosh
- Virtual and Augmented Reality Services Unit, Monash eSolutions, Monash University, Melbourne, VIC, Australia.,Human Centred Computing, Faculty of Information Technology, Monash University, Melbourne, VIC, Australia
| | - Maurizio Pacilli
- Department of Paediatric Surgery & Monash Children's Simulation, Monash Children's Hospital, 246 Clayton Road, Clayton, 3168, Melbourne, Australia.,Departments of Paediatrics & Surgery, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Ramesh M Nataraja
- Department of Paediatric Surgery & Monash Children's Simulation, Monash Children's Hospital, 246 Clayton Road, Clayton, 3168, Melbourne, Australia. .,Departments of Paediatrics & Surgery, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia.
| |
Collapse
|
13
|
Application of an Optical Tracking System for Motor Skill Assessment in Laparoscopic Surgery. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2022; 2022:2332628. [PMID: 35912156 PMCID: PMC9337947 DOI: 10.1155/2022/2332628] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Accepted: 06/25/2022] [Indexed: 11/18/2022]
Abstract
Objective Motion analysis of surgical instruments can be used to evaluate laparoscopic surgical skills, and this study assessed the validity of an optical tracking system for the assessment of laparoscopic surgical motor skills. Methods Ten experienced surgeons and ten novices were recruited to complete the transferring tasks on a laparoscopic simulator. An optical tracking system, Micron Tracker, was used to capture the marker points on each instrument and to obtain the coordinates of the marker points and the corresponding instrument tip coordinates. The data are processed to create a coordinate system based on the laparoscopic simulator and to calculate the movement parameters of the instruments, such as operating time, path length, speed, acceleration, and smoothness. At the same time, the range of motion of the instrument (insertion depth and pivoting angle) is also calculated. Results The position that the tip of the instrument can reach is a small, irregularly shaped spatial area. Significant differences (p < 0.05) were found between the surgeon and novice groups in parameters such as operating time, path length, mean speed, mean acceleration, and mean smoothness. The range of insertion depth of the instruments was approximately 150 mm to 240 mm, and the pivoting angles of the left and right instruments were 30.9° and 46.6° up and down and 28.0° and 35.0° left and right, respectively. Conclusions The optical tracking system was effective in subjectively evaluating laparoscopic surgical skills, with significant differences between the surgeon and novice groups in terms of movement parameters, but not in terms of range of motion.
Collapse
|
14
|
Development and Validation of a Virtual Reality Simulator for Robot-Assisted Minimally Invasive Liver Surgery Training. J Clin Med 2022; 11:jcm11144145. [PMID: 35887909 PMCID: PMC9322966 DOI: 10.3390/jcm11144145] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2022] [Revised: 07/09/2022] [Accepted: 07/13/2022] [Indexed: 02/01/2023] Open
Abstract
The value of kinematic data for skill assessment is being investigated. This is the first virtual reality simulator developed for liver surgery. This simulator was coded in C++ using PhysX and FleX with a novel cutting algorithm and used a patient data-derived model and two instruments functioning as ultrasonic shears. The simulator was evaluated by nine expert surgeons and nine surgical novices. Each participant performed a simulated metastasectomy after training. Kinematic data were collected for the instrument position. Each participant completed a survey. The expert participants had a mean age of 47 years and 9/9 were certified in surgery. Novices had a mean age of 30 years and 0/9 were certified surgeons. The mean path length (novice 0.76 ± 0.20 m vs. expert 0.46 ± 0.16 m, p = 0.008), movements (138 ± 45 vs. 84 ± 32, p = 0.043) and time (174 ± 44 s vs. 102 ± 42 s, p = 0.004) were significantly different for the two participant groups. There were no significant differences in activating the instrument (107 ± 25 vs. 109 ± 53). Participants considered the simulator realistic (6.5/7) (face validity), appropriate for education (5/7) (content validity) with an effective interface (6/7), consistent motion (5/7) and realistic soft tissue behavior (5/7). This study showed that the simulator differentiates between experts and novices. Simulation may be an effective way to obtain kinematic data.
Collapse
|
15
|
Shimizu A, Ito M, Lefor AK. Laparoscopic and Robot-Assisted Hepatic Surgery: An Historical Review. J Clin Med 2022; 11:jcm11123254. [PMID: 35743324 PMCID: PMC9225080 DOI: 10.3390/jcm11123254] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Revised: 05/28/2022] [Accepted: 06/05/2022] [Indexed: 12/07/2022] Open
Abstract
Hepatic surgery is a rapidly expanding component of abdominal surgery and is performed for a wide range of indications. The introduction of laparoscopic cholecystectomy in 1987 was a major change in abdominal surgery. Laparoscopic surgery was widely and rapidly adopted throughout the world for cholecystectomy initially and then applied to a variety of other procedures. Laparoscopic surgery became regularly applied to hepatic surgery, including segmental and major resections as well as organ donation. Many operations progressed from open surgery to laparoscopy to robot-assisted surgery, including colon resection, pancreatectomy, splenectomy thyroidectomy, adrenalectomy, prostatectomy, gastrectomy, and others. It is difficult to prove a data-based benefit using robot-assisted surgery, although laparoscopic and robot-assisted surgery of the liver are not inferior regarding major outcomes. When laparoscopic surgery initially became popular, many had concerns about its use to treat malignancies. Robot-assisted surgery is being used to treat a variety of benign and malignant conditions, and studies have shown no deterioration in outcomes. Robot-assisted surgery for the treatment of malignancies has become accepted and is now being used at more centers. The outcomes after robot-assisted surgery depend on its use at specialized centers, the surgeon's personal experience backed up by extensive training and maintenance of international registries. Robot-assisted hepatic surgery has been shown to be associated with slightly less intraoperative blood loss and shorter hospital lengths of stay compared to open surgery. Oncologic outcomes have been maintained, and some studies show higher rates of R0 resections. Patients who need surgery for liver lesions should identify a surgeon they trust and should not be concerned with the specific operative approach used. The growth of robot-assisted surgery of the liver has occurred in a stepwise approach which is very different from the frenzy that was seen with the introduction of laparoscopic cholecystectomy. This approach allowed the identification of areas for improvement, many of which are at the nexus of engineering and medicine. Further improvements in robot-assisted surgery depend on the combined efforts of engineers and surgeons.
Collapse
|
16
|
Deie K, Nakagawa Y, Uchida H, Hinoki A, Shirota C, Tainaka T, Sumida W, Yokota K, Makita S, Fujiogi M, Okamoto M, Takimoto A, Yasui A, Takada S, Maeda T. Evaluation of minimally invasive surgical skills training: comparing a neonatal esophageal atresia/tracheoesophageal fistula model with a dry box. Surg Endosc 2022; 36:6035-6048. [PMID: 35312850 DOI: 10.1007/s00464-022-09185-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2021] [Accepted: 03/07/2022] [Indexed: 11/27/2022]
Abstract
BACKGROUND Pediatric surgeons require highly advanced minimally invasive surgical skills to perform rare and complex surgeries in a very vulnerable population. We developed a neonatal esophageal atresia (EA) model to improve thoracoscopic surgical skills. This study aimed to evaluate the concurrent validity of the model by undertaking pre- and post-training skills assessments in two groups of students with no prior experience performing minimally invasive surgery, using the EA model and a dry box (DB). METHODS A pilot study was performed. The participants were randomly divided into two groups: one trained using the DB and one trained using the EA model. Both groups practiced a minimally invasive surgical suture task. The task completion time, 29-point checklist score, modified suturing error sheet score, and three-dimensional forceps movement in both groups were compared pre-and post-training by video analysis. RESULTS The EA model task was significantly more difficult than that of the DB. Both groups showed significant improvement in the task time, 29-point checklist score, and modified suturing error sheet score; however, the EA model training was more efficient in improving each error item. Regarding forceps movement, the EA model training significantly decreased wasted motion, whereas the DB was limited in this regard. CONCLUSIONS Short-term training on the EA model, which was more technically demanding than the DB, decreased technical error and wasted motion, and allowed learners to acquire surgical skills more efficiently than training with the DB model. These facts revealed the concurrent validity of the EA model.
Collapse
Affiliation(s)
- Kyoichi Deie
- Department of Pediatric Surgery, Graduate School of Medicine and Faculty of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
| | - Yoichi Nakagawa
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Hiroo Uchida
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan.
| | - Akinari Hinoki
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Chiyoe Shirota
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Takahisa Tainaka
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Wataru Sumida
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Kazuki Yokota
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Satoshi Makita
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Michimasa Fujiogi
- Department of Pediatric Surgery, Graduate School of Medicine and Faculty of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
| | - Masamune Okamoto
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Aitaro Takimoto
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Akihiro Yasui
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Shunya Takada
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Takuya Maeda
- Department of Pediatric Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| |
Collapse
|
17
|
Ranking surgical skills using an attention-enhanced Siamese network with piecewise aggregated kinematic data. Int J Comput Assist Radiol Surg 2022; 17:1039-1048. [DOI: 10.1007/s11548-022-02581-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2021] [Accepted: 02/16/2022] [Indexed: 11/25/2022]
|
18
|
Face and construct validity assessment of training models for intestinal anastomosis in low-birth-weight infants. Pediatr Surg Int 2021; 37:1765-1772. [PMID: 34476538 DOI: 10.1007/s00383-021-04991-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 08/01/2021] [Indexed: 10/20/2022]
Abstract
PURPOSE It is difficult to perform intestinal anastomosis in low-birth-weight infants because the intestinal diameter is small and the discrepancy in diameter of the proximal and distal intestines is often large, but there has been no optimal-sized training model. Therefore, we developed a new intestinal anastomosis training model that imitated the size of the intestine in low-birth-weight infants, and evaluated its face and construct validity. METHODS Two intestinal models were developed with crossMedical, Inc. using a hydrophilic acrylic material (wet model) or a polyurethane soft resin (dry model). The inner diameter of the simulated intestinal tract was 15 mm on the oral end and 6 mm on the anal end. Thirteen pediatric surgeons performed anastomosis and responded to the questionnaire. RESULTS In the questionnaire, the wet model had significantly higher scores than the dry model in "appearance", "softness" and "usefulness for training". In the anastomotic results of the wet model, the anastomosis leak pressure was significantly correlated with the number of intestinal anastomotic experiences in low-birth-weight infants (correlation coefficient = 0.64, P = 0.035). CONCLUSIONS The wet-type intestinal anastomosis model showed good face validity. Its leak pressure had a significant correlation with clinical experience; thus, construct validity was demonstrated.
Collapse
|
19
|
Review of automated performance metrics to assess surgical technical skills in robot-assisted laparoscopy. Surg Endosc 2021; 36:853-870. [PMID: 34750700 DOI: 10.1007/s00464-021-08792-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2021] [Accepted: 10/17/2021] [Indexed: 10/19/2022]
Abstract
INTRODUCTION Robot-assisted laparoscopy is a safe surgical approach with several studies suggesting correlations between complication rates and the surgeon's technical skills. Surgical skills are usually assessed by questionnaires completed by an expert observer. With the advent of surgical robots, automated surgical performance metrics (APMs)-objective measures related to instrument movements-can be computed. The aim of this systematic review was thus to assess APMs use in robot-assisted laparoscopic procedures. The primary outcome was the assessment of surgical skills by APMs and the secondary outcomes were the association between APM and surgeon parameters and the prediction of clinical outcomes. METHODS A systematic review following the PRISMA guidelines was conducted. PubMed and Scopus electronic databases were screened with the query "robot-assisted surgery OR robotic surgery AND performance metrics" between January 2010 and January 2021. The quality of the studies was assessed by the medical education research study quality instrument. The study settings, metrics, and applications were analysed. RESULTS The initial search yielded 341 citations of which 16 studies were finally included. The study settings were either simulated virtual reality (VR) (4 studies) or real clinical environment (12 studies). Data to compute APMs were kinematics (motion tracking), and system and specific events data (actions from the robot console). APMs were used to differentiate expertise levels, and thus validate VR modules, predict outcomes, and integrate datasets for automatic recognition models. APMs were correlated with clinical outcomes for some studies. CONCLUSIONS APMs constitute an objective approach for assessing technical skills. Evidence of associations between APMs and clinical outcomes remain to be confirmed by further studies, particularly, for non-urological procedures. Concurrent validation is also required.
Collapse
|
20
|
Johnson G, Vergis A, Unger B, Park J, Gillman L. Design and Validity Evidence for a Unique Endoscopy Simulator Using a Commercial Video Game. Cureus 2021; 13:e18379. [PMID: 34725623 PMCID: PMC8553393 DOI: 10.7759/cureus.18379] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/29/2021] [Indexed: 11/05/2022] Open
Abstract
Background Procedural simulation enhances early endoscopy training. Multiple commercial simulators are available; however, their application is limited by cost and poor user compliance. First-person "shooter" (FPS) video games are popular. In this study, we aimed to show that a novel in-house designed colonoscope controller used to play an FPS video game shares similar constructs with real-life endoscopy. Methodology Participants completed the first three levels on an FPS video game, Portal (Valve Corporation, Bellevue, WA), first using a conventional controller and then the modified endoscope controller. A total of 12 expert endoscopists and 12 surgical residents with minimal endoscopy experience were evaluated based on completion time, button presses, and hand motion analyses. Results Experts outperformed novices for completion time (expert: 944 seconds; novice: 1,515 seconds; p = 0.006) and hand movements (expert: 1,263.1; novice: 2,052.6; p = 0.004) in using the novel colonoscope controller. There was no difference in button presses or total path length traveled. Furthermore, performance did not differ using conventional game controls. Conclusions Experts outperformed novices using the endoscope but not the conventional controller with respect to the economy of movement and completion time. This result confirms that our endoscope-controlled video game shares similar paradigms with real-life endoscopy and serves as a first step toward creating a more enjoyable and cheaper alternative to commercially available endoscopy simulators.
Collapse
Affiliation(s)
| | | | - Bertram Unger
- Internal Medicine, University of Manitoba, Winnipeg, CAN
| | - Jason Park
- Surgery, University of Manitoba, Winnipeg, CAN
| | | |
Collapse
|
21
|
Beqari J, Seymour NE. Application of technology to educational needs in surgery. J Surg Oncol 2021; 124:181-192. [PMID: 34245576 DOI: 10.1002/jso.26512] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2021] [Revised: 04/14/2021] [Accepted: 04/17/2021] [Indexed: 01/22/2023]
Abstract
Innovations in surgical education follow advancing clinical technology. New surgical methods have prompted demand for systematic methods to leverage computing power and internet tools to achieve proficiency-based training goals. Virtual reality, high-fidelity patient simulation, web-based resources to facilitate performance assessment, and telementoring have become mainstream practices, although patient outcomes benefits are not well studied. Remote virtual meeting and mentoring have had transformative effects on resident experiences, the full effects of which remain to be seen.
Collapse
Affiliation(s)
- Jorind Beqari
- University of Massachusetts Medical School-Baystate, Springfield, Massachusetts, USA
| | - Neal E Seymour
- University of Massachusetts Medical School-Baystate, Springfield, Massachusetts, USA
| |
Collapse
|
22
|
Comprehensive metrics for evaluating surgical microscope use during tympanostomy tube placement. Int J Comput Assist Radiol Surg 2021; 16:1587-1594. [PMID: 34089123 DOI: 10.1007/s11548-021-02428-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2021] [Accepted: 05/28/2021] [Indexed: 01/22/2023]
Abstract
PURPOSE Learning to use a surgical microscope is a fundamental step in otolaryngology training; however, there is currently no objective method to teach or assess this skill. Tympanostomy tube placement is a common otologic procedure that requires skilled use of a surgical microscope. This study was designed to (1) implement metrics capable of evaluating microscope use and (2) establish construct validity. STUDY DESIGN This was a prospective cohort study. METHODS Eight otolaryngology trainees and three otolaryngology experts were asked to use a microscope to insert a tympanostomy tube into a cadaveric myringotomy in a standardized setting. Microscope movements were tracked in a three-dimensional space, and tracking metrics were applied to the data. The procedure was video-recorded and then analyzed by blinded experts using operational metrics. Results from both groups were compared, and discriminatory metrics were determined. RESULTS The following tracking metrics were identified as discriminatory between the trainee and expert groups: total completion time, operation time, still time, and jitter (movement perturbation). Many operational metrics were found to be discriminatory between the two groups, including several positioning metrics, optical metrics, and procedural metrics. CONCLUSIONS Performance metrics were implemented, and construct validity was established for a subset of the proposed metrics by discriminating between expert and novice participants. These discriminatory metrics could form the basis of an automated system for providing feedback to residents during training while using a myringotomy surgical simulator. Additionally, these metrics may be useful in guiding a standardized teaching and evaluation methodology for training in the use of surgical microscopes.
Collapse
|
23
|
Azargoshasb S, van Alphen S, Slof LJ, Rosiello G, Puliatti S, van Leeuwen SI, Houwing KM, Boonekamp M, Verhart J, Dell'Oglio P, van der Hage J, van Oosterom MN, van Leeuwen FWB. The Click-On gamma probe, a second-generation tethered robotic gamma probe that improves dexterity and surgical decision-making. Eur J Nucl Med Mol Imaging 2021; 48:4142-4151. [PMID: 34031721 PMCID: PMC8566398 DOI: 10.1007/s00259-021-05387-z] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2021] [Accepted: 04/25/2021] [Indexed: 11/24/2022]
Abstract
Purpose Decision-making and dexterity, features that become increasingly relevant in (robot-assisted) minimally invasive surgery, are considered key components in improving the surgical accuracy. Recently, DROP-IN gamma probes were introduced to facilitate radioguided robotic surgery. We now studied if robotic DROP-IN radioguidance can be further improved using tethered Click-On designs that integrate gamma detection onto the robotic instruments themselves. Methods Using computer-assisted drawing software, 3D printing and precision machining, we created a Click-On probe containing two press-fit connections and an additional grasping moiety for a ProGrasp instrument combined with fiducials that could be video tracked using the Firefly laparoscope. Using a dexterity phantom, the duration of the specific tasks and the path traveled could be compared between use of the Click-On or DROP-IN probe. To study the impact on surgical decision-making, we performed a blinded study, in porcine models, wherein surgeons had to identify a hidden 57Co-source using either palpation or Click-On radioguidance. Results When assembled onto a ProGrasp instrument, while preserving grasping function and rotational freedom, the fully functional prototype could be inserted through a 12-mm trocar. In dexterity assessments, the Click-On provided a 40% reduction in movements compared to the DROP-IN, which converted into a reduction in time, path length, and increase in straightness index. Radioguidance also improved decision-making; task-completion rate increased by 60%, procedural time was reduced, and movements became more focused. Conclusion The Click-On gamma probe provides a step toward full integration of radioguidance in minimal invasive surgery. The value of this concept was underlined by its impact on surgical dexterity and decision-making.
Collapse
Affiliation(s)
- Samaneh Azargoshasb
- Interventional Molecular Imaging-Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands.,Department of Urology, Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital, Amsterdam, the Netherlands
| | - Simon van Alphen
- Interventional Molecular Imaging-Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands
| | - Leon J Slof
- Interventional Molecular Imaging-Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands.,Instrumentele zaken ontwikkeling, facilitair bedrijf, Leiden University Medical Center, Leiden, the Netherlands
| | - Giuseppe Rosiello
- Department of Urology and Division of Experimental Oncology, Urological Research Institute IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Stefano Puliatti
- Department of Urology, University of Modena and Reggio Emilia, Via del Pozzo, 71, 41124, Modena, Italy.,ORSI Academy, Melle, Belgium.,Department of Urology, Onze Lieve Vrouw Hospital, Aalst, Belgium
| | - Sven I van Leeuwen
- Interventional Molecular Imaging-Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands
| | - Krijn M Houwing
- Interventional Molecular Imaging-Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands
| | - Michael Boonekamp
- Instrumentele zaken ontwikkeling, facilitair bedrijf, Leiden University Medical Center, Leiden, the Netherlands
| | - Jeroen Verhart
- Instrumentele zaken ontwikkeling, facilitair bedrijf, Leiden University Medical Center, Leiden, the Netherlands
| | - Paolo Dell'Oglio
- Interventional Molecular Imaging-Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands.,Department of Urology, Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital, Amsterdam, the Netherlands.,ORSI Academy, Melle, Belgium.,Department of Urology, ASST Grande Ospedale Metropolitano Niguarda, Milan, Italy
| | - Jos van der Hage
- Department of Surgery, Leiden University Medical Center, Leiden, the Netherlands
| | - Matthias N van Oosterom
- Interventional Molecular Imaging-Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands.,Department of Urology, Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital, Amsterdam, the Netherlands
| | - Fijs W B van Leeuwen
- Interventional Molecular Imaging-Laboratory, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands. .,Department of Urology, Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital, Amsterdam, the Netherlands. .,ORSI Academy, Melle, Belgium.
| |
Collapse
|
24
|
Lefor AK, Harada K, Dosis A, Mitsuishi M. Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set II: learning curve analysis. Int J Comput Assist Radiol Surg 2021; 16:589-595. [PMID: 33723706 DOI: 10.1007/s11548-021-02339-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2020] [Accepted: 02/25/2021] [Indexed: 01/12/2023]
Abstract
PURPOSE The Johns Hopkins-Intuitive Gesture and Skill Assessment Working Set (JIGSAWS) dataset is used to develop robotic surgery skill assessment tools, but there has been no detailed analysis of this dataset. The aim of this study is to perform a learning curve analysis of the existing JIGSAWS dataset. METHODS Five trials were performed in JIGSAWS by eight participants (four novices, two intermediates and two experts) for three exercises (suturing, knot-tying and needle passing). Global Rating Scores and time, path length and movements were analyzed quantitatively and qualitatively by graphical analysis. RESULTS There are no significant differences in Global Rating Scale scores over time. Time in the suturing exercise and path length in needle passing had significant differences. Other kinematic parameters were not significantly different. Qualitative analysis shows a learning curve only for suturing. Cumulative sum analysis suggests completion of the learning curve for suturing by trial 4. CONCLUSIONS The existing JIGSAWS dataset does not show a quantitative learning curve for Global Rating Scale scores, or most kinematic parameters which may be due in part to the limited size of the dataset. Qualitative analysis shows a learning curve for suturing. Cumulative sum analysis suggests completion of the suturing learning curve by trial 4. An expanded dataset is needed to facilitate subset analyses.
Collapse
Affiliation(s)
- Alan Kawarai Lefor
- Bioengineering, School of Engineering, The University of Tokyo, Tokyo, Japan.
| | - Kanako Harada
- Mechanical Engineering, School of Engineering, The University of Tokyo, Tokyo, Japan
- Bioengineering, School of Engineering, The University of Tokyo, Tokyo, Japan
| | | | - Mamoru Mitsuishi
- Mechanical Engineering, School of Engineering, The University of Tokyo, Tokyo, Japan
- Bioengineering, School of Engineering, The University of Tokyo, Tokyo, Japan
| |
Collapse
|
25
|
Pastewski J, Baker D, Somerset A, Leonard K, Azzie G, Roach VA, Ziegler K, Brahmamdam P. Analysis of Instrument Motion and the Impact of Residency Level and Concurrent Distraction on Laparoscopic Skills. JOURNAL OF SURGICAL EDUCATION 2021; 78:265-274. [PMID: 32741690 DOI: 10.1016/j.jsurg.2020.07.012] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/08/2020] [Revised: 06/15/2020] [Accepted: 07/13/2020] [Indexed: 06/11/2023]
Abstract
OBJECTIVE Using a laparoscopic box trainer fitted with motion analysis trackers and software, we aim to identify differences between junior and senior residents performing the peg transfer task, and the impact of a distracting secondary task on performance. DESIGN General surgery residents were asked to perform the laparoscopic peg transfer task on a trainer equipped with a motion tracker. They were also asked to perform the laparoscopic task while completing a secondary task. Extreme velocity and acceleration events of instrument movement in the 3 rotational degrees of freedom were measured during task completion. The number of extreme events, defined as velocity or acceleration exceeding 1 SD above or below their own mean, were tabulated. The performance of junior residents was compared to senior residents. SETTING Simulation learning institute, Beaumont Hospital, Royal Oak, Michigan. PARTICIPANTS Thirty-seven general surgery residents from Beaumont Hospital, Royal Oak. RESULTS When completing the primary task alone, senior residents executed significantly fewer extreme motion events specific to acceleration in pitch (16.63 vs. 20.69, p = 0.04), and executed more extreme motion events specific to velocity in roll (16.14 vs. 15.11, p = 0.038), when compared to junior residents. With addition of a secondary task, senior residents had fewer extreme acceleration events specific to pitch, (14.69 vs. 22.22, p < 0.001). CONCLUSIONS While junior and senior residents completed the peg transfer task with similar times, motion analysis identified differences in extreme motion events between the groups, even when a secondary task was added. Motion analysis may prove useful for real-time feedback during laparoscopic skill acquisition.
Collapse
Affiliation(s)
| | - Dustin Baker
- Department of Surgery, Beaumont Health, Royal Oak, Michigan
| | - Amy Somerset
- Department of Surgery, Beaumont Health, Royal Oak, Michigan
| | - Kelsey Leonard
- Department of Foundational Medical Studies, Oakland University William Beaumont School of Medicine, Rochester, Michigan
| | - Georges Azzie
- Division of General and Thoracic Surgery, Hospital for Sick Children, Toronto, Canada
| | - Victoria A Roach
- Department of Foundational Medical Studies, Oakland University William Beaumont School of Medicine, Rochester, Michigan; Department of Surgery, Oakland University William Beaumont School of Medicine, Rochester, Michigan
| | - Kathryn Ziegler
- Department of Surgery, Beaumont Health, Royal Oak, Michigan; Department of Surgery, Oakland University William Beaumont School of Medicine, Rochester, Michigan
| | - Pavan Brahmamdam
- Department of Surgery, Beaumont Health, Royal Oak, Michigan; Department of Surgery, Oakland University William Beaumont School of Medicine, Rochester, Michigan.
| |
Collapse
|
26
|
Lefor AK, Harada K, Dosis A, Mitsuishi M. Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set using Robotics Video and Motion Assessment Software. Int J Comput Assist Radiol Surg 2020; 15:2017-2025. [PMID: 33025366 PMCID: PMC7671974 DOI: 10.1007/s11548-020-02259-z] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2020] [Accepted: 09/04/2020] [Indexed: 12/01/2022]
Abstract
Purpose The JIGSAWS dataset is a fixed dataset of robot-assisted surgery kinematic data used to develop predictive models of skill. The purpose of this study is to analyze the relationships of self-defined skill level with global rating scale scores and kinematic data (time, path length and movements) from three exercises (suturing, knot-tying and needle passing) (right and left hands) in the JIGSAWS dataset. Methods Global rating scale scores are reported in the JIGSAWS dataset and kinematic data were calculated using ROVIMAS software. Self-defined skill levels are in the dataset (novice, intermediate, expert). Correlation coefficients (global rating scale-skill level and global rating scale-kinematic parameters) were calculated. Kinematic parameters were compared among skill levels. Results Global rating scale scores correlated with skill in the knot-tying exercise (r = 0.55, p = 0.0005). In the suturing exercise, time, path length (left) and movements (left) were significantly different (p < 0.05) for novices and experts. For knot-tying, time, path length (right and left) and movements (right) differed significantly for novices and experts. For needle passing, no kinematic parameter was significantly different comparing novices and experts. The only kinematic parameter that correlated with global rating scale scores is time in the knot-tying exercise. Conclusion Global rating scale scores weakly correlate with skill level and kinematic parameters. The ability of kinematic parameters to differentiate among self-defined skill levels is inconsistent. Additional data are needed to enhance the dataset and facilitate subset analyses and future model development.
Collapse
Affiliation(s)
- Alan Kawarai Lefor
- Department of Bioengineering, School of Engineering, The University of Tokyo, Tokyo, Japan.
| | - Kanako Harada
- Department of Bioengineering, School of Engineering, The University of Tokyo, Tokyo, Japan.,Department of Mechanical Engineering, School of Engineering, The University of Tokyo, Tokyo, Japan
| | | | - Mamoru Mitsuishi
- Department of Bioengineering, School of Engineering, The University of Tokyo, Tokyo, Japan.,Department of Mechanical Engineering, School of Engineering, The University of Tokyo, Tokyo, Japan
| |
Collapse
|
27
|
Azari DP, Miller BL, Le BV, Greenberg CC, Radwin RG. Quantifying surgeon maneuevers across experience levels through marker-less hand motion kinematics of simulated surgical tasks. APPLIED ERGONOMICS 2020; 87:103136. [PMID: 32501255 DOI: 10.1016/j.apergo.2020.103136] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/31/2018] [Revised: 04/20/2020] [Accepted: 04/21/2020] [Indexed: 06/11/2023]
Abstract
This paper compares clinician hand motion for common suturing tasks across a range of experience levels and tissue types. Medical students (32), residents (41), attending surgeons (10), and retirees (2) were recorded on digital video while suturing on one of: foam, pig feet, or porcine bowel. Depending on time in position, each medical student, resident, and attending participant was classified as junior or senior, yielding six experience categories. This work focuses on trends associated with increasing tenure observed from those medical students (10), residents (15), and attendings (10) who sutured on foam, and draws comparison across tissue types where pertinent. Utilizing custom software, the two-dimensional location of each of the participant's hands were automatically recorded in every video frame, producing a rich spatiotemporal feature set. While suturing on foam, increasing clinician experience was associated with conserved path length per cycle of the non-dominant hand, significantly reducing from junior medical students (mean = 73.63 cm, sd = 33.21 cm) to senior residents (mean = 46.16 cm, sd = 14.03 cm, p = 0.015), and again between senior residents and senior attendings (mean = 30.84 cm, sd = 14.51 cm, p = 0.045). Despite similar maneuver rates, attendings also accelerated less with their non-dominant hand (mean = 16.27 cm/s2, sd = 81.12 cm/s2, p = 0.002) than senior residents (mean = 24.84 cm/s2, sd = 68.29 cm/s2, p = 0.002). While tying, medical students moved their dominant hands slower (mean = 4.39 cm/s, sd = 1.73 cm/s, p = 0.033) than senior residents (mean = 6.53 cm/s, sd = 2.52 cm/s). These results suggest that increased psychomotor performance during early training manifest through faster dominant hand function, while later increases are characterized by conserving energy and efficiently distributing work between hands. Incorporating this scalable video-based motion analysis into regular formative assessment routines may enable greater quality and consistency of feedback throughout a surgical career.
Collapse
Affiliation(s)
- David P Azari
- Department of Industrial and Systems Engineering, 1550 Engineering Drive, University of Wisconsin-Madison, Madison, WI, 53706, USA
| | - Brady L Miller
- Department of Urology, Third Floor, 1685 Highland Avenue, University of Wisconsin-Madison, Madison, WI, 53705, USA
| | - Brian V Le
- Department of Urology, Third Floor, 1685 Highland Avenue, University of Wisconsin-Madison, Madison, WI, 53705, USA
| | - Caprice C Greenberg
- Wisconsin Surgical Outcomes Research (WiSOR) Program, Department of Surgery, Clinical Science Center, 600 Highland Avenue, University of Wisconsin-Madison, Madison, WI, 53792, USA
| | - Robert G Radwin
- Department of Industrial and Systems Engineering, 1550 Engineering Drive, University of Wisconsin-Madison, Madison, WI, 53706, USA; Department of Biomedical Engineering, 1415 Engineering Drive, University of Wisconsin-Madison, Madison, WI, 53706, USA.
| |
Collapse
|
28
|
Oviedo-Peñata CA, Tapia-Araya AE, Lemos JD, Riaño-Benavides C, Case JB, Maldonado-Estrada JG. Validation of Training and Acquisition of Surgical Skills in Veterinary Laparoscopic Surgery: A Review. Front Vet Sci 2020; 7:306. [PMID: 32582781 PMCID: PMC7283875 DOI: 10.3389/fvets.2020.00306] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2020] [Accepted: 05/05/2020] [Indexed: 02/06/2023] Open
Abstract
At present, veterinary laparoscopic surgery training is lacking in experiences that provide a controlled and safe environment where surgeons can practice specific techniques while receiving experts' feedback. Surgical skills acquired using simulators must be certified and transferable to the operating room. Most models for practicing laparoscopic skills in veterinary minimally invasive surgery are general task trainers and consist of boxes (simulators) designed for training human surgery. These simulators exhibit several limitations, including anatomic species and procedural differences, as well as general psychomotor training rather than in vivo skill recreation. In this paper, we review the existing methods of training, evaluation, and validation of technical skills in veterinary laparoscopic surgery. Content includes global and specific scales, and the conditions a structured curriculum should meet for improving the performance of novice surgeons during and after training. A focus on trainee-specific assessment and tailored-technical instruction should influence training programs. We provide a comprehensive analysis of current theories and concepts related to the evaluation and validation of simulators for training laparoscopic surgery in small animal surgery. We also highlight the need to develop new training models and complementary evaluation scales for the validation of training and acquisition of basic and advanced skills in veterinary laparoscopic surgery.
Collapse
Affiliation(s)
- Carlos A Oviedo-Peñata
- Tropical Animal Production Research Group, Faculty of Veterinary Medicine and Zootechny, University of Cordoba, Monteria, Colombia.,Surgery and Theriogenology Branch OHVRI-Group, College of Veterinary Medicine, University of Antioquia, Medellin, Colombia
| | | | - Juan D Lemos
- Bioinstrumentation and Clinical Engineering Research Group (GIBIC), Bioengineering Department, Engineering Faculty, Universidad de Antioquia, Medellín, Colombia
| | - Carlos Riaño-Benavides
- Surgery and Theriogenology Branch OHVRI-Group, College of Veterinary Medicine, University of Antioquia, Medellin, Colombia
| | - J Brad Case
- Department of Small Animal Clinical Sciences, College of Veterinary Medicine, University of Florida, Gainesville, FL, United States
| | - Juan G Maldonado-Estrada
- Surgery and Theriogenology Branch OHVRI-Group, College of Veterinary Medicine, University of Antioquia, Medellin, Colombia
| |
Collapse
|
29
|
Anh NX, Nataraja RM, Chauhan S. Towards near real-time assessment of surgical skills: A comparison of feature extraction techniques. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2020; 187:105234. [PMID: 31794913 DOI: 10.1016/j.cmpb.2019.105234] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/13/2019] [Revised: 10/31/2019] [Accepted: 11/18/2019] [Indexed: 05/22/2023]
Abstract
BACKGROUND AND OBJECTIVE Surgical skill assessment aims to objectively evaluate and provide constructive feedback for trainee surgeons. Conventional methods require direct observation with assessment from surgical experts which are both unscalable and subjective. The recent involvement of surgical robotic systems in the operating room has facilitated the ability of automated evaluation of the expertise level of trainees for certain representative maneuvers by using machine learning for motion analysis. The features extraction technique plays a critical role in such an automated surgical skill assessment system. METHODS We present a direct comparison of nine well-known feature extraction techniques which are statistical features, principal component analysis, discrete Fourier/Cosine transform, codebook, deep learning models and auto-encoder for automated surgical skills evaluation. Towards near real-time evaluation, we also investigate the effect of time interval on the classification accuracy and efficiency. RESULTS We validate the study on the benchmark robotic surgical training JIGSAWS dataset. An accuracy of 95.63, 90.17 and 90.26% by the Principal Component Analysis and 96.84, 92.75 and 95.36% by the deep Convolutional Neural Network for suturing, knot tying and needle passing, respectively, highlighted the effectiveness of these two techniques in extracting the most discriminative features among different surgical skill levels. CONCLUSIONS This study contributes toward the development of an online automated and efficient surgical skills assessment technique.
Collapse
Affiliation(s)
- Nguyen Xuan Anh
- Department of Mechanical and Aerospace Engineering, Monash University, Melbourne, Australia
| | - Ramesh Mark Nataraja
- Department of Surgical Simulation, Monash Children's Hospital, Melbourne, Australia
| | - Sunita Chauhan
- Department of Mechanical and Aerospace Engineering, Monash University, Melbourne, Australia.
| |
Collapse
|
30
|
Farcas MA, Azzie G. Performance assessment - The knowledge, skills and attitudes of surgical performance. Semin Pediatr Surg 2020; 29:150903. [PMID: 32423592 DOI: 10.1016/j.sempedsurg.2020.150903] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
Abstract
Pediatric surgical education has traditionally focused on the hard cognitive and psychomotor (technical) skills. While more and more attention is being paid to softer skills such as communication, collaboration, leadership, health advocacy, professionalism and scholarship, the bulk of curricula remain focused on the hard skills. An intricate part of education is the assessment of performance. This article reviews the current literature on the assessment of cognitive and psychomotor skills, focusing particularly on the assessment of technical skills in the realm of simulation.
Collapse
Affiliation(s)
- Monica A Farcas
- Department of Surgery, University of Toronto, Toronto, Canada; Division of Urology, St. Michael's Hospital, Toronto, Canada; Ki Ka Shing Knowledge Institute, Toronto, Canada; Institute for Biomedical Engineering, Science and Technology, Toronto, Canada.
| | - Georges Azzie
- Department of Surgery, University of Toronto, Toronto, Canada; Department of Pediatric Surgery, Hopitaux Pédiatriques de Nice CHU-Lenval, Canada; Division of General and Thoracic Surgery, Hospital for Sick Children, Toronto, Canada
| |
Collapse
|
31
|
Translating motion tracking data into resident feedback: An opportunity for streamlined video coaching. Am J Surg 2020; 219:552-556. [DOI: 10.1016/j.amjsurg.2020.01.032] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2019] [Revised: 11/30/2019] [Accepted: 01/19/2020] [Indexed: 11/21/2022]
|
32
|
Prebay ZJ, Peabody JO, Miller DC, Ghani KR. Video review for measuring and improving skill in urological surgery. Nat Rev Urol 2020; 16:261-267. [PMID: 30622365 DOI: 10.1038/s41585-018-0138-2] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Interest is growing within the urological surgery community for objective assessments of technical skill. Surgical video review relies on the use of objective assessment tools to evaluate both global and procedure-specific skill. These evaluations provide structured feedback to surgeons with the aim of improving technique, which has been associated with patient outcomes. Currently, skill assessments can be performed by using expert peer-review, crowdsourcing or computer-based methods. Given the relationship between skill and patient outcomes, surgeons might be required in the future to provide empirical evidence of their technical skill for certification, employment, credentialing and quality improvement. Interventions such as coaching and skills workshops incorporating video review might help surgeons improve their skill, with the ultimate goal of improving patient outcomes.
Collapse
Affiliation(s)
- Zachary J Prebay
- School of Medicine, Medical College of Wisconsin, Milwaukee, WI, USA
| | - James O Peabody
- Center for Outcomes Research, Analytics and Evaluation, Vattikuti Urology Institute, Henry Ford Hospital, Detroit, MI, USA
| | - David C Miller
- Department of Urology, University of Michigan, Ann Arbor, MI, USA
| | - Khurshid R Ghani
- Department of Urology, University of Michigan, Ann Arbor, MI, USA.
| |
Collapse
|
33
|
Nguyen XA, Ljuhar D, Pacilli M, Nataraja RM, Chauhan S. Surgical skill levels: Classification and analysis using deep neural network model and motion signals. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2019; 177:1-8. [PMID: 31319938 DOI: 10.1016/j.cmpb.2019.05.008] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/06/2019] [Revised: 04/11/2019] [Accepted: 05/11/2019] [Indexed: 06/10/2023]
Abstract
BACKGROUND AND OBJECTIVES Currently, the assessment of surgical skills relies primarily on the observations of expert surgeons. This may be time-consuming, non-scalable, inconsistent and subjective. Therefore, an automated system that can objectively identify the actual skills level of a junior trainee is highly desirable. This study aims to design an automated surgical skills evaluation system. METHODS We propose to use a deep neural network model that can analyze raw surgical motion data with minimal preprocessing. A platform with inertial measurement unit sensors was developed and participants with different levels of surgical experience were recruited to perform core open surgical skills tasks. JIGSAWS a publicly available robot based surgical training dataset was used to evaluate the generalization of our deep network model. 15 participants (4 experts, 4 intermediates and 7 novices) were recruited into the study. RESULTS The proposed deep model achieved an accuracy of 98.2%. With comparison to JIGSAWS; our method outperformed some existing approaches with an accuracy of 98.4%, 98.4% and 94.7% for suturing, needle-passing, and knot-tying, respectively. The experimental results demonstrated the applicability of this method in both open surgery and robot-assisted minimally invasive surgery. CONCLUSIONS This study demonstrated the potential ability of the proposed deep network model to learn the discriminative features between different surgical skills levels.
Collapse
Affiliation(s)
- Xuan Anh Nguyen
- Department of Mechanical and Aerospace Engineering, Monash University, Clayton, Victoria, 3800, Australia
| | - Damir Ljuhar
- Department of Surgical Simulation, Monash Children's Hospital, Melbourne, Australia; Department of Paediatrics, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Maurizio Pacilli
- Department of Surgical Simulation, Monash Children's Hospital, Melbourne, Australia; Department of Paediatrics, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Ramesh Mark Nataraja
- Department of Surgical Simulation, Monash Children's Hospital, Melbourne, Australia; Department of Paediatrics, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Sunita Chauhan
- Department of Mechanical and Aerospace Engineering, Monash University, Clayton, Victoria, 3800, Australia.
| |
Collapse
|
34
|
Walsh CM, Cohen J, Woods KL, Wang KK, Andersen DK, Anderson MA, Dunkin BJ, Edmundowicz SA, Faigel DO, Law JK, Marks JM, Sedlack RE, Thompson CC, Vargo JJ. ASGE EndoVators Summit: simulators and the future of endoscopic training. Gastrointest Endosc 2019; 90:13-26. [PMID: 31122744 DOI: 10.1016/j.gie.2018.10.031] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/23/2018] [Accepted: 10/24/2018] [Indexed: 02/07/2023]
Abstract
Interest in the use of simulation for acquiring, maintaining, and assessing skills in GI endoscopy has grown over the past decade, as evidenced by recent American Society for Gastrointestinal Endoscopy (ASGE) guidelines encouraging the use of endoscopy simulation training and its incorporation into training standards by a key accreditation organization. An EndoVators Summit, partially supported by a grant from the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) of the National Institutes of Health, (NIH) was held at the ASGE Institute for Training and Technology from November 19 to 20, 2017. The summit brought together over 70 thought leaders in simulation research and simulator development and key decision makers from industry. Proceedings opened with a historical review of the role of simulation in medicine and an outline of priority areas related to the emerging role of simulation training within medicine broadly. Subsequent sessions addressed the summit's purposes: to review the current state of endoscopy simulation and the role it could play in endoscopic training, to define the role and value of simulators in the future of endoscopic training and to reach consensus regarding priority areas for simulation-related education and research and simulator development. This white paper provides an overview of the central points raised by presenters, synthesizes the discussions on the key issues under consideration, and outlines actionable items and/or areas of consensus reached by summit participants and society leadership pertinent to each session. The goal was to provide a working roadmap for the developers of simulators, the investigators who strive to define the optimal use of endoscopy-related simulation and assess its impact on educational outcomes and health care quality, and the educators who seek to enhance integration of simulation into training and practice.
Collapse
Affiliation(s)
- Catharine M Walsh
- Division of Gastroenterology, Hepatology, and Nutrition, the Research and Learning Institutes, Hospital for Sick Children and the Wilson Centre, University of Toronto, Toronto, Ontario, Canada.
| | - Jonathan Cohen
- School of Medicine, New York University Langone Health, New York, New York
| | - Karen L Woods
- Houston Methodist Gastroenterology Associates, Underwood Center for Digestive Disorders, Houston Methodist Hospital, Houston, Texas
| | - Kenneth K Wang
- Department of Gastroenterology and Hepatology, Mayo Clinic, Rochester, Minnesota
| | - Dana K Andersen
- Division of Digestive Diseases and Nutrition, National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health, Bethesda, Maryland
| | - Michelle A Anderson
- Division of Gastroenterology and Hepatology, University of Michigan, Ann Arbor, Michigan
| | - Brian J Dunkin
- Institute for Technology, Innovation, and Education, Houston Methodist Hospital, Houston, Texas
| | - Steven A Edmundowicz
- Division of Gastroenterology and Hepatology, University of Colorado School of Medicine, Aurora, Colorado
| | - Douglas O Faigel
- Division of Gastroenterology and Hepatology, Mayo Clinic Arizona, Scottsdale, Arizona
| | - Joanna K Law
- Digestive Disease Institute, Virginia Mason Hospital and Medical Center, Seattle, Washington
| | - Jeffrey M Marks
- Department of Surgery, University Hospitals Cleveland Medical Center, Cleveland, Ohio
| | - Robert E Sedlack
- Department of Gastroenterology and Hepatology, Mayo Clinic, Rochester, Minnesota
| | - Christopher C Thompson
- Division of Gastroenterology, Hepatology, and Endoscopy, Brigham and Women's Hospital, Boston, Massachusetts
| | - John J Vargo
- Department of Gastroenterology, Hepatology, and Nutrition, Cleveland Clinic, Cleveland, Ohio
| |
Collapse
|
35
|
Bilgic E, Alyafi M, Hada T, Landry T, Fried GM, Vassiliou MC. Simulation platforms to assess laparoscopic suturing skills: a scoping review. Surg Endosc 2019; 33:2742-2762. [PMID: 31089881 DOI: 10.1007/s00464-019-06821-y] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2018] [Accepted: 05/03/2019] [Indexed: 11/30/2022]
Abstract
BACKGROUND Laparoscopic suturing (LS) has become a common technique used in a variety of advanced laparoscopic procedures. However, LS is a challenging skill to master, and many trainees may not be competent in performing LS at the end of their training. The purpose of this review is to identify simulation platforms available for assessment of LS skills, and determine the characteristics of the platforms and the LS skills that are targeted. METHODS A scoping review was conducted between January 1997 and October 2018 for full-text articles. The search was done in various databases. Only articles written in English or French were included. Additional studies were identified through reference lists. The search terms included "laparoscopic suturing" and "clinical competence." RESULTS Sixty-two studies were selected. The majority of the simulation platforms were box trainers with inanimate tissue, and targeted basic suturing and intracorporeal knot-tying techniques. Most of the validation came from internal structure (rater reliability) and relationship to other variables (compare training levels/case experience, and various metrics). Consequences were not addressed in any of the studies. CONCLUSION We identified many types of simulation platforms that were used for assessing LS skills, with most being for assessment of basic skills. Platforms assessing the competence of trainees for advanced LS skills were limited. Therefore, future research should focus on development of LS tasks that better reflect the needs of the trainees.
Collapse
Affiliation(s)
- Elif Bilgic
- Steinberg Centre for Simulation and Interactive Learning, McGill University, Montreal, QC, Canada.,Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650, Cedar Avenue, L9. 313, Montreal, QC, H3G 1A4, Canada
| | - Motaz Alyafi
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650, Cedar Avenue, L9. 313, Montreal, QC, H3G 1A4, Canada
| | - Tomonori Hada
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650, Cedar Avenue, L9. 313, Montreal, QC, H3G 1A4, Canada
| | - Tara Landry
- Montreal General Hospital Medical Library, McGill University Health Centre, Montreal, QC, Canada
| | - Gerald M Fried
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650, Cedar Avenue, L9. 313, Montreal, QC, H3G 1A4, Canada
| | - Melina C Vassiliou
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650, Cedar Avenue, L9. 313, Montreal, QC, H3G 1A4, Canada.
| |
Collapse
|
36
|
Yeo CT, Ring J, Holden MS, Ungi T, Toprak A, Fichtinger G, Zevin B. Surgery Tutor for Computational Assessment of Technical Proficiency in Soft-Tissue Tumor Resection in a Simulated Setting. JOURNAL OF SURGICAL EDUCATION 2019; 76:872-880. [PMID: 30567671 DOI: 10.1016/j.jsurg.2018.11.005] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/17/2018] [Revised: 10/19/2018] [Accepted: 11/18/2018] [Indexed: 06/09/2023]
Abstract
BACKGROUND In competency-based medical education, progression between milestones requires reliable and valid methods of assessment. Surgery Tutor is an open-source motion tracking platform developed to objectively assess technical proficiency during open soft-tissue tumor resections in a simulated setting. The objective of our study was to provide evidence in support of construct validity of the scores obtained by Surgery Tutor. We hypothesized that Surgery Tutor would discriminate between novice, intermediate, and experienced operators. METHODS Thirty participants were assigned to novice, intermediate, or experienced groups, based on the number of prior soft-tissue resections performed. Each participant resected 2 palpable and 2 nonpalpable lesions from a soft-tissue phantom. Surgery Tutor was used to track hand and instrument motions, number of tumor breaches, and time to perform each resection. Mass of excised specimens and margin status were also recorded. RESULTS Surgery Tutor scores demonstrated "moderate" to "good" internal structure (test-retest reliability) for novice, intermediate, and experienced groups (interclass correlation coefficient = 0.596, 0.569, 0.737; p < 0.001). Evidence in support of construct validity (consequences) was demonstrated by comparing scores of novice, intermediate, and experienced participantsfor number of hand and instrument motions (690 ± 190, 597 ± 169, 469 ± 110; p < 0.001), number of tumor breaches (29 ± 34, 16 ± 11, 9 ± 6; p < 0.001), time per resection (677 ± 331 seconds, 561 ± 210 seconds, 449 ± 148 seconds; p < 0.001), mass of completely excised specimens (22 ± 7g, 21 ± 11g, 17 ± 6 g; p = 0.035), and rate of positive margin (68%, 50%, 28%; p < 0.001). There was "strong" and "moderate" relationships between motion scores and Objective Structured Assessment of Technical Skill scores, and time per resection and Objective Structured Assessment of Technical Skill scores respectively (r = -0.60, p < 0.001; r = -0.54, p < 0.001). CONCLUSION Surgery Tutor scores demonstrate evidenceof construct validity with regards to good internal structure, consequences, and relationship to other variables in the assessment of technical proficiency duringopen soft-tissue tumor resections in a simulated setting. Utilization of Surgery Tutor can provide formative feedback and objective assessment of surgical proficiency in a simulated setting.
Collapse
Affiliation(s)
- Caitlin T Yeo
- Department of Surgery, Queen's University, Kingston, Ontario, Canada.
| | - Justine Ring
- Department of Surgery, Queen's University, Kingston, Ontario, Canada
| | - Matthew S Holden
- Department of Surgery, Queen's University, Kingston, Ontario, Canada
| | - Tamas Ungi
- Department of Surgery, Queen's University, Kingston, Ontario, Canada
| | - Ayca Toprak
- Department of Surgery, Queen's University, Kingston, Ontario, Canada
| | - Gabor Fichtinger
- Department of Surgery, Queen's University, Kingston, Ontario, Canada
| | - Boris Zevin
- Department of Surgery, Queen's University, Kingston, Ontario, Canada
| |
Collapse
|
37
|
Sakakura Y, Kamei M, Sakamoto R, Morii H, Itoh-Masui A, Kawamoto E, Imai H, Miyabe M, Shimaoka M. Biomechanical profiles of tracheal intubation: a mannequin-based study to make an objective assessment of clinical skills by expert anesthesiologists and novice residents. BMC MEDICAL EDUCATION 2018; 18:293. [PMID: 30514274 PMCID: PMC6280424 DOI: 10.1186/s12909-018-1410-0] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/30/2018] [Accepted: 11/26/2018] [Indexed: 06/09/2023]
Abstract
BACKGROUND Tracheal intubation (TI) is a key medical skill used by anesthesiologists and critical care physicians in airway management in operating rooms and critical care units. An objective assessment of dexterity in TI procedures would greatly enhance the quality of medical training. This study aims to investigate whether any biomechanical parameters obtained by 3D-motion analysis of body movements during TI procedures can objectively distinguish expert anesthesiologists from novice residents. METHODS Thirteen expert anesthesiologists and thirteen residents attempted TI procedures on an airway mannequin using a Macintosh laryngoscope. Motion capturing technology was utilized to digitally record movements during TI procedures. The skill with which experts and novices measured biomechanical parameters of body motions were comparatively examined. RESULTS The two groups showed similar outcomes (success rates and mean time needed to complete the TI procedures) as well as similar mean absolute velocity values in all 21 body parts examined. However, the experts exhibited significantly lower mean absolute acceleration values at the head and the left hand than the residents. In addition, the mean-absolute-jerk measurement revealed that the experts commanded potentially smoother motions at the head and the left hand. The Receiver Operating Characteristic (ROC) curves analysis demonstrated that mean-absolute-acceleration and -jerk measurements provide excellent measures for discriminating between experts and novices. CONCLUSIONS Biomechanical parameter measurements could be used as a means to objectively assess dexterity in TI procedures. Compared with novice residents, expert anesthesiologists possess a better ability to control their body movements during TI procedures, displaying smoother motions at the selected body parts.
Collapse
Affiliation(s)
- Yousuke Sakakura
- Department of Clinical Anesthesiology, Mie University Graduate School of Medicine, 2-174 Edobashi, Tsu-City, Mie 514-8507 Japan
| | - Masataka Kamei
- Department of Clinical Anesthesiology, Mie University Graduate School of Medicine, 2-174 Edobashi, Tsu-City, Mie 514-8507 Japan
| | - Ryota Sakamoto
- Center for Information Technology and Public Relations, Mie University Hospital, 2-174 Edobashi, Tsu-City, Mie 514-8507 Japan
| | - Hideyuki Morii
- Department of Mechanical Engineering, Mie University Graduate School of Engineering, 2-174 Edobashi, Tsu-City, Mie 514-8507 Japan
| | - Asami Itoh-Masui
- Department of Emergency and Disaster Medicine, Mie University Graduate School of Medicine, 2-174 Edobashi, Tsu-City, Mie 514-8507 Japan
| | - Eiji Kawamoto
- Department of Emergency and Disaster Medicine, Mie University Graduate School of Medicine, 2-174 Edobashi, Tsu-City, Mie 514-8507 Japan
| | - Hiroshi Imai
- Department of Emergency and Disaster Medicine, Mie University Graduate School of Medicine, 2-174 Edobashi, Tsu-City, Mie 514-8507 Japan
| | - Masayuki Miyabe
- Department of Clinical Anesthesiology, Mie University Graduate School of Medicine, 2-174 Edobashi, Tsu-City, Mie 514-8507 Japan
| | - Motomu Shimaoka
- Department of Molecular Pathobiology, Mie University Graduate School of Medicine, 2-174 Edobashi, Tsu-City, Mie 514-8507 Japan
| |
Collapse
|
38
|
Cox ML, Risucci DA, Gilmore BF, Nag UP, Turner MC, Sprinkle SR, Migaly J, Sudan R. Validation of the Omni: A Novel, Multimodality, and Longitudinal Surgical Skills Assessment. JOURNAL OF SURGICAL EDUCATION 2018; 75:e218-e228. [PMID: 30522827 PMCID: PMC10765322 DOI: 10.1016/j.jsurg.2018.10.012] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/02/2018] [Revised: 10/17/2018] [Accepted: 10/21/2018] [Indexed: 06/09/2023]
Abstract
OBJECTIVE The breadth of technical skills included in general surgery training continues to expand. The current competency-based training model requires assessment tools to measure acquisition, learning, and mastery of technical skill longitudinally in a reliable and valid manner. This study describes a novel skills assessment tool, the Omni, which evaluates performance in a broad range of skills over time. DESIGN The 5 Omni tasks, consisting of open bowel anastomosis, knot tying, laparoscopic clover pattern cut, robotic needle drive, and endoscopic bubble pop, were developed by general surgery faculty. Component performance metrics assessed speed, accuracy, and quality, which were scaled into an overall score ranging from 0 to 10 for each task. For each task, ANOVAs with Scheffé's post hoc comparisons and Pearson's chi-squared tests compared performance between 6 resident cohorts (clinical years (CY1-5) and research fellows (RF)). Paired samples t-tests evaluated changes in performance across academic years. Cronbach's alpha coefficient determined the internal consistency of the Omni as an overall assessment. SETTING The Omni was developed by the Department of Surgery at Duke University. Annual assessment and this research study took place in the Surgical Education and Activities Lab. PARTICIPANTS All active general surgery residents in 2 consecutive academic years spanning 2015 to 2017. RESULTS A total of 62 general surgery residents completed the Omni and 39 (67.2%) of those residents completed the assessment in 2 consecutive years. Based on data from all residents' first assessment, statistically significant differences (p < 0.05) were observed among CY cohorts for bowel anastomosis, robotic, and laparoscopic task metrics. By pair-wise comparisons, mean bowel anastomosis scores distinguished CY1 from CY3-5 and CY2 from CY5. Mean robotic scores distinguished CY1 from RF, and mean laparoscopic scores distinguished CY1 from RF, CY3, and CY5 in addition to CY2 from CY3. Mean scores in performance on the knot tying and endoscopic tasks were not significantly different. Statistically significant improvement in mean scores was observed for all tasks from year 1 to year 2 (all p < 0.02). The internal consistency analysis revealed an alpha coefficient of 0.656. CONCLUSIONS The Omni is a novel composite assessment tool for surgical technical skill that utilizes objective measures and scoring algorithms to evaluate performance. In this pilot study, 3 tasks demonstrated discriminative ability of performance by CY, and all 5 tasks demonstrated construct validity by showing longitudinal improvement in performance. Additionally, the Omni has adequate internal consistency for a formative assessment. These results suggest the Omni holds promise for the evaluation of resident technical skill and early identification of outliers requiring intervention.
Collapse
Key Words
- ABS, American Board of Surgery
- ACS, American College of Surgeons
- APDS, Association of Program Directors in Surgery
- CY, clinical year
- FES, Fundamentals of Endoscopic Surgery
- FLS, Fundamentals of Laparoscopic Surgery
- General surgery
- Medical Knowledge
- OSATS, Objective Structured Assessment of Technical Skills
- Omni
- Patient Care
- Practice-Based Learning and Improvement
- REDCap, Research Electronic Data Capture
- RF, research fellow
- Resident
- SD, standard deviation
- Skills assessment
- df, degrees of freedom
Collapse
Affiliation(s)
- Morgan L Cox
- Department of Surgery, Duke University, Durham, North Carolina.
| | | | - Brian F Gilmore
- Department of Surgery, Duke University, Durham, North Carolina
| | - Uttara P Nag
- Department of Surgery, Duke University, Durham, North Carolina
| | - Megan C Turner
- Department of Surgery, Duke University, Durham, North Carolina
| | | | - John Migaly
- Department of Surgery, Duke University, Durham, North Carolina
| | - Ranjan Sudan
- Department of Surgery, Duke University, Durham, North Carolina
| |
Collapse
|
39
|
Fahy AS, Jamal L, Gavrilovic B, Carillo B, Gerstle JT, Nasr A, Azzie G. The Impact of Simulator Size on Forces Generated in the Performance of a Defined Intracorporeal Suturing Task: A Pilot Study. J Laparoendosc Adv Surg Tech A 2018; 28:1520-1524. [PMID: 30004827 DOI: 10.1089/lap.2018.0255] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023] Open
Abstract
Background: In pediatric minimal access surgery, the operative domain may vary from that of an adult to that of a neonate. This study aimed to quantify the impact of decreased operative domain on forces generated in the performance of a defined intracorporeal suturing task. Methods: One hundred five participants performed a defined intracorporeal suturing task in small and large simulators. Time to task completion and force analysis parameters (FAPs = total, maximum, and mean forces in X, Y, and Z axes) were measured. Expertise level was assigned based on the number of laparoscopic cases. Outcomes were analyzed using paired sample t-tests, P value of <.05. Results: Time to task completion varied significantly for experts between adult and pediatric simulators but not for intermediates or novices. Total, maximum, and mean forces in the X ("side to side") axis were significantly greater in the larger laparoscopic simulator for all levels of expertise. In the Y axis ("in and out" movement) and Z axis ("up and down" movement), total and mean forces were higher in the adult simulator regardless of the level of expertise. Differences in maximum force between the adult and pediatric simulators in the Z axis ("up and down" movement) varied significantly for novices and intermediates but not for experts. Conclusion: Forces were greater, particularly in the side-to-side plane, in the larger simulator for participants of all levels in the performance of this defined intracorporeal suturing task. Further analysis will determine the reasons for and implications of the increased force parameters in the simulator of larger domain.
Collapse
Affiliation(s)
- Aodhnait S Fahy
- 1 Division of General and Thoracic Surgery, Hospital for Sick Children, Toronto, Canada
| | - Luai Jamal
- 1 Division of General and Thoracic Surgery, Hospital for Sick Children, Toronto, Canada
| | - Bojan Gavrilovic
- 2 Institute of Biomaterials and Biomedical Engineering, University of Toronto, Toronto, Canada
| | | | - Justin T Gerstle
- 1 Division of General and Thoracic Surgery, Hospital for Sick Children, Toronto, Canada
| | - Ahmed Nasr
- 3 Division of General and Thoracic Surgery, Children's Hospital of Eastern Ontario, University of Ottawa, Ottawa, Canada
| | - Georges Azzie
- 1 Division of General and Thoracic Surgery, Hospital for Sick Children, Toronto, Canada
| |
Collapse
|
40
|
Kholinne E, Gandhi MJ, Adikrishna A, Hong H, Kim H, Hong J, Jeon IH. The Dimensionless Squared Jerk: An Objective Parameter That Improves Assessment of Hand Motion Analysis during Simulated Shoulder Arthroscopy. BIOMED RESEARCH INTERNATIONAL 2018; 2018:7816160. [PMID: 30105247 PMCID: PMC6076914 DOI: 10.1155/2018/7816160] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/14/2018] [Revised: 06/04/2018] [Accepted: 06/19/2018] [Indexed: 12/26/2022]
Abstract
PURPOSE Attempts to quantify hand movements of surgeons during arthroscopic surgery faced limited progress beyond motion analysis of hands and/or instruments. Surrogate markers such as procedure time have been used. The dimensionless squared jerk (DSJ) is a measure of deliberate hand movements. This study tests the ability of DSJ to differentiate novice and expert surgeons (construct validity) whilst performing simulated arthroscopic shoulder surgical tasks. METHODS Six residents (novice group) and six consultants (expert group) participated in this study. Participants performed three validated tasks sequentially under the same experimental setup (one performance). Each participant had ten performances assessed. Hand movements were recorded with optical tracking system. The DSJ, time taken, total path length, multiple measures of acceleration, and number of movements were recorded. RESULTS There were significant differences between novices and experts when assessed using time, number of movements with average and minimal acceleration threshold, and DSJ. No significant differences were observed in maximum acceleration, total path length, and number of movements with 10m/s2 acceleration threshold. CONCLUSION DSJ is an objective parameter that can differentiate novice and expert surgeons' simulated arthroscopic performances. We propose DSJ as an adjunct to more conventional parameters for arthroscopic surgery skills assessment.
Collapse
Affiliation(s)
- Erica Kholinne
- Department of Orthopedic Surgery, St. Carolus Hospital, Jakarta, Indonesia
- Department of Orthopedic Surgery, Asan Medical Center, College of Medicine, University of Ulsan, Seoul, Republic of Korea
| | - Maulik J. Gandhi
- Upper Limb Department, Robert Jones & Agnes Hunt Hospital, Oswestry, England, UK
| | - Arnold Adikrishna
- Department of Orthopedic Surgery, Asan Medical Center, College of Medicine, University of Ulsan, Seoul, Republic of Korea
| | - Hanpyo Hong
- Department of Orthopedic Surgery, Asan Medical Center, College of Medicine, University of Ulsan, Seoul, Republic of Korea
| | - Haewon Kim
- Department of Orthopedic Surgery, Asan Medical Center, College of Medicine, University of Ulsan, Seoul, Republic of Korea
| | - Jaesung Hong
- Department of Robotics Engineering, Daegu Gyeongbuk Institute of Science and Technology, Daegu, Republic of Korea
| | - In-Ho Jeon
- Department of Orthopedic Surgery, Asan Medical Center, College of Medicine, University of Ulsan, Seoul, Republic of Korea
| |
Collapse
|
41
|
Lin CC, Huang SC, Lin HH, Huang WJ, Chen WS, Yang SH. Naked-eye box trainer and training box games have similar training effect as conventional video-based box trainer for novices: A randomized controlled trial. Am J Surg 2018; 216:1022-1027. [PMID: 29859628 DOI: 10.1016/j.amjsurg.2018.05.015] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2017] [Revised: 04/26/2018] [Accepted: 05/06/2018] [Indexed: 11/17/2022]
Abstract
BACKGROUND Laparoscopic surgery has become a well-established technique for management of various surgical problems. A more efficient training methods are of upmost importance for current surgery residents. METHODS This is a prospective, randomized, 3-arm trial to compare the training efficient of the naked-eye box trainer, training box games and conventional video-based box trainer in training laparoscopic suturing skill. RESULTS The three training models were well acceptable and all could improve the acquisition of laparoscopic suturing and knotting skill in novices. The completion time was 604 ± 298 s in the box trainer games, 617 ± 335 s in the naked-eye training module, and 491 ± 334 s in the video-based box trainer (p = 0.322). Using the structured procedure-specific checklist, there was no significant difference in scores between these three groups (p = 0.977). CONCLUSIONS Naked-eye box trainer and training box games produce similar training effect as the conventional video-based box trainer. The naked-eye box trainer may serve as a convenient way for novice trainees to acquire laparoscopic suturing technique skills before video-based simulation.
Collapse
Affiliation(s)
- Chun-Chi Lin
- Division of Colon and Rectal Surgery, Department of Surgery, Taipei Veterans General Hospital, Taipei, Taiwan; School of Medicine, National Yang-Ming University, Taipei, Taiwan; Institute of Clinical Medicine, National Yang-Ming University, Taipei, Taiwan
| | - Sheng-Chieh Huang
- Division of Colon and Rectal Surgery, Department of Surgery, Taipei Veterans General Hospital, Taipei, Taiwan; School of Medicine, National Yang-Ming University, Taipei, Taiwan
| | - Hung-Hsin Lin
- Division of Colon and Rectal Surgery, Department of Surgery, Taipei Veterans General Hospital, Taipei, Taiwan; School of Medicine, National Yang-Ming University, Taipei, Taiwan
| | - William J Huang
- Department of Urology, Taipei Veterans General Hospital, Taipei, Taiwan; Department of Urology and Physiology, School of Medicine, Shu-Tien Urological Research Center, National Yang-Ming University, Taiwan
| | - Wei-Shone Chen
- Division of Colon and Rectal Surgery, Department of Surgery, Taipei Veterans General Hospital, Taipei, Taiwan; School of Medicine, National Yang-Ming University, Taipei, Taiwan
| | - Shung-Haur Yang
- Division of Colon and Rectal Surgery, Department of Surgery, Taipei Veterans General Hospital, Taipei, Taiwan; School of Medicine, National Yang-Ming University, Taipei, Taiwan.
| |
Collapse
|
42
|
Buescher JF, Mehdorn AS, Neumann PA, Becker F, Eichelmann AK, Pankratius U, Bahde R, Foell D, Senninger N, Rijcken E. Effect of Continuous Motion Parameter Feedback on Laparoscopic Simulation Training: A Prospective Randomized Controlled Trial on Skill Acquisition and Retention. JOURNAL OF SURGICAL EDUCATION 2018; 75:516-526. [PMID: 28864265 DOI: 10.1016/j.jsurg.2017.08.015] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/02/2017] [Revised: 08/06/2017] [Accepted: 08/06/2017] [Indexed: 06/07/2023]
Abstract
OBJECTIVE To investigate the effect of motion parameter feedback on laparoscopic basic skill acquisition and retention during a standardized box training curriculum. DESIGN A Lap-X Hybrid laparoscopic simulator was designed to provide individual and continuous motion parameter feedback in a dry box trainer setting. In a prospective controlled trial, surgical novices were randomized into 2 groups (regular box group, n = 18, and Hybrid group, n = 18) to undergo an identical 5-day training program. In each group, 7 standardized tasks on laparoscopic basic skills were completed twice a day on 4 consecutive days in fixed pairs. Additionally, each participant performed a simulated standard laparoscopic cholecystectomy before (day 1) and after training (day 5) on a LAP Mentor II virtual reality (VR) trainer, allowing an independent control of skill progress in both groups. A follow-up assessment of skill retention was performed after 6 weeks with repetition of both the box tasks and VR cholecystectomy. SETTING Muenster University Hospital Training Center, Muenster, Germany. PARTICIPANTS Medical students without previous surgical experience. RESULTS Laparoscopic skills in both groups improved significantly during the training period, measured by the overall task performance time. The 6 week follow-up showed comparable skill retention in both groups. Evaluation of the VR cholecystectomies demonstrated significant decrease of operation time (p < 0.01), path length of the left and right instrument, and the number of movements of the left and right instruments for the Hybrid group (all p < 0.001), compared to the box group. Similar results were found at the assessment of skill retention. CONCLUSION Simulation training on both trainers enables reliable acquisition of laparoscopic basic skills. Furthermore, individual and continuous motion feedback improves laparoscopic skill enhancement significantly in several aspects. Thus, training systems with feedback of motion parameters should be considered to achieve long-term improvement of motion economy among surgical trainees.
Collapse
Affiliation(s)
| | - Anne-Sophie Mehdorn
- Department of General and Visceral Surgery, University Hospital Muenster, Muenster, Germany
| | | | - Felix Becker
- Department of General and Visceral Surgery, University Hospital Muenster, Muenster, Germany
| | - Ann-Kathrin Eichelmann
- Department of General and Visceral Surgery, University Hospital Muenster, Muenster, Germany
| | - Ulrich Pankratius
- Department of General and Visceral Surgery, University Hospital Muenster, Muenster, Germany
| | - Ralf Bahde
- Department of General and Visceral Surgery, University Hospital Muenster, Muenster, Germany
| | - Daniel Foell
- Department of General and Visceral Surgery, University Hospital Muenster, Muenster, Germany
| | - Norbert Senninger
- Department of General and Visceral Surgery, University Hospital Muenster, Muenster, Germany
| | - Emile Rijcken
- Department of General and Visceral Surgery, University Hospital Muenster, Muenster, Germany.
| |
Collapse
|
43
|
Harris DJ, Vine SJ, Wilson MR, McGrath JS, LeBel ME, Buckingham G. The effect of observing novice and expert performance on acquisition of surgical skills on a robotic platform. PLoS One 2017; 12:e0188233. [PMID: 29141046 PMCID: PMC5687728 DOI: 10.1371/journal.pone.0188233] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2017] [Accepted: 11/02/2017] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND Observational learning plays an important role in surgical skills training, following the traditional model of learning from expertise. Recent findings have, however, highlighted the benefit of observing not only expert performance but also error-strewn performance. The aim of this study was to determine which model (novice vs. expert) would lead to the greatest benefits when learning robotically assisted surgical skills. METHODS 120 medical students with no prior experience of robotically-assisted surgery completed a ring-carrying training task on three occasions; baseline, post-intervention and at one-week follow-up. The observation intervention consisted of a video model performing the ring-carrying task, with participants randomly assigned to view an expert model, a novice model, a mixed expert/novice model or no observation (control group). Participants were assessed for task performance and surgical instrument control. RESULTS There were significant group differences post-intervention, with expert and novice observation groups outperforming the control group, but there were no clear group differences at a retention test one week later. There was no difference in performance between the expert-observing and error-observing groups. CONCLUSIONS Similar benefits were found when observing the traditional expert model or the error-strewn model, suggesting that viewing poor performance may be as beneficial as viewing expertise in the early acquisition of robotic surgical skills. Further work is required to understand, then inform, the optimal curriculum design when utilising observational learning in surgical training.
Collapse
Affiliation(s)
- David J. Harris
- Department of Sport and Health Sciences, University of Exeter, Exeter, United Kingdom
| | - Samuel J. Vine
- Department of Sport and Health Sciences, University of Exeter, Exeter, United Kingdom
| | - Mark R. Wilson
- Department of Sport and Health Sciences, University of Exeter, Exeter, United Kingdom
| | - John S. McGrath
- Exeter Surgical Health Services Research Unit, RD&E Hospital, Exeter, United Kingdom
- University of Exeter Medical School, Exeter, United Kingdom
| | - Marie-Eve LeBel
- Division of Orthopaedic Surgery, University of Western Ontario, London, Canada
| | - Gavin Buckingham
- Department of Sport and Health Sciences, University of Exeter, Exeter, United Kingdom
| |
Collapse
|
44
|
Oquendo YA, Riddle EW, Hiller D, Blinman TA, Kuchenbecker KJ. Automatically rating trainee skill at a pediatric laparoscopic suturing task. Surg Endosc 2017; 32:1840-1857. [PMID: 29071419 PMCID: PMC5845064 DOI: 10.1007/s00464-017-5873-6] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2017] [Accepted: 09/04/2017] [Indexed: 12/29/2022]
Abstract
BACKGROUND Minimally invasive surgeons must acquire complex technical skills while minimizing patient risk, a challenge that is magnified in pediatric surgery. Trainees need realistic practice with frequent detailed feedback, but human grading is tedious and subjective. We aim to validate a novel motion-tracking system and algorithms that automatically evaluate trainee performance of a pediatric laparoscopic suturing task. METHODS Subjects (n = 32) ranging from medical students to fellows performed two trials of intracorporeal suturing in a custom pediatric laparoscopic box trainer after watching a video of ideal performance. The motions of the tools and endoscope were recorded over time using a magnetic sensing system, and both tool grip angles were recorded using handle-mounted flex sensors. An expert rated the 63 trial videos on five domains from the Objective Structured Assessment of Technical Skill (OSATS), yielding summed scores from 5 to 20. Motion data from each trial were processed to calculate 280 features. We used regularized least squares regression to identify the most predictive features from different subsets of the motion data and then built six regression tree models that predict summed OSATS score. Model accuracy was evaluated via leave-one-subject-out cross-validation. RESULTS The model that used all sensor data streams performed best, achieving 71% accuracy at predicting summed scores within 2 points, 89% accuracy within 4, and a correlation of 0.85 with human ratings. 59% of the rounded average OSATS score predictions were perfect, and 100% were within 1 point. This model employed 87 features, including none based on completion time, 77 from tool tip motion, 3 from tool tip visibility, and 7 from grip angle. CONCLUSIONS Our novel hardware and software automatically rated previously unseen trials with summed OSATS scores that closely match human expert ratings. Such a system facilitates more feedback-intensive surgical training and may yield insights into the fundamental components of surgical skill.
Collapse
Affiliation(s)
- Yousi A Oquendo
- Department of Mechanical Engineering & Applied Mechanics, University of Pennsylvania, Philadelphia, USA.,Department of Computer & Information Science, University of Pennsylvania, Philadelphia, USA
| | - Elijah W Riddle
- Division of Pediatric General, Thoracic and Fetal Surgery, Children's Hospital of Philadelphia, Philadelphia, USA
| | - Dennis Hiller
- Division of Pediatric General, Thoracic and Fetal Surgery, Children's Hospital of Philadelphia, Philadelphia, USA
| | - Thane A Blinman
- Division of Pediatric General, Thoracic and Fetal Surgery, Children's Hospital of Philadelphia, Philadelphia, USA
| | - Katherine J Kuchenbecker
- Department of Mechanical Engineering & Applied Mechanics, University of Pennsylvania, Philadelphia, USA. .,Department of Computer & Information Science, University of Pennsylvania, Philadelphia, USA. .,Haptic Intelligence Department, Max Planck Institute for Intelligent Systems, Heisenbergstr. 3, 70569, Stuttgart, Germany.
| |
Collapse
|
45
|
Abstract
Simulation-based surgical skills training addresses several concerns associated with the traditional apprenticeship model, including patient safety, efficient acquisition of complex skills, and cost. The surgical specialties already recognize the advantages of surgical training using simulation, and simulation-based methods are appearing in surgical education and assessment for board certification. The necessity of simulation-based methods in surgical education along with valid, objective, standardized techniques for measuring learned skills using simulators has become apparent. The most commonly used surgical skill measurement techniques in simulation-based training include questionnaires and post-training surveys, objective structured assessment of technical skills and global rating scale of performance scoring systems, structured assessments using video recording, and motion tracking software. The literature shows that the application of many of these techniques varies based on investigator preference and the convenience of the technique. As simulators become more accepted as a teaching tool, techniques to measure skill proficiencies will need to be standardized nationally and internationally.
Collapse
|
46
|
Comparison of the goals and MISTELS scores for the evaluation of surgeons on training benches. Int J Comput Assist Radiol Surg 2017; 13:95-103. [DOI: 10.1007/s11548-017-1645-y] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2017] [Accepted: 07/10/2017] [Indexed: 12/23/2022]
|
47
|
Tracking Motion Devices as Assessment Tools in Anesthesia Procedures: Have We Been Using Them Well? CAN J EMERG MED 2017. [DOI: 10.1017/cem.2017.337] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
|
48
|
Poursartip B, LeBel ME, Patel RV, Naish MD, Trejos AL. Analysis of Energy-Based Metrics for Laparoscopic Skills Assessment. IEEE Trans Biomed Eng 2017; 65:1532-1542. [PMID: 28541193 DOI: 10.1109/tbme.2017.2706499] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
OBJECTIVE The complexity of minimally invasive surgery (MIS) requires that trainees practice MIS skills in numerous training sessions. The goal of these training sessions is to learn how to move the instruments smoothly without damaging the surrounding tissue and achieving operative tasks with accuracy. In order to enhance the efficiency of these training sessions, the proficiency of the trainees should be assessed using an objective assessment method. Several performance metrics have been proposed and analyzed for MIS tasks. The differentiation of various levels of expertise is limited without the presence of an external evaluator. METHODS In this study, novel objective performance metrics are proposed based on mechanical energy expenditure and work. The three components of these metrics are potential energy, kinetic energy, and work. These components are optimally combined through both one-step and two-step methods. Evaluation of these metrics is accomplished for suturing and knot-tying tasks based on the performance of 30 subjects across four levels of experience. RESULTS The results of this study show that the one-step combined metric provides 47 and 60 accuracy in determining the level of expertise of subjects for the suturing and knot-tying tasks, respectively. The two-step combined metric provided 67 accuracy for both of the tasks studied. CONCLUSION The results indicate that energy expenditure is a useful metric for developing objective and efficient assessment methods. SIGNIFICANCE These metrics can be used to evaluate and determine the proficiency levels of trainees, provide feedback and, consequently, enhance surgical simulators.
Collapse
|
49
|
Våpenstad C, Hofstad EF, Bø LE, Kuhry E, Johnsen G, Mårvik R, Langø T, Hernes TN. Lack of transfer of skills after virtual reality simulator training with haptic feedback. MINIM INVASIV THER 2017; 26:346-354. [DOI: 10.1080/13645706.2017.1319866] [Citation(s) in RCA: 30] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Cecilie Våpenstad
- Faculty of Medicine, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
- Department of Medical Technology, SINTEF Technology and Society, Trondheim, Norway
- The Central Norway Regional Health Authority, Trondheim, Norway
- The Norwegian National Advisory Unit for Advanced Laparoscopic Surgery, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
- The Norwegian National Advisory Unit for Ultrasound and Image-Guided Therapy, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - Erlend Fagertun Hofstad
- Department of Medical Technology, SINTEF Technology and Society, Trondheim, Norway
- The Norwegian National Advisory Unit for Advanced Laparoscopic Surgery, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
- The Norwegian National Advisory Unit for Ultrasound and Image-Guided Therapy, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - Lars Eirik Bø
- Faculty of Medicine, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
- Department of Medical Technology, SINTEF Technology and Society, Trondheim, Norway
- The Central Norway Regional Health Authority, Trondheim, Norway
- The Norwegian National Advisory Unit for Advanced Laparoscopic Surgery, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
- The Norwegian National Advisory Unit for Ultrasound and Image-Guided Therapy, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - Esther Kuhry
- Faculty of Medicine, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
- Department of Gastrointestinal Surgery, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - Gjermund Johnsen
- The Norwegian National Advisory Unit for Advanced Laparoscopic Surgery, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
- Department of Gastrointestinal Surgery, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - Ronald Mårvik
- Faculty of Medicine, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
- The Norwegian National Advisory Unit for Advanced Laparoscopic Surgery, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
- The Norwegian National Advisory Unit for Ultrasound and Image-Guided Therapy, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
- Department of Gastrointestinal Surgery, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - Thomas Langø
- Department of Medical Technology, SINTEF Technology and Society, Trondheim, Norway
- The Norwegian National Advisory Unit for Ultrasound and Image-Guided Therapy, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - Toril Nagelhus Hernes
- Faculty of Medicine, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
- The Norwegian National Advisory Unit for Ultrasound and Image-Guided Therapy, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| |
Collapse
|
50
|
Hofstad EF, Våpenstad C, Bø LE, Langø T, Kuhry E, Mårvik R. Psychomotor skills assessment by motion analysis in minimally invasive surgery on an animal organ. MINIM INVASIV THER 2017. [DOI: 10.1080/13645706.2017.1284131] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Erlend Fagertun Hofstad
- Department of Medical Technology, SINTEF Technology and Society, Trondheim, Norway
- The Norwegian National Advisory Unit for Ultrasound and Image-Guided Therapy, St. Olavs Hospital-Trondheim University Hospital, Trondheim, Norway
| | - Cecilie Våpenstad
- Department of Medical Technology, SINTEF Technology and Society, Trondheim, Norway
- The Norwegian National Advisory Unit for Ultrasound and Image-Guided Therapy, St. Olavs Hospital-Trondheim University Hospital, Trondheim, Norway
- Department of Circulation and Medical Imaging, Faculty of Medicine, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
| | - Lars Eirik Bø
- Department of Medical Technology, SINTEF Technology and Society, Trondheim, Norway
- The Norwegian National Advisory Unit for Ultrasound and Image-Guided Therapy, St. Olavs Hospital-Trondheim University Hospital, Trondheim, Norway
- Department of Circulation and Medical Imaging, Faculty of Medicine, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
| | - Thomas Langø
- Department of Medical Technology, SINTEF Technology and Society, Trondheim, Norway
- The Norwegian National Advisory Unit for Ultrasound and Image-Guided Therapy, St. Olavs Hospital-Trondheim University Hospital, Trondheim, Norway
| | - Esther Kuhry
- Department of Cancer Research and Molecular Medicine, Faculty of Medicine, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
- Department of Surgery, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - Ronald Mårvik
- The Norwegian National Advisory Unit for Ultrasound and Image-Guided Therapy, St. Olavs Hospital-Trondheim University Hospital, Trondheim, Norway
- Department of Cancer Research and Molecular Medicine, Faculty of Medicine, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
- Department of Surgery, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
- National Center for Advanced Laparoscopic Surgery, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| |
Collapse
|