1
|
Boyajian GP, Zulbaran-Rojas A, Najafi B, Atique MMU, Loor G, Gilani R, Schutz A, Wall MJ, Coselli JS, Moon MR, Rosengart TK, Ghanta RK. Development of a Sensor Technology to Objectively Measure Dexterity for Cardiac Surgical Proficiency. Ann Thorac Surg 2024; 117:635-643. [PMID: 37517533 DOI: 10.1016/j.athoracsur.2023.07.013] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Revised: 05/04/2023] [Accepted: 07/11/2023] [Indexed: 08/01/2023]
Abstract
BACKGROUND Technical skill is essential for good outcomes in cardiac surgery. However, no objective methods exist to measure dexterity while performing surgery. The purpose of this study was to validate sensor-based hand motion analysis (HMA) of technical dexterity while performing a graft anastomosis within a validated simulator. METHODS Surgeons at various training levels performed an anastomosis while wearing flexible sensors (BioStamp nPoint, MC10 Inc) with integrated accelerometers and gyroscopes on each hand to quantify HMA kinematics. Groups were stratified as experts (n = 8) or novices (n = 18). The quality of the completed anastomosis was scored using the 10 Point Microsurgical Anastomosis Rating Scale (MARS10). HMA parameters were compared between groups and correlated with quality. Logistic regression was used to develop a predictive model from HMA parameters to distinguish experts from novices. RESULTS Experts were faster (11 ± 6 minutes vs 21 ± 9 minutes; P = .012) and used fewer movements in both dominant (340 ± 166 moves vs 699 ± 284 moves; P = .003) and nondominant (359 ± 188 moves vs 567 ± 201 moves; P = .02) hands compared with novices. Experts' anastomoses were of higher quality compared with novices (9.0 ± 1.2 MARS10 vs 4.9 ± 3.2 MARS10; P = .002). Higher anastomosis quality correlated with 9 of 10 HMA parameters, including fewer and shorter movements of both hands (dominant, r = -0.65, r = -0.46; nondominant, r = -0.58, r = -0.39, respectively). CONCLUSIONS Sensor-based HMA can distinguish technical dexterity differences between experts and novices, and correlates with quality. Objective quantification of hand dexterity may be a valuable adjunct to training and education in cardiac surgery training programs.
Collapse
Affiliation(s)
- Gregory P Boyajian
- Division of Cardiothoracic Surgery, Michael E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Alejandro Zulbaran-Rojas
- Division of Vascular Surgery and Endovascular Therapy, Michael E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Bijan Najafi
- Division of Vascular Surgery and Endovascular Therapy, Michael E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Md Moin Uddin Atique
- Division of Vascular Surgery and Endovascular Therapy, Michael E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Gabriel Loor
- Division of Cardiothoracic Surgery, Michael E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Ramyar Gilani
- Division of Vascular Surgery and Endovascular Therapy, Michael E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Alexander Schutz
- Division of Cardiothoracic Surgery, Michael E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Matthew J Wall
- Division of Cardiothoracic Surgery, Michael E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Joseph S Coselli
- Division of Cardiothoracic Surgery, Michael E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Marc R Moon
- Division of Cardiothoracic Surgery, Michael E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Todd K Rosengart
- Division of Cardiothoracic Surgery, Michael E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Ravi K Ghanta
- Division of Cardiothoracic Surgery, Michael E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas.
| |
Collapse
|
2
|
Kulik D, Bell CR, Holden MS. FAST skill assessment from kinematics data using convolutional neural networks. Int J Comput Assist Radiol Surg 2024; 19:43-49. [PMID: 37093528 DOI: 10.1007/s11548-023-02908-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Accepted: 04/03/2023] [Indexed: 04/25/2023]
Abstract
PURPOSE FAST is a point of care ultrasound study that evaluates for the presence of free fluid, typically hemoperitoneum in trauma patients. FAST is an essential skill for Emergency Physicians. Thus, it requires objective evaluation tools that can reduce the necessity of direct observation for proficiency assessment. In this work, we use deep neural networks to automatically assess operators' FAST skills. METHODS We propose a deep convolutional neural network for FAST proficiency assessment based on motion data. Prior work has shown that operators demonstrate different domain-specific dexterity metrics that can distinguish novices, intermediates, and experts. Therefore, we augment our dataset with this domain knowledge and employ fine-tuning to improve the model's classification capabilities. Our model, however, does not require specific points of interest (POIs) to be defined for scanning. RESULTS The results show that the proposed deep convolutional neural network can classify FAST proficiency with 87.5% accuracy and 0.884, 0.886, 0.247 sensitivity for novices, intermediates, and experts, respectively. It demonstrates the potential of using kinematics data as an input in FAST skill assessment tasks. We also show that the proposed domain-specific features and region fine-tuning increase the model's classification accuracy and sensitivity. CONCLUSIONS Variations in probe motion at different learning stages can be derived from kinematics data. These variations can be used for automatic and objective skill assessment without prior identification of clinical POIs. The proposed approach can improve the quality and objectivity of FAST proficiency evaluation. Furthermore, skill assessment combining ultrasound images and kinematics data can provide a more rigorous and diversified evaluation than using ultrasound images alone.
Collapse
Affiliation(s)
- Daniil Kulik
- School of Computer Science, Carleton University, 1125 Colonel By Dr, Ottawa, K1S 5B6, ON, Canada.
| | - Colin R Bell
- Department of Emergency Medicine and Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| | - Matthew S Holden
- School of Computer Science, Carleton University, 1125 Colonel By Dr, Ottawa, K1S 5B6, ON, Canada
| |
Collapse
|
3
|
Chan AHY, Lee WF, Van Gerven PWM, Chenkin J. Assessment of changes in gaze patterns during training in point-of-care ultrasound. BMC MEDICAL EDUCATION 2022; 22:658. [PMID: 36056331 PMCID: PMC9440555 DOI: 10.1186/s12909-022-03680-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Accepted: 07/05/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND Point-of-care ultrasound (POCUS) is a core skill in emergency medicine (EM), however, there is a lack of objective competency measures. Eye-tracking technology is a potentially useful assessment tool, as gaze patterns can reliably discriminate between experts and novices across medical specialties. We aim to determine if gaze metrics change in an independent and predictable manner during ultrasound training. METHODS A convenience sample of first-year residents from a single academic emergency department was recruited. Participants interpreted 16 ultrasound videos of the focused assessment with sonography for trauma (FAST) scan while their gaze patterns were recorded using a commercially available eye-tracking device. The intervention group then completed an introductory ultrasound course whereas the control group received no additional education. The gaze assessment was subsequently repeated. The primary outcome was total gaze duration on the area of interest (AOI). Secondary outcomes included time to fixation, mean duration of first fixation and mean number of fixations on the AOI. RESULTS 10 EM residents in the intervention group and 10 non-EM residents in the control group completed the study. After training, there was an 8.8 s increase in the total gaze time on the AOI in the intervention group compared to a 4.0 s decrease in the control group (p = .03). EM residents were also 3.8 s quicker to fixate on the AOI whereas the control group became 2.5 s slower (p = .04). There were no significant interactions on the number of fixations (0.43 vs. 0.18, p = .65) or duration of first fixation on the AOI (0.02 s vs. 0.06 s, p = .63). CONCLUSIONS There are significant and quantifiable changes in gaze metrics, which occur with incremental learning after an ultrasound course. Further research is needed to validate the serial use of eye-tracking technology in following a learner's progress toward competency in point-of-care ultrasound image interpretation.
Collapse
Affiliation(s)
- Alice H Y Chan
- Department of Emergency Medicine, Sunnybrook Health Sciences Center, University of Toronto, 2075 Bayview Avenue, AG245, Toronto, ON, M4N 3M5, Canada.
| | - Wei Feng Lee
- Department of Emergency Medicine, Sunnybrook Health Sciences Center, University of Toronto, 2075 Bayview Avenue, AG245, Toronto, ON, M4N 3M5, Canada
- Department of Emergency Medicine, Ng Teng Fong General Hospital, 1 Jurong East Street 21, Singapore, Singapore, 609606
| | - Pascal W M Van Gerven
- Department of Educational Development and Research, School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, Netherlands
| | - Jordan Chenkin
- Department of Emergency Medicine, Sunnybrook Health Sciences Center, University of Toronto, 2075 Bayview Avenue, AG245, Toronto, ON, M4N 3M5, Canada
| |
Collapse
|
4
|
Johnson GGRJ, Park J, Vergis A, Gillman LM, Rivard JD. Contextual interference for skills development and transfer in laparoscopic surgery: a randomized controlled trial. Surg Endosc 2022; 36:6377-6386. [PMID: 34981234 DOI: 10.1007/s00464-021-08946-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2021] [Accepted: 12/06/2021] [Indexed: 10/19/2022]
Abstract
INTRODUCTION Past education literature has shown benefits for random practice schedules (termed contextual interference) for skills retention and transfer to novel tasks. The purpose of fundamentals of laparoscopic surgery (FLS) training is to develop skills in simulation and transfer to new in vivo intraoperative experiences. The study objective was to assess whether individuals trained over a fixed number of trials in the FLS tasks would outperform untrained controls on an unpracticed previously validated bile duct cannulation task and scoring system and to determine whether random training schedules conferred any relative advantage. METHODS 44 trainees with no laparoscopic experience were recruited to participate. 35 were randomized to practice the FLS tasks using either a blocked or random training schedule. Nine were randomized to no additional training (controls). Participant performance was measured throughout training to monitor skills acquisition and were then tested on an unpracticed bile duct cannulation simulation task 4 to 6 weeks later. Outcomes included previously validated FLS scores and hand-motion analyses. RESULTS All 44 participants completed the study. Trained individuals in both groups showed significant improvements in all FLS tasks after training. There were no differences between groups in performance on the cannulation task median scores (Blocked: 89.8 [IQR:37.6]; Random: 83.2 [32.3]; Control: 83.6 [19.1]; p = 0.955), number of hand motions (Blocked: 42.5 [IQR:130.3]; Random: 75.3 [111.3]; Control: 63.0 [71.8]; p = 0.912), or distance traveled by participants hands (Blocked: 2.0 m [IQR:5.8]; Random: 3.8 [8.9]; Control: 2.6 [2.5]; p = 0.816). Cannulation task performance had no correlation with total FLS performance, R2 linear = 0.014, p = 0.445. CONCLUSIONS Skills acquired from conventional FLS tasks did not effectively transfer to a laparoscopic bile duct cannulation task. Neither blocked nor random practice schedules conferred a relative advantage. These findings provide evidence that cannulation is a distinct skill from what is taught and assessed in FLS.
Collapse
Affiliation(s)
- Garrett G R J Johnson
- Department of Surgery, Section of General Surgery, University of Manitoba, Winnipeg, MB, Canada
- Clinician Investigator Program, University of Manitoba, Winnipeg, MB, Canada
| | - Jason Park
- Department of Surgery, Division of General Surgery, University of British Columbia, Vancouver, BC, Canada
| | - Ashley Vergis
- Department of Surgery, Section of General Surgery, University of Manitoba, Winnipeg, MB, Canada
| | - Lawrence M Gillman
- Department of Surgery, Section of General Surgery, University of Manitoba, Winnipeg, MB, Canada
| | - Justin D Rivard
- Department of Surgery, Section of General Surgery, University of Manitoba, Winnipeg, MB, Canada.
- Health Sciences Centre, GF436C-820 Sherbrook Street, Winnipeg, MB, R3A 1R9, Canada.
| |
Collapse
|
5
|
Kornblith AE, Addo N, Dong R, Rogers R, Grupp-Phelan J, Butte A, Gupta P, Callcut RA, Arnaout R. Development and Validation of a Deep Learning Strategy for Automated View Classification of Pediatric Focused Assessment With Sonography for Trauma. JOURNAL OF ULTRASOUND IN MEDICINE : OFFICIAL JOURNAL OF THE AMERICAN INSTITUTE OF ULTRASOUND IN MEDICINE 2022; 41:1915-1924. [PMID: 34741469 PMCID: PMC9072593 DOI: 10.1002/jum.15868] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/13/2021] [Revised: 10/14/2021] [Accepted: 10/19/2021] [Indexed: 06/09/2023]
Abstract
OBJECTIVE Pediatric focused assessment with sonography for trauma (FAST) is a sequence of ultrasound views rapidly performed by clinicians to diagnose hemorrhage. A technical limitation of FAST is the lack of expertise to consistently acquire all required views. We sought to develop an accurate deep learning view classifier using a large heterogeneous dataset of clinician-performed pediatric FAST. METHODS We developed and conducted a retrospective cohort analysis of a deep learning view classifier on real-world FAST studies performed on injured children less than 18 years old in two pediatric emergency departments by 30 different clinicians. FAST was randomly distributed to training, validation, and test datasets, 70:20:10; each child was represented in only one dataset. The primary outcome was view classifier accuracy for video clips and still frames. RESULTS There were 699 FAST studies, representing 4925 video clips and 1,062,612 still frames, performed by 30 different clinicians. The overall classification accuracy was 97.8% (95% confidence interval [CI]: 96.0-99.0) for video clips and 93.4% (95% CI: 93.3-93.6) for still frames. Per view still frames were classified with an accuracy: 96.0% (95% CI: 95.9-96.1) cardiac, 99.8% (95% CI: 99.8-99.8) pleural, 95.2% (95% CI: 95.0-95.3) abdominal upper quadrants, and 95.9% (95% CI: 95.8-96.0) suprapubic. CONCLUSION A deep learning classifier can accurately predict pediatric FAST views. Accurate view classification is important for quality assurance and feasibility of a multi-stage deep learning FAST model to enhance the evaluation of injured children.
Collapse
Affiliation(s)
- Aaron E Kornblith
- Department of Emergency Medicine, University of California, San Francisco, CA, USA
- Department of Pediatrics, University of California, San Francisco, CA, USA
- Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, USA
| | - Newton Addo
- Department of Emergency Medicine, University of California, San Francisco, CA, USA
- Department of Medicine, Division of Cardiology, University of California, San Francisco, CA, USA
| | - Ruolei Dong
- Department of Bioengineering, University of California, Berkeley, CA, USA
- Department of Bioengineering and Therapeutic Sciences, University of California, San Francisco, CA, USA
| | - Robert Rogers
- Center for Digital Health Innovation, University of California, San Francisco, CA, USA
| | - Jacqueline Grupp-Phelan
- Department of Emergency Medicine, University of California, San Francisco, CA, USA
- Department of Pediatrics, University of California, San Francisco, CA, USA
| | - Atul Butte
- Department of Pediatrics, University of California, San Francisco, CA, USA
- Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, USA
| | - Pavan Gupta
- Center for Digital Health Innovation, University of California, San Francisco, CA, USA
| | - Rachael A Callcut
- Center for Digital Health Innovation, University of California, San Francisco, CA, USA
- Department of Surgery, University of California, Davis, CA, USA
| | - Rima Arnaout
- Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, USA
- Department of Medicine, Division of Cardiology, University of California, San Francisco, CA, USA
| |
Collapse
|
6
|
Task model-specific operator skill assessment in routine fetal ultrasound scanning. Int J Comput Assist Radiol Surg 2022; 17:1437-1444. [PMID: 35556206 PMCID: PMC9307537 DOI: 10.1007/s11548-022-02642-y] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2022] [Accepted: 04/08/2022] [Indexed: 11/20/2022]
Abstract
Purpose For highly operator-dependent ultrasound scanning, skill assessment approaches evaluate operator competence given available data, such as acquired images and tracked probe movement. Operator skill level can be quantified by the completeness, speed, and precision of performing a clinical task, such as biometry. Such clinical tasks are increasingly becoming assisted or even replaced by automated machine learning models. In addition to measurement, operators need to be competent at the upstream task of acquiring images of sufficient quality. To provide computer assistance for this task requires a new definition of skill. Methods This paper focuses on the task of selecting ultrasound frames for biometry, for which operator skill is assessed by quantifying how well the tasks are performed with neural network-based frame classifiers. We first develop a frame classification model for each biometry task, using a novel label-efficient training strategy. Once these task models are trained, we propose a second task model-specific network to predict two skill assessment scores, based on the probability of identifying positive frames and accuracy of model classification. Results We present comprehensive results to demonstrate the efficacy of both the frame-classification and skill-assessment networks, using clinically acquired data from two biometry tasks for a total of 139 subjects, and compare the proposed skill assessment with metrics of operator experience. Conclusion Task model-specific skill assessment is feasible and can be predicted by the proposed neural networks, which provide objective assessment that is a stronger indicator of task model performance, compared to existing skill assessment methods. Supplementary Information The online version contains supplementary material available at 10.1007/s11548-022-02642-y.
Collapse
|
7
|
Prager R, Pageau P, Hodges T, Yan C, Woo M, Nemnom M, Millington S, Holden M, St‐Gelais R, Cheung WJ. Characterizing the biomechanical differences between novice and expert point-of-care ultrasound practitioners using a low-cost gyroscope and accelerometer integrated sensor: A pilot study. AEM EDUCATION AND TRAINING 2022; 6:e10733. [PMID: 35392491 PMCID: PMC8963729 DOI: 10.1002/aet2.10733] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Revised: 01/20/2022] [Accepted: 02/11/2022] [Indexed: 06/14/2023]
Abstract
INTRODUCTION Point-of-care ultrasound (POCUS) has become an important diagnostic tool in acute care medicine; however, little is known about the biomechanical differences between novice and expert practitioners. METHODS A low-cost ($50 CAD) gyroscope and accelerometer integrated sensor was assembled and affixed to an ultrasound probe. Seventeen participants, nine novices and eight experts, were recruited to perform three abdominal and four cardiac scans on a standardized patient. Participant demographics, time per scan, average acceleration, average angular velocity, decay in acceleration and angular velocity over time, and frequency of probe movements were analyzed. Video capture with blinded video review was scored. RESULTS On video review, experts had higher image optimization and acquisition scores for both abdominal and cardiac scans. Experts had shorter scan times for abdominal (7 s vs. 26 s, p = 0.003) and cardiac (11 s vs. 26 s, p < 0.001) scans. There was no difference in average acceleration (g) between novices and experts performing abdominal (1.02 vs. 1.01, p = 0.50) and cardiac (1.01 vs. 1.01, p = 0.45) scans. Experts had lower angular velocity (°/s) for abdominal scans (10.00 vs. 18.73, p < 0.001) and cardiac scans (15.61 vs. 20.33, p = 0.02) There was a greater decay in acceleration over time for experts performing cardiac scans compared to novices (-0.194 vs. -0.050, p = 0.03) but not for abdominal scans or when measuring angular velocity. The frequency of movements (Hz) was higher for novices compared to experts for abdominal (16.68 vs. 13.79, p < 0.001) and cardiac (17.60 vs. 13.63, p = 0.002) scans. DISCUSSION This study supports the feasibility of a low-cost gyroscope and accelerometer integrated sensor to quantify the biomechanics of POCUS. It may also support the concept of "window shopping" as a method by which experts obtain abdominal and cardiac views, where sliding is used to find an acoustic window, then smaller rocking and tilting probe movements are used to refine the image.
Collapse
Affiliation(s)
- Ross Prager
- Department of MedicineUniversity of OttawaOttawaOntarioCanada
| | - Paul Pageau
- Department of Emergency MedicineUniversity of OttawaOttawaOntarioCanada
| | - Timothy Hodges
- Faculty of EngineeringUniversity of OttawaOttawaOntarioCanada
| | - Christina Yan
- Faculty of MedicineUniversity of OttawaOttawaOntarioCanada
| | - Michael Woo
- Department of Emergency MedicineUniversity of OttawaOttawaOntarioCanada
| | - Marie‐Joe Nemnom
- Clinical Epidemiology ProgramOttawa Hospital Research InstituteOttawaOntarioCanada
| | - Scott Millington
- Division of Critical CareUniversity of OttawaOttawaOntarioCanada
| | - Matthew Holden
- School of Computer ScienceCarleton UniversityOttawaOntarioCanada
| | | | - Warren J. Cheung
- Department of Emergency MedicineUniversity of OttawaOttawaOntarioCanada
| |
Collapse
|
8
|
Hochman JB, Pisa J, Kazmerik K, Unger B. Hand Motion Analysis Illustrates Differences When Drilling Cadaveric and Printed Temporal Bone. Ann Otol Rhinol Laryngol 2021; 131:1224-1230. [PMID: 34872376 PMCID: PMC9452853 DOI: 10.1177/00034894211059310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
OBJECTIVE Temporal bone simulation is now commonly used to augment cadaveric education. Assessment of these tools is ongoing, with haptic modeling illustrating dissimilar motion patterns compared to cadaveric opportunities. This has the potential to result in maladaptive skill development. It is hypothesized that trainee drill motion patterns during printed model dissection may likewise demonstrate dissimilar hand motion patterns. METHODS Resident surgeons dissected 3D-printed temporal bones generated from microCT data and cadaveric simulations. A magnetic position tracking system (TrakSTAR Ascension, Yarraville, Australia) captured drill position and orientation. Skill assessment included cortical mastoidectomy, thinning procedures (sigmoid sinus, dural plate, posterior canal wall) and facial recess development. Dissection was performed by 8 trainees (n = 5 < PGY3 > n = 3) using k-cos metrics to analyze drill strokes within position recordings. K-cos metrics define strokes by change in direction, providing metrics for stroke duration, curvature, and length. RESULTS T-tests between models showed no significant difference in drill stroke frequency (cadaveric = 1.36/s, printed = 1.50/s, P < .40) but demonstrate significantly shorter duration (cadaveric = 0.37 s, printed = 0.16 s, P < .01) and a higher percentage of curved strokes (cadaveric = 31, printed = 67, P < .01) employed in printed bone dissection. Junior staff used a higher number of short strokes (junior = 0.54, senior = 0.38, P < .01) and higher percentage of curved strokes (junior = 35%, senior = 21%, P < .01). CONCLUSIONS Significant differences in hand motions were present between simulations, however the significance is unclear. This may indicate that printed bone is not best positioned to be the principal training schema.
Collapse
Affiliation(s)
- Jordan B Hochman
- Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada.,Department of Otolaryngology - Head and Neck Surgery, Faculty of Medicine, University of Manitoba, Winnipeg, MB, Canada
| | - Justyn Pisa
- Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada.,Department of Otolaryngology - Head and Neck Surgery, Faculty of Medicine, University of Manitoba, Winnipeg, MB, Canada.,Department of Otolaryngology - Head and Neck Surgery, Health Sciences Centre, Winnipeg, MB, Canada
| | - Katrice Kazmerik
- Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada.,Department of Family Medicine, Pure Lifestyle, Winnipeg, MB, Canada
| | - Bertram Unger
- Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada.,Department of Medical Education, Faculty of Medicine, University of Manitoba, Winnipeg, MB, Canada
| |
Collapse
|
9
|
Johnson G, Vergis A, Unger B, Park J, Gillman L. Design and Validity Evidence for a Unique Endoscopy Simulator Using a Commercial Video Game. Cureus 2021; 13:e18379. [PMID: 34725623 PMCID: PMC8553393 DOI: 10.7759/cureus.18379] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/29/2021] [Indexed: 11/05/2022] Open
Abstract
Background Procedural simulation enhances early endoscopy training. Multiple commercial simulators are available; however, their application is limited by cost and poor user compliance. First-person "shooter" (FPS) video games are popular. In this study, we aimed to show that a novel in-house designed colonoscope controller used to play an FPS video game shares similar constructs with real-life endoscopy. Methodology Participants completed the first three levels on an FPS video game, Portal (Valve Corporation, Bellevue, WA), first using a conventional controller and then the modified endoscope controller. A total of 12 expert endoscopists and 12 surgical residents with minimal endoscopy experience were evaluated based on completion time, button presses, and hand motion analyses. Results Experts outperformed novices for completion time (expert: 944 seconds; novice: 1,515 seconds; p = 0.006) and hand movements (expert: 1,263.1; novice: 2,052.6; p = 0.004) in using the novel colonoscope controller. There was no difference in button presses or total path length traveled. Furthermore, performance did not differ using conventional game controls. Conclusions Experts outperformed novices using the endoscope but not the conventional controller with respect to the economy of movement and completion time. This result confirms that our endoscope-controlled video game shares similar paradigms with real-life endoscopy and serves as a first step toward creating a more enjoyable and cheaper alternative to commercially available endoscopy simulators.
Collapse
Affiliation(s)
| | | | - Bertram Unger
- Internal Medicine, University of Manitoba, Winnipeg, CAN
| | - Jason Park
- Surgery, University of Manitoba, Winnipeg, CAN
| | | |
Collapse
|
10
|
Holden MS, Portillo A, Salame G. Skills Classification in Cardiac Ultrasound with Temporal Convolution and Domain Knowledge Using a Low-Cost Probe Tracker. ULTRASOUND IN MEDICINE & BIOLOGY 2021; 47:3002-3013. [PMID: 34344562 DOI: 10.1016/j.ultrasmedbio.2021.06.011] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/10/2020] [Revised: 04/29/2021] [Accepted: 06/17/2021] [Indexed: 06/13/2023]
Abstract
As point-of-care ultrasound (POCUS) becomes more integrated into clinical practice, it is essential to address all aspects of ultrasound operator proficiency. Ultrasound proficiency requires the ability to acquire, interpret and integrate bedside ultrasound images. The difference in image acquisition psychomotor skills between novice (trainee) and expert (instructor) ultrasonographer has not been described. We created an inexpensive system, called Probe Watch, to record probe motion and assess image acquisition in cardiac POCUS using an inertial measurement device and software for data recording based on open-source components. We designed a temporal convolutional network for skills classification from probe motion that integrates clinical domain knowledge. We further designed data augmentation methods to improve its generalization. Subsequently, we validated the setup and assessment method on a set of novice and expert sonographers performing cardiac ultrasound in a simulation-based training environment. The proposed methods classified participants as novice or expert with areas under the receiver operating characteristic curve of 0.931 and 0.761 for snippets and trials, respectively. Integrating domain knowledge into the neural network had added value. Furthermore, we identified the most discriminative features for assessment. Probe Watch quantifies motion during cardiac ultrasound and provides insight into probe motion behavior. It may be deployed during cardiac ultrasound training to monitor learning curves objectively and automatically.
Collapse
Affiliation(s)
- Matthew S Holden
- School of Computer Science, Carleton University, Ottawa, Ontario, Canada.
| | | | | |
Collapse
|
11
|
Ackil DJ, Toney A, Good R, Ross D, Germano R, Sabbadini L, Thiessen M, Bell C, Kendall JL. Use of Hand-motion Analysis to Assess Competence and Skill Decay for Cardiac and Lung Point-of-care Ultrasound. AEM EDUCATION AND TRAINING 2021; 5:e10560. [PMID: 34124508 PMCID: PMC8171793 DOI: 10.1002/aet2.10560] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/06/2020] [Revised: 10/31/2020] [Accepted: 11/11/2020] [Indexed: 06/12/2023]
Abstract
OBJECTIVES Assessment of competence in technical skills, including point-of-care ultrasound (POCUS), is required before a novice can safely perform the skill independently. Ongoing assessment of competence is also required because technical skills degrade over time, especially when they are infrequently performed or complex. Hand-motion analysis (HMA) is an objective assessment tool that has been used to evaluate competency in many technical skills. The purpose of this study was to demonstrate the feasibility and validity of HMA as an assessment tool for competence in both simple and complex technical skills as well as skill degradation over time. METHODS This prospective cohort study included 36 paramedics with no POCUS experience and six physicians who were fellowship trained in POCUS. The novices completed a 4-hour didactic and hands-on training program for cardiac and lung POCUS. HMA measurements, objective structured clinical examinations (OSCE), and written examinations were collected for novices immediately before and after training as well as 2 and 4 months after training. Expert HMA metrics were also recorded. RESULTS Expert HMA metrics for cardiac and lung POCUS were significantly better than those of novices. After completion of the training program, the novices improved significantly in all HMA metrics, knowledge test scores, and OSCE scores. Novices showed skill degradation in cardiac POCUS based on HMA metrics and OSCE scores while lung POCUS image acquisition skills were preserved. Novices deemed competent by OSCE score performed significantly better in HMA metrics than those not deemed competent. CONCLUSION We have demonstrated that HMA is a feasible and valid tool for assessment of competence in technical skills and can also evaluate skill degradation over time. Skill degradation appears more apparent in complex skills, such as cardiac POCUS. HMA may provide a more efficient and reliable assessment of technical skills, including POCUS, when compared to traditional assessment tools.
Collapse
Affiliation(s)
- Daniel J. Ackil
- From theDepartment of SurgeryDivision of Emergency MedicineUniversity of VermontLarner College of MedicineBurlingtonVTUSA
| | - Amanda Toney
- theDepartment of Pediatric Emergency MedicineUniversity of Colorado School of MedicineDenver Health Medical CenterDenverCOUSA
| | - Ryan Good
- theDepartment of PediatricsSection of Critical CareUniversity of Colorado School of MedicineAnschutz Medical Campus and Children’s Hospital ColoradoAuroraCOUSA
| | - David Ross
- theRocky Vista University College of Osteopathic MedicineParkerCOUSA
| | - Rocco Germano
- theFacolta di Medicina e ChirurgiaUniversita degli Studi di BresciaSpedali Civil di BresciaBresciaItaly
| | - Linda Sabbadini
- theFacolta di Medicina e ChirurgiaUniversita degli Studi di BresciaSpedali Civil di BresciaBresciaItaly
| | - Molly Thiessen
- and theDepartment of Emergency MedicineUniversity of Colorado School of MedicineDenver Health Medical CenterDenverCOUSA
| | - Colin Bell
- and theDepartment of Emergency MedicineQueen’s UniversityKingstonOntarioCanada
| | - John L. Kendall
- and theDepartment of Emergency MedicineUniversity of Colorado School of MedicineDenver Health Medical CenterDenverCOUSA
| |
Collapse
|
12
|
Zulbaran-Rojas A, Najafi B, Arita N, Rahemi H, Razjouyan J, Gilani R. Utilization of Flexible-Wearable Sensors to Describe the Kinematics of Surgical Proficiency. J Surg Res 2021; 262:149-158. [PMID: 33581385 DOI: 10.1016/j.jss.2021.01.006] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2020] [Revised: 12/02/2020] [Accepted: 01/08/2021] [Indexed: 12/23/2022]
Abstract
BACKGROUND Traditional assessment (e.g., checklists, videotaping) for surgical proficiency may lead to subjectivity and does not predict performance in the clinical setting. Hand motion analysis is evolving as an objective tool for grading technical dexterity; however, most devices accompany with technical limitations or discomfort. We purpose the use of flexible wearable sensors to evaluate the kinematics of surgical proficiency. METHODS Surgeons were recruited and performed a vascular anastomosis task in a single institution. A modified objective structured assessment of technical skills (mOSATS) was used for technical qualification. Flexible wearable sensors (BioStamp RCTM, mc10 Inc., Lexington, MA) were placed on the dorsum of the dominant hand (DH) and nondominant hand (nDH) to measure kinematic parameters: path length (Tpath), mean (Vmean) and peak (Vpeak) velocity, number of hand movements (Nmove), ratio of DH to nDH movements (rMov), and time of task (tTask) and further compared with the mOSATS score. RESULTS Participants were categorized as experts (n = 12) and novices (n = 8) based on a cutoff mean mOSATS score. Significant differences for tTask (P = 0.02), rMov (P = 0.07), DH Tpath (P = 0.04), Vmean (P = 0.07), Vpeak (P = 0.04), and nDH Nmove (P = 0.02) were in favor of the experts. Overall, mOSATS had significant correlation with tTask (r = -0.69, P = 0.001), Nmove of DH (r = -0.44, P = 0.047) and nDH (r = -0.66, P = 0.001), and rMov (r = 0.52, P = 0.017). CONCLUSIONS Hand motion analysis evaluated by flexible wearable sensors is feasible and informative. Experts utilize coordinated two-handed motion, whereas novices perform one-handed tasks in a hastily jerky manner. These tendencies create opportunity for improvement in surgical proficiency among trainees.
Collapse
Affiliation(s)
- Alejandro Zulbaran-Rojas
- Division of Vascular Surgery and Endovascular Therapy, Michal E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Bijan Najafi
- Division of Vascular Surgery and Endovascular Therapy, Michal E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Nestor Arita
- Division of Vascular Surgery and Endovascular Therapy, Michal E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Hadi Rahemi
- Division of Vascular Surgery and Endovascular Therapy, Michal E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Javad Razjouyan
- Division of Vascular Surgery and Endovascular Therapy, Michal E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas
| | - Ramyar Gilani
- Division of Vascular Surgery and Endovascular Therapy, Michal E DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas.
| |
Collapse
|
13
|
Castillo-Segura P, Fernández-Panadero C, Alario-Hoyos C, Muñoz-Merino PJ, Delgado Kloos C. Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review. Artif Intell Med 2021; 112:102007. [PMID: 33581827 DOI: 10.1016/j.artmed.2020.102007] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2020] [Revised: 11/25/2020] [Accepted: 12/28/2020] [Indexed: 11/18/2022]
Abstract
The assessment of surgical technical skills to be acquired by novice surgeons has been traditionally done by an expert surgeon and is therefore of a subjective nature. Nevertheless, the recent advances on IoT (Internet of Things), the possibility of incorporating sensors into objects and environments in order to collect large amounts of data, and the progress on machine learning are facilitating a more objective and automated assessment of surgical technical skills. This paper presents a systematic literature review of papers published after 2013 discussing the objective and automated assessment of surgical technical skills. 101 out of an initial list of 537 papers were analyzed to identify: 1) the sensors used; 2) the data collected by these sensors and the relationship between these data, surgical technical skills and surgeons' levels of expertise; 3) the statistical methods and algorithms used to process these data; and 4) the feedback provided based on the outputs of these statistical methods and algorithms. Particularly, 1) mechanical and electromagnetic sensors are widely used for tool tracking, while inertial measurement units are widely used for body tracking; 2) path length, number of sub-movements, smoothness, fixation, saccade and total time are the main indicators obtained from raw data and serve to assess surgical technical skills such as economy, efficiency, hand tremor, or mind control, and distinguish between two or three levels of expertise (novice/intermediate/advanced surgeons); 3) SVM (Support Vector Machines) and Neural Networks are the preferred statistical methods and algorithms for processing the data collected, while new opportunities are opened up to combine various algorithms and use deep learning; and 4) feedback is provided by matching performance indicators and a lexicon of words and visualizations, although there is considerable room for research in the context of feedback and visualizations, taking, for example, ideas from learning analytics.
Collapse
Affiliation(s)
- Pablo Castillo-Segura
- Universidad Carlos III de Madrid, Av. Universidad 30, 28911, Leganés, Madrid, Spain.
| | | | - Carlos Alario-Hoyos
- Universidad Carlos III de Madrid, Av. Universidad 30, 28911, Leganés, Madrid, Spain.
| | - Pedro J Muñoz-Merino
- Universidad Carlos III de Madrid, Av. Universidad 30, 28911, Leganés, Madrid, Spain.
| | - Carlos Delgado Kloos
- Universidad Carlos III de Madrid, Av. Universidad 30, 28911, Leganés, Madrid, Spain.
| |
Collapse
|
14
|
Bell CR, Szulewski A, Walker M, McKaigney C, Ross G, Rang L, Newbigging J, Kendall J. Differences in Gaze Fixation Location and Duration Between Resident and Fellowship Sonographers Interpreting a Focused Assessment With Sonography in Trauma. AEM EDUCATION AND TRAINING 2021; 5:28-36. [PMID: 33521488 PMCID: PMC7821074 DOI: 10.1002/aet2.10439] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Revised: 01/22/2020] [Accepted: 02/07/2020] [Indexed: 05/30/2023]
Abstract
OBJECTIVES We quantified the gaze fixation duration of resident and fellowship sonographers interpreting a prerecorded focused assessment with sonography in trauma (FAST). We hypothesized that all sonographers would fixate on each relevant anatomic relationship but that the duration of fixation would differ. METHODS We conducted a cross-sectional study collecting and analyzing the gaze fixations of a convenience sample of current resident and fellowship sonographers. All sonographers viewed a standardized FAST video, and their gaze fixations were recorded using a Tobii X3-120 eye-tracking bar. Gaze fixations over nine anatomic regions of interest (ROIs) were identified. These were assessed for normality and analyzed using the Wilcoxon rank sum test at an alpha of 0.05 and Bonferroni correction p value of <0.0034. The chi-square test and Pearson's correlation were performed to assess statistical association. RESULTS The gaze fixation recordings of 24 resident and eight fellowship sonographers were suitable for analysis. Fourteen of the 24 resident sonographers viewed all ROIs in the FAST, whereas all eight fellowship sonographers viewed each of the nine relevant ROIs. Five ROIs were identified over which at least one resident sonographer did not have a gaze fixation. No statistically significant difference was identified between groups. Resident sonographers gaze fixated over the left upper quadrant (LUQ) splenorenal interface for a median (interquartile range) of 10.64 (9.73-11.60) seconds. The fellowship group viewed the same ROI for 8.43 (6.64-8.95) seconds (p < 0.003). All participants viewed this ROI. No other ROIs had a statistical difference. CONCLUSION Five ROIs were identified that were not visually interrogated by all resident sonographers. Only 14 of 24 resident sonographers visually interrogated every area in the FAST, whereas all fellowship sonographers interrogated every ROI. A statistically significant difference was found in gaze fixation duration between resident and fellowship sonographers in one ROI. Further study is required for gaze fixation assessment to become a tool for the interpretation component of point-of-care ultrasound.
Collapse
Affiliation(s)
- Colin R. Bell
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - Adam Szulewski
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - Melanie Walker
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
| | - Conor McKaigney
- Department of Emergency MedicineSouth Health, Alberta Health Services and Cumming School of Medicine at the University of CalgaryCalgaryAlbertaCanada
| | - Graeme Ross
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - Louise Rang
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - Joseph Newbigging
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - John Kendall
- Department of Emergency MedicineDenver Health Medical Center and University of Colorado Medical SchoolDenverCO
| |
Collapse
|
15
|
Tyrrell RE, Holden MS. Ultrasound video analysis for skill level assessment in FAST ultrasound. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2020. [DOI: 10.1080/21681163.2020.1835549] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
16
|
Wang Y, Droste R, Jiao J, Sharma H, Drukker L, Papageorghiou AT, Noble JA. Differentiating Operator Skill during Routine Fetal Ultrasound Scanning using Probe Motion Tracking. MEDICAL ULTRASOUND, AND PRETERM, PERINATAL AND PAEDIATRIC IMAGE ANALYSIS 2020; 12437:180-188. [PMID: 33103166 DOI: 10.1007/978-3-030-60334-2_18] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/04/2022]
Abstract
In this paper, we consider differentiating operator skill during fetal ultrasound scanning using probe motion tracking. We present a novel convolutional neural network-based deep learning framework to model ultrasound probe motion in order to classify operator skill levels, that is invariant to operators' personal scanning styles. In this study, probe motion data during routine second-trimester fetal ultrasound scanning was acquired by operators of known experience levels (2 newly-qualified operators and 10 expert operators). The results demonstrate that the proposed model can successfully learn underlying probe motion features that distinguish operator skill levels during routine fetal ultrasound with 95% accuracy.
Collapse
Affiliation(s)
- Yipei Wang
- Institute of Biomedical Engineering, University of Oxford, Oxford, UK
| | - Richard Droste
- Institute of Biomedical Engineering, University of Oxford, Oxford, UK
| | - Jianbo Jiao
- Institute of Biomedical Engineering, University of Oxford, Oxford, UK
| | - Harshita Sharma
- Institute of Biomedical Engineering, University of Oxford, Oxford, UK
| | - Lior Drukker
- Nuffield Department of Women's and Reproductive Health, University of Oxford, Oxford, UK
| | - Aris T Papageorghiou
- Nuffield Department of Women's and Reproductive Health, University of Oxford, Oxford, UK
| | - J Alison Noble
- Institute of Biomedical Engineering, University of Oxford, Oxford, UK
| |
Collapse
|
17
|
Ma IWY. Consensus-Based Expert Development of Critical Items for Direct Observation of Point-of-Care Ultrasound Skills. J Grad Med Educ 2020; 12:176-184. [PMID: 32322351 PMCID: PMC7161337 DOI: 10.4300/jgme-d-19-00531.1] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/28/2019] [Revised: 12/11/2019] [Accepted: 12/31/2019] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND Point-of-care ultrasound (POCUS) is increasingly used in a number of medical specialties. To support competency-based POCUS education, workplace-based assessments are essential. OBJECTIVE We developed a consensus-based assessment tool for POCUS skills and determined which items are critical for competence. We then performed standards setting to set cut scores for the tool. METHODS Using a modified Delphi technique, 25 experts voted on 32 items over 3 rounds between August and December 2016. Consensus was defined as agreement by at least 80% of the experts. Twelve experts then performed 3 rounds of a standards setting procedure in March 2017 to establish cut scores. RESULTS Experts reached consensus for 31 items to include in the tool. Experts reached consensus that 16 of those items were critically important. A final cut score for the tool was established at 65.2% (SD 17.0%). Cut scores for critical items are significantly higher than those for noncritical items (76.5% ± SD 12.4% versus 53.1% ± SD 12.2%, P < .0001). CONCLUSIONS We reached consensus on a 31-item workplace-based assessment tool for identifying competence in POCUS. Of those items, 16 were considered critically important. Their importance is further supported by higher cut scores compared with noncritical items.
Collapse
|
18
|
Kåsine T, Romundstad L, Rosseland LA, Ullensvang K, Fagerland MW, Hol PK, Kessler P, Sauter AR. Needle tip tracking for ultrasound-guided peripheral nerve block procedures-An observer blinded, randomised, controlled, crossover study on a phantom model. Acta Anaesthesiol Scand 2019; 63:1055-1062. [PMID: 31037724 PMCID: PMC6767137 DOI: 10.1111/aas.13379] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2018] [Revised: 02/25/2019] [Accepted: 03/22/2019] [Indexed: 11/29/2022]
Abstract
BACKGROUND The Onvision needle tip tracking (NTT) is a new technology consisting of a needle with an ultrasound sensor close to the needle tip and a console for computerised signal processing. The aim of the study was to evaluate NTT technology during ultrasound-guided simulated peripheral nerve block procedures in a porcine phantom model. METHODS Forty anaesthesiologists performed in-plane and out-of-plane simulated nerve blocks with and without NTT guidance. The primary outcome measure was procedure time. Secondary outcomes were hand movements and the path length travelled by the hands measured by motion analysis, precision of the needle tip related to the target structure, success rates and violations of the target structure, and the participants confidence whether their procedure would be successful or not. RESULTS Procedure time was reduced from 66.7 (SD = 47.5) seconds to 43.8 (SD = 29.2) seconds when NTT was used for out-of-plane procedures (P = 0.002). The number of hand movements of the probe hand was 13.9 (SD = 30.2) with NTT and 22.8 (SD = 30.0) without NTT (P = 0.019). No significant differences were registered during the performance of in-plane procedures. The participants confidence in a presumed block success was increased with both in-plane procedures (8.50 (SD = 1.18) with NTT vs 7.65 (SD = 1.96), P = 0.004) and out-of-plane procedures (8.50 (SD = 1.09) vs 7.10 (SD = 1.89), P = 0.0001). CONCLUSIONS The new NTT technology significantly reduced the procedure time and the number of hand movements for ultrasound-guided out-of-plane PNB procedures. No significant differences were found for the in-plane procedures.
Collapse
Affiliation(s)
- Trine Kåsine
- Division of Emergencies and Critical Care, Department of Anaesthesiology Oslo University Hospital Oslo Norway
- Institute of Clinical Medicine, Faculty of Medicine University of Oslo Oslo Norway
| | - Luis Romundstad
- Division of Emergencies and Critical Care, Department of Anaesthesiology Oslo University Hospital Oslo Norway
| | - Leiv Arne Rosseland
- Institute of Clinical Medicine, Faculty of Medicine University of Oslo Oslo Norway
- Division of Emergencies and Critical Care, Department of Research and Development Oslo University Hospital Oslo Norway
| | - Kyrre Ullensvang
- Division of Emergencies and Critical Care, Department of Anaesthesiology Oslo University Hospital Oslo Norway
| | - Morten Wang Fagerland
- Oslo Centre for Biostatistics and Epidemiology (OCBE) Research Support Services, Oslo University Hospital Oslo Norway
| | - Per Kristian Hol
- Institute of Clinical Medicine, Faculty of Medicine University of Oslo Oslo Norway
- The Intervention Centre Oslo University Hospital Oslo Norway
| | - Paul Kessler
- Department of Anaesthesiology, Intensive Care and Pain Medicine Orthopedic University Hospital Friedrichsheim Frankfurt Germany
| | - Axel Rudolf Sauter
- Division of Emergencies and Critical Care, Department of Anaesthesiology Oslo University Hospital Oslo Norway
| |
Collapse
|
19
|
Wanted: automated objective proficiency assessment metrics for the FAST exam (and other POCUS studies). Eur J Trauma Emerg Surg 2019; 45:763-764. [PMID: 31119321 DOI: 10.1007/s00068-019-01156-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2019] [Accepted: 05/15/2019] [Indexed: 01/22/2023]
|
20
|
Zago M, Sforza C, Mariani D, Marconi M, Biloslavo A, Greca AL, Kurihara H, Casamassima A, Bozzo S, Caputo F, Galli M, Zago M. Educational impact of hand motion analysis in the evaluation of FAST examination skills. Eur J Trauma Emerg Surg 2019; 46:1421-1428. [DOI: 10.1007/s00068-019-01112-6] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2018] [Accepted: 03/06/2019] [Indexed: 01/22/2023]
|
21
|
Oh CJ, Tripathi PB, Gu JT, Borden P, Wong BJF. Development and evaluation of rhinoplasty spreader graft suture simulator for novice surgeons. Laryngoscope 2018; 129:344-350. [PMID: 30194858 DOI: 10.1002/lary.27326] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2018] [Revised: 04/01/2018] [Accepted: 05/07/2018] [Indexed: 01/03/2023]
Abstract
OBJECTIVE Surgical simulators aimed at mimicking elements of rhinoplasty surgery, specifically those aimed at improving cartilage suturing, are not available. Here, we present a surgical simulator for spreader graft placement that uses cartilage rather than synthetic materials and gauge improvement using objective measures for suture placement accuracy, speed, and efficiency of hand motion. METHODS Twenty-two otolaryngologists in two groups (residents [10] and experts [12]) were instructed to secure the two spreader graft specimen into position with three mattress sutures on a nose model that used porcine septal cartilage as a proxy for the human counterpart. Hand motion was tracked using an electromagnetic position sensing device. The time required to complete the suture task, total hand displacement, cumulative number of hand motion direction changes, and accuracy of suture insertion were measured. These measurements were compared between the two cohort groups for construct validity. The subjects completed a survey to evaluate realism and value of the model. RESULTS The expert group had a lower mean time required to complete the task (P < 0.05), total hand displacement (P < 0.01), and number of hand motion direction changes (P < 0.001). No significant difference was observed between the two groups in suture precision measurement. The subjects agreed on the face validity and usefulness of the trainer. CONCLUSIONS Our study suggests that the simulator may be a useful tool to objectively gauge suturing efficiency. Devices such as this may be useful for developing skill with suturing cartilage tissue and potentially be used to assess resident acquisition of surgical skill. LEVEL OF EVIDENCE NA Laryngoscope, 129:344-350, 2019.
Collapse
Affiliation(s)
- Connie J Oh
- Loma Linda University School of Medicine, Loma, Linda.,The Beckman Laser Institute and Medical Clinic, Irvine, California, U.S.A
| | - Prem B Tripathi
- Department of Otolaryngology-Head and Neck Surgery, Irvine, California, U.S.A
| | - Jeffrey T Gu
- Department of Otolaryngology-Head and Neck Surgery, Irvine, California, U.S.A
| | - Pamela Borden
- The Beckman Laser Institute and Medical Clinic, Irvine, California, U.S.A
| | - Brian J-F Wong
- Loma Linda University School of Medicine, Loma, Linda.,The Beckman Laser Institute and Medical Clinic, Irvine, California, U.S.A.,Department of Otolaryngology-Head and Neck Surgery, Irvine, California, U.S.A.,Department of Biomedical Engineering, University of California, Irvine, California, U.S.A
| |
Collapse
|
22
|
Oliveira LGO, Tagliari D, Becker MJ, Adame T, Cruvinel Neto J, Spencer Netto FAC. Basic ultrasound training assessment in the initial abdominal trauma screening. Rev Col Bras Cir 2018; 45:e1556. [PMID: 29466514 DOI: 10.1590/0100-6991e-20181556] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2017] [Accepted: 11/02/2017] [Indexed: 11/22/2022] Open
Abstract
OBJECTIVE to verify the efficiency and usefulness of basic ultrasound training in trauma (FAST - Focused Assessment with Sonography in Trauma) for emergency physicians in the primary evaluation of abdominal trauma. METHODS a longitudinal and observational study was carried out from 2015 to 2017, with 11 emergency physicians from Hospital Universitário do Oeste do Paraná, submitted to ultrasound training in emergency and trauma (USET® - SBAIT). FAST results started to be collected two months after the course. These were compared with a composite score of complementary exams and surgical findings. Information was stored in a Microsoft Excel program database and submitted to statistical analysis. RESULTS FAST was performed in 120 patients. In the study, 38.4% of the assessed patients had a shock index ≥0.9. The composite score detected 40 patients with free peritoneal fluid, whereas FAST detected 27 cases. The method sensitivity was 67.5%, specificity was 98.7%, the positive predictive value was 96.4%, the negative predictive value was 85.39% and accuracy was 88%. All those with a positive FAST had a shock index ≥0.9. Fifteen patients with positive FAST and signs of instability were immediately submitted to surgery. CONCLUSIONS the basic training of emergency physicians in FAST showed efficiency and usefulness in abdominal trauma assessment. Due to its low cost and easy implementation, this modality should be considered as a screening strategy for patients with abdominal trauma in health systems.
Collapse
Affiliation(s)
| | - Debora Tagliari
- - Universidade Estadual do Oeste do Paraná, Serviço de Cirurgia Geral, Cascavel, PR, Brasil
| | - Mariana Juliato Becker
- - Universidade Estadual do Oeste do Paraná, Serviço de Cirurgia Geral, Cascavel, PR, Brasil
| | - Thiago Adame
- - Universidade Estadual do Oeste do Paraná, Serviço de Cirurgia Geral, Cascavel, PR, Brasil
| | - José Cruvinel Neto
- - Universidade Estadual do Oeste do Paraná, Serviço de Cirurgia Geral, Cascavel, PR, Brasil
| | | |
Collapse
|
23
|
The Effects of Music on Microsurgical Technique and Performance: A Motion Analysis Study. Ann Plast Surg 2018; 78:S243-S247. [PMID: 28399026 DOI: 10.1097/sap.0000000000001047] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
BACKGROUND Music is commonly played in operating rooms (ORs) throughout the country. If a preferred genre of music is played, surgeons have been shown to perform surgical tasks quicker and with greater accuracy. However, there are currently no studies investigating the effects of music on microsurgical technique. Motion analysis technology has recently been validated in the objective assessment of plastic surgery trainees' performance of microanastomoses. Here, we aimed to examine the effects of music on microsurgical skills using motion analysis technology as a primary objective assessment tool. METHODS Residents and fellows in the Plastic and Reconstructive Surgery program were recruited to complete a demographic survey and participate in microsurgical tasks. Each participant completed 2 arterial microanastomoses on a chicken foot model, one with music playing, and the other without music playing. Participants were blinded to the study objectives and encouraged to perform their best. The order of music and no music was randomized. Microanastomoses were video recorded using a digitalized S-video system and deidentified. Video segments were analyzed using ProAnalyst motion analysis software for automatic noncontact markerless video tracking of the needle driver tip. RESULTS Nine residents and 3 plastic surgery fellows were tested. Reported microsurgical experience ranged from 1 to 10 arterial anastomoses performed (n = 2), 11 to 100 anastomoses (n = 9), and 101 to 500 anastomoses (n = 1). Mean age was 33 years (range, 29-36 years), with 11 participants right-handed and 1 ambidextrous. Of the 12 subjects tested, 11 (92%) preferred music in the OR. Composite instrument motion analysis scores significantly improved with playing preferred music during testing versus no music (paired t test, P <0.001). Improvement with music was significant even after stratifying scores by order in which variables were tested (music first vs no music first), postgraduate year, and number of anastomoses (analysis of variance, P < 0.01). CONCLUSIONS Preferred music in the OR may have a positive effect on trainees' microsurgical performance; as such, trainees should be encouraged to participate in setting the conditions of the OR to optimize their comfort and, possibly, performance. Moreover, motion analysis technology is a useful tool with a wide range of applications for surgical education and outcomes optimization.
Collapse
|
24
|
Bell CR, McKaigney CJ, Holden M, Fichtinger G, Rang L. Sonographic Accuracy as a Novel Tool for Point-of-care Ultrasound Competency Assessment. AEM EDUCATION AND TRAINING 2017; 1:316-324. [PMID: 30051050 PMCID: PMC6001820 DOI: 10.1002/aet2.10064] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/06/2017] [Revised: 08/09/2017] [Accepted: 08/25/2017] [Indexed: 05/09/2023]
Abstract
OBJECTIVES The Focused Assessment with Sonography in Trauma (FAST) is a point-of-care ultrasound (PoCUS) study that is routine in trauma patient assessment. Many organizations have published training guidelines, which grant competency through the completion of a fixed number of observed scans. This approach is incongruent with current trends in competency-based medical education. We aim to objectively quantify probe motion and user accuracy to differentiate groups of PoCUS operators. METHODS Emergency medicine residents were recruited in two groups. The novice group (n = 15) had limited PoCUS experience, whereas the intermediate group (n = 14) had completed at least 50 supervised FAST examinations. Both groups underwent assessment on a live human model. Residents from the novice group returned (n = 9) after completing a curriculum and repeated the assessment using the identical experimental construct. RESULTS Significant differences (p < 0.05) were found between the novice and both the intermediate and the novice returned groups in time, path length, and points of interest (POIs) scanned. Novices required more time to complete the full examination (290.82 seconds vs. 197.41 seconds vs. 271.79 seconds), utilized more motion (9392.07 mm vs. 4052.73 mm vs. 4985.05 mm), and imaged fewer POIs (48.13% vs. 95.00% vs. 100.00%) when compared to intermediates and returning novices, respectively. No difference was found between the intermediate and novice returned groups for the complete examination. Spearman's correlation was calculated between variables within each group. Correlations between time and path length were statistically significant (p < 0.05) with novice, intermediate, and novice returned values of 0.67, 0.65, and 0.90. Interestingly, neither time nor path length consistently correlated with POIs scanned in any group. CONCLUSION Differences in probe motion efficiency and POIs scanned between novices and intermediate or returning novice users show promise for use as a quantitative objective assessment tool. Unlike in surgical literature, accuracy did not correlate with path length or time to examination completion.
Collapse
Affiliation(s)
- Colin R. Bell
- Department of Emergency MedicineQueen's UniversityKingstonONCanada
| | - Conor J. McKaigney
- Department of Emergency MedicineQueen's UniversityKingstonONCanada
- Present address:
Department of Emergency MedicineUniversity of CalgaryCalgaryABCanada
| | - Matthew Holden
- School of Computing Goodwin Hall Queen's UniversityKingstonONCanada
| | - Gabor Fichtinger
- School of Computing Goodwin Hall Queen's UniversityKingstonONCanada
| | - Louise Rang
- Department of Emergency MedicineQueen's UniversityKingstonONCanada
| |
Collapse
|
25
|
Zago M, Martinez Casas I, Pereira J, Mariani D, Silva AR, Casamassima A, Barbosa E, Ferreira F, Ruesseler M, Bass GA, Ponchietti L, Butti F, Marconi M, Pinheiro LF. Tailored ultrasound learning for acute care surgeons: a review of the MUSEC (Modular UltraSound ESTES Course) project. Eur J Trauma Emerg Surg 2016; 42:161-8. [DOI: 10.1007/s00068-016-0651-z] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2016] [Accepted: 02/10/2016] [Indexed: 01/17/2023]
|
26
|
Re: Assessment of competence in focused assessment with sonography for trauma (FAST) examination. J Trauma Acute Care Surg 2015; 80:353-4. [PMID: 26595705 DOI: 10.1097/ta.0000000000000921] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
27
|
Assessment of competence in focused assessment with sonography for trauma (FAST) examination. J Trauma Acute Care Surg 2015; 80:353. [PMID: 26595706 DOI: 10.1097/ta.0000000000000920] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|