1
|
Kiani P, Dolling-Boreham R, Hameed MS, Masino C, Fecso A, Okrainec A, Madani A. Usability, Ergonomics, and Educational Value of a Novel Telestration Tool for Surgical Coaching: Usability Study. JMIR Hum Factors 2024; 11:e57243. [PMID: 39255487 PMCID: PMC11422725 DOI: 10.2196/57243] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2024] [Revised: 06/26/2024] [Accepted: 07/17/2024] [Indexed: 09/12/2024] Open
Abstract
BACKGROUND Telementoring studies found technical challenges in achieving accurate and stable annotations during live surgery using commercially available telestration software intraoperatively. To address the gap, a wireless handheld telestration device was developed to facilitate dynamic user interaction with live video streams. OBJECTIVE This study aims to find the perceived usability, ergonomics, and educational value of a first-generation handheld wireless telestration platform. METHODS A prototype was developed with four core hand-held functions: (1) free-hand annotation, (2) cursor navigation, (3) overlay and manipulation (rotation) of ghost (avatar) instrumentation, and (4) hand-held video feed navigation on a remote monitor. This device uses a proprietary augmented reality platform. Surgeons and trainees were invited to test the core functions of the platform by performing standardized tasks. Usability and ergonomics were evaluated with a validated system usability scale and a 5-point Likert scale survey, which also evaluated the perceived educational value of the device. RESULTS In total, 10 people (9 surgeons and 1 senior resident; 5 male and 5 female) participated. Participants strongly agreed or agreed (SA/A) that it was easy to perform annotations (SA/A 9, 90% and neutral 0, 0%), video feed navigation (SA/A 8, 80% and neutral 1, 10%), and manipulation of ghost (avatar) instruments on the monitor (SA/A 6, 60% and neutral 3, 30%). Regarding ergonomics, 40% (4) of participants agreed or strongly agreed (neutral 4, 40%) that the device was physically comfortable to use and hold. These results are consistent with open-ended comments on the device's size and weight. The average system usability scale was 70 (SD 12.5; median 75, IQR 63-84) indicating an above average usability score. Participants responded favorably to the device's perceived educational value, particularly for postoperative coaching (agree 6, 60%, strongly agree 4, 40%). CONCLUSIONS This study presents the preliminary usability results of a novel first-generation telestration tool customized for use in surgical coaching. Favorable usability and perceived educational value were reported. Future iterations of the device should focus on incorporating user feedback and additional studies should be conducted to evaluate its effectiveness for improving surgical education. Ultimately, such tools can be incorporated into pedagogical models of surgical coaching to optimize feedback and training.
Collapse
Affiliation(s)
- Parmiss Kiani
- Surgical Artificial Intelligence Research Academy, University Health Network, Toronto, ON, Canada
| | - Roberta Dolling-Boreham
- Surgical Artificial Intelligence Research Academy, University Health Network, Toronto, ON, Canada
- Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Mohamed Saif Hameed
- Surgical Artificial Intelligence Research Academy, University Health Network, Toronto, ON, Canada
| | - Caterina Masino
- Surgical Artificial Intelligence Research Academy, University Health Network, Toronto, ON, Canada
| | - Andras Fecso
- Department of Surgery, University of Toronto, Toronto, ON, Canada
| | - Allan Okrainec
- Department of Surgery, University of Toronto, Toronto, ON, Canada
| | - Amin Madani
- Surgical Artificial Intelligence Research Academy, University Health Network, Toronto, ON, Canada
- Department of Surgery, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
2
|
Schuetz I, Karimpur H, Fiehler K. vexptoolbox: A software toolbox for human behavior studies using the Vizard virtual reality platform. Behav Res Methods 2023; 55:570-582. [PMID: 35322350 PMCID: PMC10027796 DOI: 10.3758/s13428-022-01831-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/09/2022] [Indexed: 11/08/2022]
Abstract
Virtual reality (VR) is a powerful tool for researchers due to its potential to study dynamic human behavior in highly naturalistic environments while retaining full control over the presented stimuli. Due to advancements in consumer hardware, VR devices are now very affordable and have also started to include technologies such as eye tracking, further extending potential research applications. Rendering engines such as Unity, Unreal, or Vizard now enable researchers to easily create complex VR environments. However, implementing the experimental design can still pose a challenge, and these packages do not provide out-of-the-box support for trial-based behavioral experiments. Here, we present a Python toolbox, designed to facilitate common tasks when developing experiments using the Vizard VR platform. It includes functionality for common tasks like creating, randomizing, and presenting trial-based experimental designs or saving results to standardized file formats. Moreover, the toolbox greatly simplifies continuous recording of eye and body movements using any hardware supported in Vizard. We further implement and describe a simple goal-directed reaching task in VR and show sample data recorded from five volunteers. The toolbox, example code, and data are all available on GitHub under an open-source license. We hope that our toolbox can simplify VR experiment development, reduce code duplication, and aid reproducibility and open-science efforts.
Collapse
Affiliation(s)
- Immo Schuetz
- Experimental Psychology, Justus Liebig University, Otto-Behaghel-Str. 10 F, 35394, Giessen, Germany.
- Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen, Giessen, Germany.
| | - Harun Karimpur
- Experimental Psychology, Justus Liebig University, Otto-Behaghel-Str. 10 F, 35394, Giessen, Germany
- Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen, Giessen, Germany
| | - Katja Fiehler
- Experimental Psychology, Justus Liebig University, Otto-Behaghel-Str. 10 F, 35394, Giessen, Germany
- Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen, Giessen, Germany
| |
Collapse
|
3
|
Kuhlmann de Canaviri L, Meiszl K, Hussein V, Abbassi P, Mirraziroudsari SD, Hake L, Potthast T, Ratert F, Schulten T, Silberbach M, Warnecke Y, Wiswede D, Schiprowski W, Heß D, Brüngel R, Friedrich CM. Static and Dynamic Accuracy and Occlusion Robustness of SteamVR Tracking 2.0 in Multi-Base Station Setups. SENSORS (BASEL, SWITZERLAND) 2023; 23:725. [PMID: 36679522 PMCID: PMC9862184 DOI: 10.3390/s23020725] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/04/2022] [Revised: 01/04/2023] [Accepted: 01/05/2023] [Indexed: 06/17/2023]
Abstract
The tracking of objects and person position, orientation, and movement is relevant for various medical use cases, e.g., practical training of medical staff or patient rehabilitation. However, these demand high tracking accuracy and occlusion robustness. Expensive professional tracking systems fulfill these demands, however, cost-efficient and potentially adequate alternatives can be found in the gaming industry, e.g., SteamVR Tracking. This work presents an evaluation of SteamVR Tracking in its latest version 2.0 in two experimental setups, involving two and four base stations. Tracking accuracy, both static and dynamic, and occlusion robustness are investigated using a VIVE Tracker (3.0). A dynamic analysis further compares three different velocities. An error evaluation is performed using a Universal Robots UR10 robotic arm as ground-truth system under nonlaboratory conditions. Results are presented using the Root Mean Square Error. For static experiments, tracking errors in the submillimeter and subdegree range are achieved by both setups. Dynamic experiments achieved errors in the submillimeter range as well, yet tracking accuracy suffers from increasing velocity. Four base stations enable generally higher accuracy and robustness, especially in the dynamic experiments. Both setups enable adequate accuracy for diverse medical use cases. However, use cases demanding very high accuracy should primarily rely on SteamVR Tracking 2.0 with four base stations.
Collapse
Affiliation(s)
- Lara Kuhlmann de Canaviri
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | - Katharina Meiszl
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | - Vana Hussein
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | - Pegah Abbassi
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | | | - Laurin Hake
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | - Tobias Potthast
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | - Fabian Ratert
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | - Tessa Schulten
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | - Marc Silberbach
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | - Yannik Warnecke
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | - Daniel Wiswede
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | - Witold Schiprowski
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | - Daniel Heß
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
- Institut für die Digitalisierung von Arbeits- und Lebenswelten (IDiAL), University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
| | - Raphael Brüngel
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
- Institute for Medical Informatics, Biometry and Epidemiology (IMIBE), University Hospital Essen, 45122 Essen, Germany
- Institute for Artificial Intelligence in Medicine (IKIM), University Hospital Essen, 45131 Essen, Germany
| | - Christoph M. Friedrich
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), 44227 Dortmund, Germany
- Institute for Medical Informatics, Biometry and Epidemiology (IMIBE), University Hospital Essen, 45122 Essen, Germany
| |
Collapse
|
4
|
Švejda M, Goubej M, Jáger A, Reitinger J, Severa O. Affordable Motion Tracking System for Intuitive Programming of Industrial Robots. SENSORS (BASEL, SWITZERLAND) 2022; 22:4962. [PMID: 35808453 PMCID: PMC9269710 DOI: 10.3390/s22134962] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Revised: 06/28/2022] [Accepted: 06/28/2022] [Indexed: 06/15/2023]
Abstract
The paper deals with a lead-through method of programming for industrial robots. The goal is to automatically reproduce 6DoF trajectories of a tool wielded by a human operator demonstrating a motion task. We present a novel motion-tracking system built around the HTC Vive pose estimation system. Our solution allows complete automation of the robot teaching process. Specific algorithmic issues of system calibration and motion data post-processing are also discussed, constituting the paper's theoretical contribution. The motion tracking system is successfully deployed in a pilot application of robot-assisted spray painting.
Collapse
|
5
|
Kapp S, Lauer F, Beil F, Rheinländer CC, Wehn N, Kuhn J. Smart Sensors for Augmented Electrical Experiments. SENSORS (BASEL, SWITZERLAND) 2021; 22:256. [PMID: 35009805 PMCID: PMC8749546 DOI: 10.3390/s22010256] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/28/2021] [Revised: 12/07/2021] [Accepted: 12/24/2021] [Indexed: 12/31/2022]
Abstract
With the recent increase in the use of augmented reality (AR) in educational laboratory settings, there is a need for new intelligent sensor systems capturing all aspects of the real environment. We present a smart sensor system meeting these requirements for STEM (science, technology, engineering, and mathematics) experiments in electrical circuits. The system consists of custom experiment boxes and cables combined with an application for the Microsoft HoloLens 2, which creates an AR experiment environment. The boxes combine sensors for measuring the electrical voltage and current at the integrated electrical components as well as a reconstruction of the currently constructed electrical circuit and the position of the sensor box on a table. Combing these data, the AR application visualizes the measurement data spatially and temporally coherent to the real experiment boxes, thus fulfilling demands derived from traditional multimedia learning theory. Following an evaluation of the accuracy and precision of the presented sensors, the usability of the system was evaluated with n=20 pupils in a German high school. In this evaluation, the usability of the system was rated with a system usability score of 94 out of 100.
Collapse
Affiliation(s)
- Sebastian Kapp
- Physics Education Research Group, Department of Physics, Technische Universität Kaiserslautern, 67663 Kaiserslautern, Germany; (F.B.); (J.K.)
| | - Frederik Lauer
- Microelectronic Systems Design Research Group, Department of Electrical and Computer Engineering, Technische Universität Kaiserslautern, 67663 Kaiserslautern, Germany; (F.L.); (C.C.R.); (N.W.)
| | - Fabian Beil
- Physics Education Research Group, Department of Physics, Technische Universität Kaiserslautern, 67663 Kaiserslautern, Germany; (F.B.); (J.K.)
| | - Carl C. Rheinländer
- Microelectronic Systems Design Research Group, Department of Electrical and Computer Engineering, Technische Universität Kaiserslautern, 67663 Kaiserslautern, Germany; (F.L.); (C.C.R.); (N.W.)
| | - Norbert Wehn
- Microelectronic Systems Design Research Group, Department of Electrical and Computer Engineering, Technische Universität Kaiserslautern, 67663 Kaiserslautern, Germany; (F.L.); (C.C.R.); (N.W.)
| | - Jochen Kuhn
- Physics Education Research Group, Department of Physics, Technische Universität Kaiserslautern, 67663 Kaiserslautern, Germany; (F.B.); (J.K.)
| |
Collapse
|
6
|
Fayat R, Delgado Betancourt V, Goyallon T, Petremann M, Liaudet P, Descossy V, Reveret L, Dugué GP. Inertial Measurement of Head Tilt in Rodents: Principles and Applications to Vestibular Research. SENSORS (BASEL, SWITZERLAND) 2021; 21:6318. [PMID: 34577524 PMCID: PMC8472891 DOI: 10.3390/s21186318] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/31/2021] [Revised: 09/03/2021] [Accepted: 09/13/2021] [Indexed: 12/21/2022]
Abstract
Inertial sensors are increasingly used in rodent research, in particular for estimating head orientation relative to gravity, or head tilt. Despite this growing interest, the accuracy of tilt estimates computed from rodent head inertial data has never been assessed. Using readily available inertial measurement units mounted onto the head of freely moving rats, we benchmarked a set of tilt estimation methods against concurrent 3D optical motion capture. We show that, while low-pass filtered head acceleration signals only provided reliable tilt estimates in static conditions, sensor calibration combined with an appropriate choice of orientation filter and parameters could yield average tilt estimation errors below 1.5∘ during movement. We then illustrate an application of inertial head tilt measurements in a preclinical rat model of unilateral vestibular lesion and propose a set of metrics describing the severity of associated postural and motor symptoms and the time course of recovery. We conclude that headborne inertial sensors are an attractive tool for quantitative rodent behavioral analysis in general and for the study of vestibulo-postural functions in particular.
Collapse
Affiliation(s)
- Romain Fayat
- Neurophysiologie des Circuits Cérébraux, Institut de Biologie de l’ENS (IBENS), Ecole Normale Supérieure, UMR CNRS 8197, INSERM U1024, Université PSL, 75005 Paris, France;
- Laboratoire MAP5, UMR CNRS 8145, Université Paris Descartes, 75006 Paris, France
| | | | - Thibault Goyallon
- Laboratoire Jean Kuntzmann, Université Grenoble Alpes, UMR CNRS 5224, INRIA, 38330 Montbonnot-Saint-Martin, France; (T.G.); (L.R.)
| | - Mathieu Petremann
- Preclinical Development, Sensorion SA, 34080 Montpellier, France; (V.D.B.); (M.P.); (P.L.); (V.D.)
| | - Pauline Liaudet
- Preclinical Development, Sensorion SA, 34080 Montpellier, France; (V.D.B.); (M.P.); (P.L.); (V.D.)
| | - Vincent Descossy
- Preclinical Development, Sensorion SA, 34080 Montpellier, France; (V.D.B.); (M.P.); (P.L.); (V.D.)
| | - Lionel Reveret
- Laboratoire Jean Kuntzmann, Université Grenoble Alpes, UMR CNRS 5224, INRIA, 38330 Montbonnot-Saint-Martin, France; (T.G.); (L.R.)
| | - Guillaume P. Dugué
- Neurophysiologie des Circuits Cérébraux, Institut de Biologie de l’ENS (IBENS), Ecole Normale Supérieure, UMR CNRS 8197, INSERM U1024, Université PSL, 75005 Paris, France;
| |
Collapse
|
7
|
Vox JP, Weber A, Wolf KI, Izdebski K, Schüler T, König P, Wallhoff F, Friemert D. An Evaluation of Motion Trackers with Virtual Reality Sensor Technology in Comparison to a Marker-Based Motion Capture System Based on Joint Angles for Ergonomic Risk Assessment. SENSORS 2021; 21:s21093145. [PMID: 34062827 PMCID: PMC8124554 DOI: 10.3390/s21093145] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Revised: 04/18/2021] [Accepted: 04/26/2021] [Indexed: 11/16/2022]
Abstract
The reproduction and simulation of workplaces, and the analysis of body postures during work processes, are parts of ergonomic risk assessments. A commercial virtual reality (VR) system offers the possibility to model complex work scenarios as virtual mock-ups and to evaluate their ergonomic designs by analyzing motion behavior while performing work processes. In this study a VR tracking sensor system (HTC Vive tracker) combined with an inverse kinematic model (Final IK) was compared with a marker-based optical motion capture system (Qualisys). Marker-based optical motion capture systems are considered the gold standard for motion analysis. Therefore, Qualisys was used as the ground truth in this study. The research question to be answered was how accurately the HTC Vive System combined with Final IK can measure joint angles used for ergonomic evaluation. Twenty-six subjects were observed simultaneously with both tracking systems while performing 20 defined movements. Sixteen joint angles were analyzed. Joint angle deviations between ±6∘ and ±42∘ were identified. These high deviations must be considered in ergonomic risk assessments when using a VR system. The results show that commercial low-budget tracking systems have the potential to map joint angles. Nevertheless, substantial weaknesses and inaccuracies in some body regions must be taken into account. Recommendations are provided to improve tracking accuracy and avoid systematic errors.
Collapse
Affiliation(s)
- Jan P. Vox
- Institut of Technical Assistance Systems (ITAS), Jade University of Applied Sciences, Ofener Str. 16/19, 26121 Oldenburg, Germany;
- Division Hearing, Speech and Audio Technology HSA, Fraunhofer Institute for Digital Media Technology IDMT, Marie-Curie-Str. 2, 26129 Oldenburg, Germany;
- Correspondence:
| | - Anika Weber
- Department of Mathematics and Technology, University of Applied Sciences Koblenz, Joseph-Rovan-Allee 2, 53424 Remagen, Germany; (A.W.); (D.F.)
- Sport and Exercise Science Research Centre, School of Applied Sciences, London South Bank University, 103 Borough Road, London SE1 0AA, UK
| | - Karen Insa Wolf
- Division Hearing, Speech and Audio Technology HSA, Fraunhofer Institute for Digital Media Technology IDMT, Marie-Curie-Str. 2, 26129 Oldenburg, Germany;
| | - Krzysztof Izdebski
- Halocline GmbH & Co. KG, Netter Platz 3, 49090 Osnabrück, Germany; (K.I.); (T.S.)
| | - Thomas Schüler
- Halocline GmbH & Co. KG, Netter Platz 3, 49090 Osnabrück, Germany; (K.I.); (T.S.)
| | - Peter König
- Institute of Cognitive Science, University of Osnabrück, Wachsbleiche 27, 49090 Osnabrück, Germany;
- Institute of Neurophysiology und Pathophysiology, University Medical Center Hamburg-Eppendorf, 20246 Hamburg, Germany
| | - Frank Wallhoff
- Institut of Technical Assistance Systems (ITAS), Jade University of Applied Sciences, Ofener Str. 16/19, 26121 Oldenburg, Germany;
- Division Hearing, Speech and Audio Technology HSA, Fraunhofer Institute for Digital Media Technology IDMT, Marie-Curie-Str. 2, 26129 Oldenburg, Germany;
| | - Daniel Friemert
- Department of Mathematics and Technology, University of Applied Sciences Koblenz, Joseph-Rovan-Allee 2, 53424 Remagen, Germany; (A.W.); (D.F.)
| |
Collapse
|
8
|
Colocation for SLAM-Tracked VR Headsets with Hand Tracking. COMPUTERS 2021. [DOI: 10.3390/computers10050058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
In colocated multi-user Virtual Reality applications, relative user positions in the virtual environment need to match their relative positions in the physical tracking space. A mismatch between virtual and real relative user positions might lead to harmful events such as physical user collisions. This paper examines three calibration methods that enable colocated Virtual Reality scenarios for SLAM-tracked head-mounted displays without the need for an external tracking system. Two of these methods—fixed-point calibration and marked-based calibration—have been described in previous research; the third method that uses hand tracking capabilities of head-mounted displays is novel. We evaluated the accuracy of these three methods in an experimental procedure with two colocated Oculus Quest devices. The results of the evaluation show that our novel hand tracking-based calibration method provides better accuracy and consistency while at the same time being easy to execute. The paper further discusses the potential of all evaluated calibration methods.
Collapse
|