1
|
Yang SY, Oh YH. Development of neonatal Apgar scoring training program utilizing contactless hand tracking in immersive virtual reality. NURSE EDUCATION TODAY 2024; 140:106294. [PMID: 38944937 DOI: 10.1016/j.nedt.2024.106294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Revised: 06/03/2024] [Accepted: 06/19/2024] [Indexed: 07/02/2024]
Abstract
BACKGROUND The rapid and accurate assessment of neonatal asphyxia is critical to preventing potentially fatal outcomes. Therefore, nursing students must acquire the skills to assess newborn conditions immediately after birth and implement appropriate interventions. Virtual reality (VR) simulation education has emerged as a promising tool for nursing education, offering repetitive and customizable clinical training while ensuring patient safety and overcoming spatiotemporal limitations. AIM This study investigated the effects of a contactless hand-tracking-based immersive VR neonatal Apgar scoring program, adapted from experiential learning theory. DESIGN A non-randomized controlled trial with a pre-post-test, quasi-experimental design was conducted. SETTINGS The study was conducted at two nursing schools from July to October 2023. PARTICIPANTS Participants comprised nursing students holding bachelor's degrees in nursing, with three or four years of experience and successful completion of a neonatal nursing theory course. Additionally, individuals with at least six months of experience working in a neonatal ward or delivery room before enrolling in nursing school were eligible. METHODS The participants were divided into three groups: the VR group (n = 27) received contactless hand-tracking-based immersive VR neonatal Apgar scoring training; the simulation group (n = 28) received face-to-face Apgar scoring simulation training; and the control group (n = 26) received instruction on the Apgar scoring criteria. Changes in scores among the VR, simulation, and control groups were statistically compared using ANOVA with SPSS-WIN 27.0. RESULTS The VR group exhibited significant improvements in knowledge, learning satisfaction, self-confidence, immersion, and motivation compared to the simulation and control groups. Moreover, satisfaction was significantly higher in the VR group than in the simulation group. CONCLUSIONS The hand-tracking-based immersive VR neonatal Apgar scoring program represents an innovative and effective educational tool, prioritizing the privacy and rights of mothers and infants. It can potentially replace traditional delivery-room clinical training, which is observation-based and limited.
Collapse
Affiliation(s)
- Sun-Yi Yang
- College of Nursing, Konyang University, Daejeon, Republic of Korea.
| | - Yun-Hee Oh
- Department of Nursing, Cheju-Halla University, Jeju, Republic of Korea
| |
Collapse
|
2
|
Arensmeyer J, Bedetti B, Schnorr P, Buermann J, Zalepugas D, Schmidt J, Feodorovici P. A System for Mixed-Reality Holographic Overlays of Real-Time Rendered 3D-Reconstructed Imaging Using a Video Pass-through Head-Mounted Display-A Pathway to Future Navigation in Chest Wall Surgery. J Clin Med 2024; 13:2080. [PMID: 38610849 PMCID: PMC11012529 DOI: 10.3390/jcm13072080] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2024] [Revised: 03/26/2024] [Accepted: 03/27/2024] [Indexed: 04/14/2024] Open
Abstract
Background: Three-dimensional reconstructions of state-of-the-art high-resolution imaging are progressively being used more for preprocedural assessment in thoracic surgery. It is a promising tool that aims to improve patient-specific treatment planning, for example, for minimally invasive or robotic-assisted lung resections. Increasingly available mixed-reality hardware based on video pass-through technology enables the projection of image data as a hologram onto the patient. We describe the novel method of real-time 3D surgical planning in a mixed-reality setting by presenting three representative cases utilizing volume rendering. Materials: A mixed-reality system was set up using a high-performance workstation running a video pass-through-based head-mounted display. Image data from computer tomography were imported and volume-rendered in real-time to be customized through live editing. The image-based hologram was projected onto the patient, highlighting the regions of interest. Results: Three oncological cases were selected to explore the potentials of the mixed-reality system. Two of them presented large tumor masses in the thoracic cavity, while a third case presented an unclear lesion of the chest wall. We aligned real-time rendered 3D holographic image data onto the patient allowing us to investigate the relationship between anatomical structures and their respective body position. Conclusions: The exploration of holographic overlay has proven to be promising in improving preprocedural surgical planning, particularly for complex oncological tasks in the thoracic surgical field. Further studies on outcome-related surgical planning and navigation should therefore be conducted. Ongoing technological progress of extended reality hardware and intelligent software features will most likely enhance applicability and the range of use in surgical fields within the near future.
Collapse
Affiliation(s)
- Jan Arensmeyer
- Division of Thoracic Surgery, Department of General, Thoracic and Vascular Surgery, University Hospital Bonn, 53127 Bonn, Germany (P.F.)
- Bonn Surgical Technology Center (BOSTER), University Hospital Bonn, 53227 Bonn, Germany
| | - Benedetta Bedetti
- Division of Thoracic Surgery, Department of General, Thoracic and Vascular Surgery, University Hospital Bonn, 53127 Bonn, Germany (P.F.)
- Department of Thoracic Surgery, Helios Hospital Bonn/Rhein-Sieg, 53123 Bonn, Germany
| | - Philipp Schnorr
- Division of Thoracic Surgery, Department of General, Thoracic and Vascular Surgery, University Hospital Bonn, 53127 Bonn, Germany (P.F.)
- Department of Thoracic Surgery, Helios Hospital Bonn/Rhein-Sieg, 53123 Bonn, Germany
| | - Jens Buermann
- Division of Thoracic Surgery, Department of General, Thoracic and Vascular Surgery, University Hospital Bonn, 53127 Bonn, Germany (P.F.)
- Department of Thoracic Surgery, Helios Hospital Bonn/Rhein-Sieg, 53123 Bonn, Germany
| | - Donatas Zalepugas
- Division of Thoracic Surgery, Department of General, Thoracic and Vascular Surgery, University Hospital Bonn, 53127 Bonn, Germany (P.F.)
- Department of Thoracic Surgery, Helios Hospital Bonn/Rhein-Sieg, 53123 Bonn, Germany
| | - Joachim Schmidt
- Division of Thoracic Surgery, Department of General, Thoracic and Vascular Surgery, University Hospital Bonn, 53127 Bonn, Germany (P.F.)
- Bonn Surgical Technology Center (BOSTER), University Hospital Bonn, 53227 Bonn, Germany
- Department of Thoracic Surgery, Helios Hospital Bonn/Rhein-Sieg, 53123 Bonn, Germany
| | - Philipp Feodorovici
- Division of Thoracic Surgery, Department of General, Thoracic and Vascular Surgery, University Hospital Bonn, 53127 Bonn, Germany (P.F.)
- Bonn Surgical Technology Center (BOSTER), University Hospital Bonn, 53227 Bonn, Germany
| |
Collapse
|
3
|
Everard G, Burton Q, Van de Sype V, Bibentyo TN, Auvinet E, Edwards MG, Batcho CS, Lejeune T. Extended reality to assess post-stroke manual dexterity: contrasts between the classic box and block test, immersive virtual reality with controllers, with hand-tracking, and mixed-reality tests. J Neuroeng Rehabil 2024; 21:36. [PMID: 38491540 PMCID: PMC10941416 DOI: 10.1186/s12984-024-01332-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Accepted: 03/03/2024] [Indexed: 03/18/2024] Open
Abstract
BACKGROUND Recent technological advancements present promising opportunities to enhance the frequency and objectivity of functional assessments, aligning with recent stroke rehabilitation guidelines. Within this framework, we designed and adapted different manual dexterity tests in extended reality (XR), using immersive virtual reality (VR) with controllers (BBT-VR-C), immersive VR with hand-tracking (BBT-VR-HT), and mixed-reality (MD-MR). OBJECTIVE This study primarily aimed to assess and compare the validity of the BBT-VR-C, BBT-VR-HT and MD-MR to assess post-stroke manual dexterity. Secondary objectives were to evaluate reliability, usability and to define arm kinematics measures. METHODS A sample of 21 healthy control participants (HCP) and 21 stroke individuals with hemiparesis (IHP) completed three trials of the traditional BBT, the BBT-VR-C, BBT-VR-HT and MD-MR. Content validity of the different tests were evaluated by asking five healthcare professionals to rate the difficulty of performing each test in comparison to the traditional BBT. Convergent validity was evaluated through correlations between the scores of the traditional BBT and the XR tests. Test-retest reliability was assessed through correlations between the second and third trial and usability was assessed using the System Usability Scale (SUS). Lastly, upper limb movement smoothness (SPARC) was compared between IHP and HCP for both BBT-VR test versions. RESULTS For content validity, healthcare professionals rated the BBT-VR-HT (0[0-1]) and BBT-MR (0[0-1]) as equally difficult to the traditional BBT, whereas they rated BBT-VR-C as more difficult than the traditional BBT (1[0-2]). For IHP convergent validity, the Pearson tests demonstrated larger correlations between the scores of BBT and BBT-VR-HT (r = 0.94;p < 0.001), and BBT and MD-MR (r = 0.95;p < 0.001) than BBT and BBT-VR-C (r = 0.65;p = 0.001). BBT-VR-HT and MD-MR usability were both rated as excellent, with median SUS scores of 83[57.5-91.3] and 83[53.8-92.5] respectively. Excellent reliability was found for the BBT-VR-C (ICC = 0.96;p < 0.001), BBT-VR-HT (ICC = 0.96;p < 0.001) and BBT-MR (ICC = 0.99;p < 0.001). The usability of the BBT-VR-C was rated as good with a median SUS of 70[43.8-83.8]. Upper limb movements of HCP were significantly smoother than for IHP when completing either the BBT-VR-C (t = 2.05;p = 0.043) and the BBT-VR-HT (t = 5.21;p < 0.001). CONCLUSION The different XR manual tests are valid, short-term reliable and usable tools to assess post-stroke manual dexterity. TRIAL REGISTRATION https://clinicaltrials.gov/ct2/show/NCT04694833 ; Unique identifier: NCT04694833, Date of registration: 11/24/2020.
Collapse
Affiliation(s)
- Gauthier Everard
- Centre interdisciplinaire de recherche en réadaptation et intégration sociale, Université Laval, Québec, Canada
- Department of rehabilitation, Faculty of medicine, Laval University, Quebec, QC, Canada
- Neuro Musculo Skeletal Lab (NMSK), Secteur des Sciences de la Santé, Institut de Recherche Expérimentale et Clinique, Université catholique de Louvain, Brussels, Belgium
| | - Quentin Burton
- Neuro Musculo Skeletal Lab (NMSK), Secteur des Sciences de la Santé, Institut de Recherche Expérimentale et Clinique, Université catholique de Louvain, Brussels, Belgium
| | - Vincent Van de Sype
- Service de médecine physique et réadaptation, Cliniques universitaires Saint-Luc, Avenue Hippocrate 10, Brussels, 1200, Belgium
| | | | | | - Martin Gareth Edwards
- Psychological Sciences Research Institute (IPSY), Université Catholique de Louvain, Louvain‑la‑Neuve, Belgium
- Louvain Bionics, Université catholique de Louvain, Louvain-la-Neuve, Belgium
| | - Charles Sebiyo Batcho
- Centre interdisciplinaire de recherche en réadaptation et intégration sociale, Université Laval, Québec, Canada
- Department of rehabilitation, Faculty of medicine, Laval University, Quebec, QC, Canada
| | - Thierry Lejeune
- Neuro Musculo Skeletal Lab (NMSK), Secteur des Sciences de la Santé, Institut de Recherche Expérimentale et Clinique, Université catholique de Louvain, Brussels, Belgium.
- Service de médecine physique et réadaptation, Cliniques universitaires Saint-Luc, Avenue Hippocrate 10, Brussels, 1200, Belgium.
- Louvain Bionics, Université catholique de Louvain, Louvain-la-Neuve, Belgium.
| |
Collapse
|
4
|
Feodorovici P, Arensmeyer J, Schnorr P, Schmidt J. [Extended Reality (XR) - Applications in Thoracic Surgery]. Zentralbl Chir 2023; 148:367-375. [PMID: 37562396 DOI: 10.1055/a-2121-6478] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/12/2023]
Abstract
Extended reality (XR) includes the sub-terms of virtual reality (VR), augmented reality (AR) and mixed reality (MR) and describes interactive and immersive technologies that replace the real world with digital elements or seamlessly extend it with such approaches. XR thus offers a very wide range of possible applications in medicine. In surgery, and thoracic surgery in particular, XR technologies can be harnessed for treatment planning, navigation, training, and patient information. Such applications are increasingly being tested and need to be evaluated. We provide an overview of the status quo of technical development, current surgical applications of XR, and look into the future of the medical XR landscape with integration of artificial intelligence (AI).
Collapse
Affiliation(s)
- Philipp Feodorovici
- Sektion Thoraxchirurgie, Klinik und Poliklinik für Allgemein-, Viszeral-, Thorax- und Gefäßchirurgie, Universitätsklinikum Bonn, Bonn, Deutschland
| | - Jan Arensmeyer
- Sektion Thoraxchirurgie, Klinik und Poliklinik für Allgemein-, Viszeral-, Thorax- und Gefäßchirurgie, Universitätsklinikum Bonn, Bonn, Deutschland
| | - Philipp Schnorr
- Klinik für Thoraxchirurgie, Helios Klinikum Bonn/Rhein-Sieg, Bonn, Deutschland
| | - Joachim Schmidt
- Sektion Thoraxchirurgie, Klinik und Poliklinik für Allgemein-, Viszeral-, Thorax- und Gefäßchirurgie, Universitätsklinikum Bonn, Bonn, Deutschland
- Klinik für Thoraxchirurgie, Helios Klinikum Bonn/Rhein-Sieg, Bonn, Deutschland
| |
Collapse
|
5
|
Varela-Aldás J, Buele J, López I, Palacios-Navarro G. Influence of Hand Tracking in Immersive Virtual Reality for Memory Assessment. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2023; 20:4609. [PMID: 36901618 PMCID: PMC10002257 DOI: 10.3390/ijerph20054609] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/14/2023] [Revised: 02/26/2023] [Accepted: 03/03/2023] [Indexed: 06/18/2023]
Abstract
Few works analyze the parameters inherent to immersive virtual reality (IVR) in applications for memory evaluation. Specifically, hand tracking adds to the immersion of the system, placing the user in the first person with full awareness of the position of their hands. Thus, this work addresses the influence of hand tracking in memory assessment with IVR systems. For this, an application based on activities of daily living was developed, where the user must remember the location of the elements. The data collected by the application are the accuracy of the answers and the response time; the participants are 20 healthy subjects who pass the MoCA test with an age range between 18 to 60 years of age; the application was evaluated with classic controllers and with the hand tracking of the Oculus Quest 2. After the experimentation, the participants carried out presence (PQ), usability (UMUX), and satisfaction (USEQ) tests. The results indicate no difference with statistical significance between both experiments; controller experiments have 7.08% higher accuracy and 0.27 ys. faster response time. Contrary to expectations, presence was 1.3% lower for hand tracking, and usability (0.18%) and satisfaction (1.43%) had similar results. The findings indicate no evidence to determine better conditions in the evaluation of memory in this case of IVR with hand tracking.
Collapse
Affiliation(s)
- José Varela-Aldás
- Centro de Investigaciones de Ciencias Humanas y de la Educación—CICHE, Universidad Indoamérica, Ambato 180103, Ecuador
- SISAu Research Group, Facultad de Ingeniería, Industria y Producción FAINPRO, Universidad Indoamérica, Ambato 180103, Ecuador
| | - Jorge Buele
- SISAu Research Group, Facultad de Ingeniería, Industria y Producción FAINPRO, Universidad Indoamérica, Ambato 180103, Ecuador
- Department of Electronic Engineering and Communications, University of Zaragoza, 44003 Teruel, Spain
| | - Irene López
- SISAu Research Group, Facultad de Ingeniería, Industria y Producción FAINPRO, Universidad Indoamérica, Ambato 180103, Ecuador
| | | |
Collapse
|
6
|
Venkatakrishnan R, Venkatakrishnan R, Raveendranath B, Pagano CC, Robb AC, Lin WC, Babu SV. How Virtual Hand Representations Affect the Perceptions of Dynamic Affordances in Virtual Reality. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2023; PP:2258-2268. [PMID: 37027700 DOI: 10.1109/tvcg.2023.3247041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
User representations are critical to the virtual experience, and involve both the input device used to support interactions as well as how the user is virtually represented in the scene. Inspired by previous work that has shown effects of user representations on the perceptions of relatively static affordances, we attempt to investigate how end-effector representations affect the perceptions of affordances that dynamically change over time. Towards this end, we empirically evaluated how different virtual hand representations affect users' perceptions of dynamic affordances in an object retrieval task wherein users were tasked with retrieving a target from a box for a number of trials while avoiding collisions with its moving doors. We employed a 3 (virtual end-effector representation) X 13 (frequency of moving doors) X 2 (target object size) multi-factorial design, manipulating the input modality and its concomitant virtual end-effector representation as a between-subjects factor across three experimental conditions: (1) Controller (using a controller represented as a virtual controller); (2) Controller-hand (using a controller represented as a virtual hand); (3) Glove (using a hand tracked hi-fidelity glove represented as a virtual hand). Results indicated that the controller-hand condition produced lower levels of performance than both the other conditions. Furthermore, users in this condition exhibited a diminished ability to calibrate their performance over trials. Overall, we find that representing the end-effector as a hand tends to increase embodiment but can also come at the cost of performance, or an increased workload due to a discordant mapping between the virtual representation and the input modality used. It follows that VR system designers should carefully consider the priorities and target requirements of the application being developed when choosing the type of end-effector representation for users to embody in immersive virtual experiences.
Collapse
|
7
|
Trade-Off between Task Accuracy, Task Completion Time and Naturalness for Direct Object Manipulation in Virtual Reality. MULTIMODAL TECHNOLOGIES AND INTERACTION 2022. [DOI: 10.3390/mti6010006] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Virtual reality devices are used for several application domains, such as medicine, entertainment, marketing and training. A handheld controller is the common interaction method for direct object manipulation in virtual reality environments. Using hands would be a straightforward way to directly manipulate objects in the virtual environment if hand-tracking technology were reliable enough. In recent comparison studies, hand-based systems compared unfavorably against the handheld controllers in task completion times and accuracy. In our controlled study, we compare these two interaction techniques with a new hybrid interaction technique which combines the controller tracking with hand gestures for a rigid object manipulation task. The results demonstrate that the hybrid interaction technique is the most preferred because it is intuitive, easy to use, fast, reliable and it provides haptic feedback resembling the real-world object grab. This suggests that there is a trade-off between naturalness, task accuracy and task completion time when using these direct manipulation interaction techniques, and participants prefer to use interaction techniques that provide a balance between these three factors.
Collapse
|