1
|
Karnatz N, Schwerter M, Liu S, Parviz A, Wilkat M, Rana M. Mixed Reality as a Digital Visualisation Solution for the Head and Neck Tumour Board: Application Creation and Implementation Study. Cancers (Basel) 2024; 16:1392. [PMID: 38611070 PMCID: PMC11011089 DOI: 10.3390/cancers16071392] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2024] [Revised: 03/21/2024] [Accepted: 03/26/2024] [Indexed: 04/14/2024] Open
Abstract
The preparation and implementation of interdisciplinary oncological case reviews are time-consuming and complex. The variety of clinical and radiological information must be presented in a clear and comprehensible manner. Only if all relevant patient-specific information is demonstrated in a short time frame can well-founded treatment decisions be made on this basis. Mixed reality (MR) technology as a multimodal interactive user interface could enhance understanding in multidisciplinary collaboration by visualising radiological or clinical data. The aim of the work was to develop an MR-based software prototype for a head and neck tumour board (HNTB) to support clinical decision-making. The article describes the development phases and workflows in the planning and creation of a MR-based software prototype that were required to meet the multidisciplinary characteristics of a HNTB.
Collapse
Affiliation(s)
- Nadia Karnatz
- Department of Oral and Plastic Maxillofacial Surgery, Heinrich Heine University Hospital Düsseldorf, Moorenstraße 5, 40225 Düsseldorf, Germany (M.R.)
| | | | - Shufang Liu
- Brainlab AG, Olof-Palme-Str. 9, 81829 München, Germany
| | - Aida Parviz
- Department of Oral and Plastic Maxillofacial Surgery, Heinrich Heine University Hospital Düsseldorf, Moorenstraße 5, 40225 Düsseldorf, Germany (M.R.)
| | - Max Wilkat
- Department of Oral and Plastic Maxillofacial Surgery, Heinrich Heine University Hospital Düsseldorf, Moorenstraße 5, 40225 Düsseldorf, Germany (M.R.)
| | - Majeed Rana
- Department of Oral and Plastic Maxillofacial Surgery, Heinrich Heine University Hospital Düsseldorf, Moorenstraße 5, 40225 Düsseldorf, Germany (M.R.)
| |
Collapse
|
2
|
van Doormaal JAM, van Doormaal TPC. Augmented Reality in Neurosurgery. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2024; 1462:351-374. [PMID: 39523276 DOI: 10.1007/978-3-031-64892-2_21] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2024]
Abstract
Augmented Reality (AR) involves superimposing digital content onto the real environment. AR has evolved into a viable tool in neurosurgery, enhancing intraoperative navigation, medical education and surgical training by integrating anatomical data with the real world. Neurosurgical AR relies on several key techniques to be successful, which includes image segmentation, model rendering, AR projection, and image-to-patient registration. For each of these technical components, different solutions exist, with each having their own advantages and limitations.Intraoperative AR applications cover diverse neurosurgical disciplines including vascular, oncological, spinal, and functional surgeries. Preliminary studies indicate that AR may improve the understanding of complex anatomical structures and offer sufficient accuracy for use as a navigational tool. Additionally, AR shows promise in enhancing surgical training and patient education through interactive 3D models, aiding in the comprehension of intricate anatomical details. Despite its potential, the widespread adoption of AR in clinical settings depends on overcoming technical limitations and validating its clinical efficacy.
Collapse
Affiliation(s)
- Jesse A M van Doormaal
- Department of Neurosurgery, University Medical Centre Utrecht, Utrecht, The Netherlands.
| | | |
Collapse
|
3
|
Wu Y, Ye J, Xiao M. A Pterygopalatine Fossa Abscess Caused by Streptococcus Constellatus With Ptosis as Initial Presentation: Case Report. EAR, NOSE & THROAT JOURNAL 2023:1455613231212047. [PMID: 38031416 DOI: 10.1177/01455613231212047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2023] Open
Abstract
Pterygopalatine fossa (PPF) abscess is an exceedingly rare occurrence. Typically, it has been reported in association with odontogenic infections or hematogenous dissemination. In this report, we present the first case of PPF abscess where no obvious underlying cause or primary infection was evident. The patient initially presented with a 2-week history of left ptosis and persistent headaches, which were subsequently accompanied by fever and limited mouth opening. Based on elevated inflammatory markers and imaging findings, the diagnosis of a PPF abscess was established. Successful treatment was achieved through surgical drainage using an external cervical approach. Cultures from the aspirated pus identified Streptococcus constellatus as the causative agent. This case highlights the importance of considering PPF abscess in patients presenting with ptosis as an initial symptom. When diagnosed, surgical drainage should be considered as a viable therapeutic option.
Collapse
Affiliation(s)
- Yuting Wu
- Department of Otolaryngology Head and Neck Surgery, Sir Run Run Shaw Hospital, College of Medicine, Zhejiang University, Hangzhou, Zhejiang, China
| | - Jing Ye
- Department of Otolaryngology Head and Neck Surgery, Sir Run Run Shaw Hospital, College of Medicine, Zhejiang University, Hangzhou, Zhejiang, China
| | - Mang Xiao
- Department of Otolaryngology Head and Neck Surgery, Sir Run Run Shaw Hospital, College of Medicine, Zhejiang University, Hangzhou, Zhejiang, China
| |
Collapse
|
4
|
Gsaxner C, Li J, Pepe A, Jin Y, Kleesiek J, Schmalstieg D, Egger J. The HoloLens in medicine: A systematic review and taxonomy. Med Image Anal 2023; 85:102757. [PMID: 36706637 DOI: 10.1016/j.media.2023.102757] [Citation(s) in RCA: 30] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Revised: 01/05/2023] [Accepted: 01/18/2023] [Indexed: 01/22/2023]
Abstract
The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.
Collapse
Affiliation(s)
- Christina Gsaxner
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; BioTechMed, 8010 Graz, Austria.
| | - Jianning Li
- Institute of AI in Medicine, University Medicine Essen, 45131 Essen, Germany; Cancer Research Center Cologne Essen, University Medicine Essen, 45147 Essen, Germany
| | - Antonio Pepe
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; BioTechMed, 8010 Graz, Austria
| | - Yuan Jin
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; Research Center for Connected Healthcare Big Data, Zhejiang Lab, Hangzhou, 311121 Zhejiang, China
| | - Jens Kleesiek
- Institute of AI in Medicine, University Medicine Essen, 45131 Essen, Germany; Cancer Research Center Cologne Essen, University Medicine Essen, 45147 Essen, Germany
| | - Dieter Schmalstieg
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; BioTechMed, 8010 Graz, Austria
| | - Jan Egger
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; Institute of AI in Medicine, University Medicine Essen, 45131 Essen, Germany; BioTechMed, 8010 Graz, Austria; Cancer Research Center Cologne Essen, University Medicine Essen, 45147 Essen, Germany
| |
Collapse
|
5
|
Palumbo A. Microsoft HoloLens 2 in Medical and Healthcare Context: State of the Art and Future Prospects. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22207709. [PMID: 36298059 PMCID: PMC9611914 DOI: 10.3390/s22207709] [Citation(s) in RCA: 42] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/07/2022] [Revised: 09/29/2022] [Accepted: 10/07/2022] [Indexed: 05/08/2023]
Abstract
In the world reference context, although virtual reality, augmented reality and mixed reality have been emerging methodologies for several years, only today technological and scientific advances have made them suitable to revolutionize clinical care and medical contexts through the provision of enhanced functionalities and improved health services. This systematic review provides the state-of-the-art applications of the Microsoft® HoloLens 2 in a medical and healthcare context. Focusing on the potential that this technology has in providing digitally supported clinical care, also but not only in relation to the COVID-19 pandemic, studies that proved the applicability and feasibility of HoloLens 2 in a medical and healthcare scenario were considered. The review presents a thorough examination of the different studies conducted since 2019, focusing on HoloLens 2 medical sub-field applications, device functionalities provided to users, software/platform/framework used, as well as the study validation. The results provided in this paper could highlight the potential and limitations of the HoloLens 2-based innovative solutions and bring focus to emerging research topics, such as telemedicine, remote control and motor rehabilitation.
Collapse
Affiliation(s)
- Arrigo Palumbo
- Department of Medical and Surgical Sciences, Magna Græcia University, 88100 Catanzaro, Italy
| |
Collapse
|
6
|
The intraoperative use of augmented and mixed reality technology to improve surgical outcomes: A systematic review. Int J Med Robot 2022; 18:e2450. [DOI: 10.1002/rcs.2450] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Revised: 07/23/2022] [Accepted: 07/27/2022] [Indexed: 11/07/2022]
|
7
|
Digital Transformation Will Change Medical Education and Rehabilitation in Spine Surgery. Medicina (B Aires) 2022; 58:medicina58040508. [PMID: 35454347 PMCID: PMC9030988 DOI: 10.3390/medicina58040508] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2022] [Revised: 03/22/2022] [Accepted: 03/31/2022] [Indexed: 12/25/2022] Open
Abstract
The concept of minimally invasive spine therapy (MIST) has been proposed as a treatment strategy to reduce the need for overall patient care, including not only minimally invasive spine surgery (MISS) but also conservative treatment and rehabilitation. To maximize the effectiveness of patient care in spine surgery, the educational needs of medical students, residents, and patient rehabilitation can be enhanced by digital transformation (DX), including virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality (XR), three-dimensional (3D) medical images and holograms; wearable sensors, high-performance video cameras, fifth-generation wireless system (5G) and wireless fidelity (Wi-Fi), artificial intelligence, and head-mounted displays (HMDs). Furthermore, to comply with the guidelines for social distancing due to the unexpected COVID-19 pandemic, the use of DX to maintain healthcare and education is becoming more innovative than ever before. In medical education, with the evolution of science and technology, it has become mandatory to provide a highly interactive educational environment and experience using DX technology for residents and medical students, known as digital natives. This study describes an approach to pre- and intraoperative medical education and postoperative rehabilitation using DX in the field of spine surgery that was implemented during the COVID-19 pandemic and will be utilized thereafter.
Collapse
|