1
|
Liu Y, Wu J, Liu D, Xiang D, Wu X, Wang T. The application of mixed reality technique in oromaxillo-facial reconstruction with the perforator flap for malignant tumor patients. Front Oncol 2024; 14:1437598. [PMID: 39099694 PMCID: PMC11294116 DOI: 10.3389/fonc.2024.1437598] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2024] [Accepted: 07/08/2024] [Indexed: 08/06/2024] Open
Abstract
Objectives The integration of quantitative imaging techniques such as computed tomography (CT) and magnetic resonance imaging (MRI) with mixed reality (MR) technology holds promise for enhancing the diagnosis, prognosis, and treatment monitoring of cancer. This study compares the characteristics and effects of MR and color Doppler ultrasound (CDU) in the localization of perforator blood vessels in the lower extremities. Methods Two techniques were used to locate the perforator vessels in 40 cases of maxillofacial defect repair using perforator flaps from the lower extremities. The number of perforator vessels located in the flap area and the actual number of perforator vessels explored during the surgery were recorded. The recognition rate was calculated and the operation time and blood loss were recorded for each case. Results The recognition rates of MR technology and CDU in perforating vessels of the lower limbs were 93.9% and 97.2%, respectively (p > 0.05). The operation time was 52-74 minutes, 65-88 minutes (p > 0.05). The average bleeding volumes were 24 and 56 ml (p < 0.05), respectively. All perforator flaps were alive. One flap had a crisis and recovered after emergency exploratory treatment. Thirty donor sites of the lower extremities were directly sutured, and wounds were closed by abdominal skin grafting in 10 cases. Conclusion MR technology for successfully identifying perforator vessels can shorten the operation time, reduce the amount of bleeding in the donor site, and reduce trauma to the donor site.
Collapse
Affiliation(s)
- Yixiu Liu
- Head and Neck Surgery, Chongqing University Cancer Hospital, Chongqing, China
| | - Jian Wu
- Head and Neck Surgery, Chongqing University Cancer Hospital, Chongqing, China
| | - Daide Liu
- Department of Surgery, People’s Hospital of Shizhu, Chongqing, China
| | - Dalan Xiang
- Department of Surgery, People’s Hospital of Shizhu, Chongqing, China
| | - Xiaoyue Wu
- Head and Neck Surgery, Chongqing University Cancer Hospital, Chongqing, China
| | - Ting Wang
- Internal Medicine-Oncology, Chongqing University Cancer Hospital, Chongqing, China
| |
Collapse
|
2
|
Boretto L, Pelanis E, Regensburger A, Fretland ÅA, Edwin B, Elle OJ. Hybrid optical-vision tracking in laparoscopy: accuracy of navigation and ultrasound reconstruction. MINIM INVASIV THER 2024; 33:176-183. [PMID: 38334755 DOI: 10.1080/13645706.2024.2313032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Accepted: 01/11/2024] [Indexed: 02/10/2024]
Abstract
INTRODUCTION The use of laparoscopic and robotic liver surgery is increasing. However, it presents challenges such as limited field of view and organ deformations. Surgeons rely on laparoscopic ultrasound (LUS) for guidance, but mentally correlating ultrasound images with pre-operative volumes can be difficult. In this direction, surgical navigation systems are being developed to assist with intra-operative understanding. One approach is performing intra-operative ultrasound 3D reconstructions. The accuracy of these reconstructions depends on tracking the LUS probe. MATERIAL AND METHODS This study evaluates the accuracy of LUS probe tracking and ultrasound 3D reconstruction using a hybrid tracking approach. The LUS probe is tracked from laparoscope images, while an optical tracker tracks the laparoscope. The accuracy of hybrid tracking is compared to full optical tracking using a dual-modality tool. Ultrasound 3D reconstruction accuracy is assessed on an abdominal phantom with CT transformed into the optical tracker's coordinate system. RESULTS Hybrid tracking achieves a tracking error < 2 mm within 10 cm between the laparoscope and the LUS probe. The ultrasound reconstruction accuracy is approximately 2 mm. CONCLUSION Hybrid tracking shows promising results that can meet the required navigation accuracy for laparoscopic liver surgery.
Collapse
Affiliation(s)
- Luca Boretto
- Department of Informatics, The Faculty of Mathematics and Natural Sciences, University of Oslo, Oslo, Norway
- Siemens Healthcare AS, Oslo, Norway
| | - Egidijus Pelanis
- The Intervention Centre, Oslo University Hospital Rikshospitalet, Oslo, Norway
| | | | - Åsmund Avdem Fretland
- The Intervention Centre, Oslo University Hospital Rikshospitalet, Oslo, Norway
- Department of HPB Surgery, Oslo University Hospital Rikshospitalet, Oslo, Norway
| | - Bjørn Edwin
- The Intervention Centre, Oslo University Hospital Rikshospitalet, Oslo, Norway
- Department of HPB Surgery, Oslo University Hospital Rikshospitalet, Oslo, Norway
- Faculty of Medicine, Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Ole Jakob Elle
- Department of Informatics, The Faculty of Mathematics and Natural Sciences, University of Oslo, Oslo, Norway
- The Intervention Centre, Oslo University Hospital Rikshospitalet, Oslo, Norway
| |
Collapse
|
3
|
Shi X, Guo H, Zhu C, Qiu G, Liang T, Lian J, Ma Y, Wang S, Li X. Mixed reality in primary retroperitoneal tumour surgery: Evaluation of preoperative and intraoperative application value. Int J Med Robot 2023:e2584. [PMID: 37792998 DOI: 10.1002/rcs.2584] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Revised: 09/06/2023] [Accepted: 09/25/2023] [Indexed: 10/06/2023]
Abstract
OBJECTIVE To evaluate the feasibility and application value of mixed reality technology (MR) in Primary retroperitoneal tumour (PRT) surgery. METHODS From 276 patients who underwent PRT resection at the First Affiliated Hospital of Xi'an Jiaotong University, we screened 46 patients who underwent MR-assisted retroperitoneal tumour resection and 46 patients who underwent tumour resection without MR assistance. The intraoperative and postoperative recovery of the patients in both groups were compared, and the reliability and validity of the application of MR were further examined using the Likert scale. RESULTS There was a significant difference in the mean intraoperative bleeding volume between the two groups, but it was reduced in the MR group. The results of the Likert scale showed higher scores in the MR group than non-MR group. CONCLUSIONS MR can be used to assist PRT resection and has great potential to improve the rate of complete retroperitoneal tumour resection.
Collapse
Affiliation(s)
- Xiaoqiang Shi
- Department of General Surgery, The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, China
| | - Hainan Guo
- Department of General Surgery, The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, China
| | - Chao Zhu
- Department of General Surgery, The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, China
- Department of General Surgery, The People's Hospital of Suide County, Suide, Shaanxi, China
| | - Guanglin Qiu
- Department of General Surgery, The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, China
| | - Ting Liang
- Department of Radiology, The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, China
- Department of Biomedical Engineering, the Key Laboratory of Biomedical Information Engineering of the Ministry of Education, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an, Shaanxi, China
| | - Jie Lian
- Department of Pathology, The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, China
| | - Yanfei Ma
- Department of General Surgery, The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, China
- Second Department of General Surgery, The Suide Campus, The First Hospital of Yulin, Yulin, Shaanxi, China
| | - Shufeng Wang
- Department of General Surgery, The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, China
| | - Xuqi Li
- Department of General Surgery, The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, China
| |
Collapse
|
4
|
Montalti R, Rompianesi G, Cassese G, Pegoraro F, Giglio MC, De Simone G, Rashidian N, Venetucci P, Troisi RI. Role of preoperative 3D rendering for minimally invasive parenchyma sparing liver resections. HPB (Oxford) 2023:S1365-182X(23)00125-9. [PMID: 37149483 DOI: 10.1016/j.hpb.2023.04.008] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Revised: 03/23/2023] [Accepted: 04/13/2023] [Indexed: 05/08/2023]
Abstract
BACKGROUND 3D rendering (3DR) represents a promising approach to plan surgical strategies. The study aimed to compare the results of minimally invasive liver resections (MILS) in patients with 3DR versus conventional 2D CT-scan. METHODS We performed 118 3DR for various indications; the patients underwent a preoperative tri-phasic CT-scan and rendered with Synapse3D® Software. Fifty-six patients undergoing MILS with pre-operative 3DR were compared to a similar cohort of 127 patients undergoing conventional pre-operative 2D CT-scan using the propensity score matching (PSM) analysis. RESULTS The 3DR mandated pre-operative surgical plan variations in 33.9% cases, contraindicated surgery in 12.7%, providing a new surgical indication in 5.9% previously excluded cases. PSM identified 39 patients in both groups with comparable results in terms of conversion rates, blood loss, blood transfusions, parenchymal R1-margins, grade ≥3 Clavien-Dindo complications, 90-days mortality, and hospital stay respectively in 3DR and conventional 2D. Operative time was significantly increased in the 3DR group (402 vs. 347 min, p = 0.020). Vascular R1 resections were 25.6% vs 7.7% (p = 0.068), while the conversion rate was 0% vs 10.2% (p = 0.058), respectively, for 3DR group vs conventional 2D. CONCLUSION 3DR may help in surgical planning increasing resectability rate while reducing conversion rates, allowing the precise identification of anatomical landmarks in minimally invasive parenchyma-preserving liver resections.
Collapse
Affiliation(s)
- Roberto Montalti
- Department of Public Health, Federico II University, Naples, Italy; Department of Clinical Medicine and Surgery, Division of HPB, Minimally Invasive and Robotic Surgery, Renal Transplant Service, Federico II University, Naples, Italy
| | - Gianluca Rompianesi
- Department of Clinical Medicine and Surgery, Division of HPB, Minimally Invasive and Robotic Surgery, Renal Transplant Service, Federico II University, Naples, Italy
| | - Gianluca Cassese
- Department of Clinical Medicine and Surgery, Division of HPB, Minimally Invasive and Robotic Surgery, Renal Transplant Service, Federico II University, Naples, Italy
| | - Francesca Pegoraro
- Department of Clinical Medicine and Surgery, Division of HPB, Minimally Invasive and Robotic Surgery, Renal Transplant Service, Federico II University, Naples, Italy
| | - Mariano C Giglio
- Department of Clinical Medicine and Surgery, Division of HPB, Minimally Invasive and Robotic Surgery, Renal Transplant Service, Federico II University, Naples, Italy
| | - Giuseppe De Simone
- Department of Anesthesiology and Intensive Care, Federico II University, Naples, Italy
| | - Nikdokht Rashidian
- Department of Hepatobiliary and Liver Transplantation, Ghent University Hospital, Ghent, Belgium
| | - Pietro Venetucci
- Division of Medical Imaging and Radiotherapy, Department of Onco-Hematology, Diagnostic and Morphologic Imaging, and Forensic Medicine, Federico II University, Naples, Italy
| | - Roberto I Troisi
- Department of Clinical Medicine and Surgery, Division of HPB, Minimally Invasive and Robotic Surgery, Renal Transplant Service, Federico II University, Naples, Italy.
| |
Collapse
|
5
|
Gsaxner C, Li J, Pepe A, Jin Y, Kleesiek J, Schmalstieg D, Egger J. The HoloLens in medicine: A systematic review and taxonomy. Med Image Anal 2023; 85:102757. [PMID: 36706637 DOI: 10.1016/j.media.2023.102757] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Revised: 01/05/2023] [Accepted: 01/18/2023] [Indexed: 01/22/2023]
Abstract
The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.
Collapse
Affiliation(s)
- Christina Gsaxner
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; BioTechMed, 8010 Graz, Austria.
| | - Jianning Li
- Institute of AI in Medicine, University Medicine Essen, 45131 Essen, Germany; Cancer Research Center Cologne Essen, University Medicine Essen, 45147 Essen, Germany
| | - Antonio Pepe
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; BioTechMed, 8010 Graz, Austria
| | - Yuan Jin
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; Research Center for Connected Healthcare Big Data, Zhejiang Lab, Hangzhou, 311121 Zhejiang, China
| | - Jens Kleesiek
- Institute of AI in Medicine, University Medicine Essen, 45131 Essen, Germany; Cancer Research Center Cologne Essen, University Medicine Essen, 45147 Essen, Germany
| | - Dieter Schmalstieg
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; BioTechMed, 8010 Graz, Austria
| | - Jan Egger
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; Institute of AI in Medicine, University Medicine Essen, 45131 Essen, Germany; BioTechMed, 8010 Graz, Austria; Cancer Research Center Cologne Essen, University Medicine Essen, 45147 Essen, Germany
| |
Collapse
|
6
|
Shahbaz M, Miao H, Farhaj Z, Gong X, Weikai S, Dong W, Jun N, Shuwei L, Yu D. Mixed reality navigation training system for liver surgery based on a high-definition human cross-sectional anatomy data set. Cancer Med 2023; 12:7992-8004. [PMID: 36607128 PMCID: PMC10134360 DOI: 10.1002/cam4.5583] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Revised: 11/24/2022] [Accepted: 12/17/2022] [Indexed: 01/07/2023] Open
Abstract
OBJECTIVES This study aims to use the three-dimensional (3D) mixed-reality model of liver, entailing complex intrahepatic systems and to deeply study the anatomical structures and to promote the training, diagnosis and treatment of liver diseases. METHODS Vascular perfusion human specimens were used for thin-layer frozen milling to obtain liver cross-sections. The 104-megapixel-high-definition cross sectional data set was established and registered to achieve structure identification and manual segmentation. The digital model was reconstructed and data was used to print a 3D hepatic model. The model was combined with HoloLens mixed reality technology to reflect the complex relationships of intrahepatic systems. We simulated 3D patient specific anatomy for identification and preoperative planning, conducted a questionnaire survey, and evaluated the results. RESULTS The 3D digital model and 1:1 transparent and colored model of liver established truly reflected intrahepatic vessels and their complex relationships. The reconstructed model imported into HoloLens could be accurately matched with the 3D model. Only 7.7% participants could identify accessory hepatic veins. The depth and spatial-relationship of intrahepatic structures were better understandable for 92%. The 100%, 84.6%, 69% and 84% believed the 3D models were useful in planning, safer surgical paths, reducing intraoperative complications and training of young surgeons respectively. CONCLUSIONS A detailed 3D model can be reconstructed using the higher quality cross-sectional anatomical data set. When combined with 3D printing and HoloLens technology, a novel hybrid-reality navigation-training system for liver surgery is created. Mixed Reality training is a worthy alternative to provide 3D information to clinicians and its possible application in surgery. This conclusion was obtained based on a questionnaire and evaluation. Surgeons with extensive experience in surgical operations perceived in the questionnaire that this technology might be useful in liver surgery, would help in precise preoperative planning, accurate intraoperative identification, and reduction of hepatic injury.
Collapse
Affiliation(s)
- Muhammad Shahbaz
- Department of Radiology, Qilu Hospital of Shandong UniversityJinanShandongChina
- Research Center for Sectional and Imaging AnatomyDigital Human Institute, School of Basic Medical Science, Shandong UniversityJinanShandongChina
- Department of General SurgeryQilu Hospital of Shandong UniversityJinanShandongChina
| | - Huachun Miao
- Department of Anatomy, Wannan Medical CollegeWuhuAnhuiChina
| | - Zeeshan Farhaj
- Department of Cardiovascular Surgery, Shandong Qianfoshan Hospital, Cheeloo College of MedicineShandong UniversityJinanShandongChina
| | - Xin Gong
- Department of Anatomy, Wannan Medical CollegeWuhuAnhuiChina
| | - Sun Weikai
- Department of Radiology, Qilu Hospital of Shandong UniversityJinanShandongChina
| | - Wenqing Dong
- Department of Anatomy, Wannan Medical CollegeWuhuAnhuiChina
| | - Niu Jun
- Department of General SurgeryQilu Hospital of Shandong UniversityJinanShandongChina
| | - Liu Shuwei
- Research Center for Sectional and Imaging AnatomyDigital Human Institute, School of Basic Medical Science, Shandong UniversityJinanShandongChina
| | - Dexin Yu
- Department of Radiology, Qilu Hospital of Shandong UniversityJinanShandongChina
| |
Collapse
|
7
|
Aghayan DL, d'Albenzio G, Fretland ÅA, Pelanis E, Røsok BI, Yaqub S, Palomar R, Edwin B. Laparoscopic parenchyma-sparing liver resection for large (≥ 50 mm) colorectal metastases. Surg Endosc 2023; 37:225-233. [PMID: 35922606 PMCID: PMC9839797 DOI: 10.1007/s00464-022-09493-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2022] [Accepted: 07/16/2022] [Indexed: 01/18/2023]
Abstract
BACKGROUND Traditionally, patients with large liver tumors (≥ 50 mm) have been considered for anatomic major hepatectomy. Laparoscopic resection of large liver lesions is technically challenging and often performed by surgeons with extensive experience. The current study aimed to evaluate the surgical and oncologic safety of laparoscopic parenchyma-sparing liver resection in patients with large colorectal metastases. METHODS Patients who primarily underwent laparoscopic parenchyma-sparing liver resection (less than 3 consecutive liver segments) for colorectal liver metastases between 1999 and 2019 at Oslo University Hospital were analyzed. In some recent cases, a computer-assisted surgical planning system was used to better visualize and understand the patients' liver anatomy, as well as a tool to further improve the resection strategy. The surgical and oncologic outcomes of patients with large (≥ 50 mm) and small (< 50 mm) tumors were compared. Multivariable Cox-regression analysis was performed to identify risk factors for survival. RESULTS In total 587 patients met the inclusion criteria (large tumor group, n = 59; and small tumor group, n = 528). Median tumor size was 60 mm (range, 50-110) in the large tumor group and 21 mm (3-48) in the small tumor group (p < 0.001). Patient age and CEA level were higher in the large tumor group (8.4 μg/L vs. 4.6 μg/L, p < 0.001). Operation time and conversion rate were similar, while median blood loss was higher in the large tumor group (500 ml vs. 200 ml, p < 0.001). Patients in the large tumor group had shorter 5 year overall survival (34% vs 49%, p = 0.027). However, in the multivariable Cox-regression analysis tumor size did not impact survival, unlike parameters such as age, ASA score, CEA level, extrahepatic disease at liver surgery, and positive lymph nodes in the primary tumor. CONCLUSION Laparoscopic parenchyma-sparing resections for large colorectal liver metastases provide satisfactory short and long-term outcomes.
Collapse
Affiliation(s)
- Davit L Aghayan
- The Intervention Centre, Oslo University Hospital - Rikshospitalet, 0027, Oslo, Norway.
- Department of Surgery N1, Yerevan State Medical University After M. Heratsi, Yerevan, Armenia.
| | - Gabriella d'Albenzio
- The Intervention Centre, Oslo University Hospital - Rikshospitalet, 0027, Oslo, Norway
- Department of Informatics, University of Oslo, Oslo, Norway
| | - Åsmund A Fretland
- The Intervention Centre, Oslo University Hospital - Rikshospitalet, 0027, Oslo, Norway
- Institute of Clinical Medicine, Medical Faculty, University of Oslo, Oslo, Norway
- Department of HPB Surgery, Oslo University Hospital - Rikshospitalet, Oslo, Norway
| | - Egidijus Pelanis
- The Intervention Centre, Oslo University Hospital - Rikshospitalet, 0027, Oslo, Norway
- Institute of Clinical Medicine, Medical Faculty, University of Oslo, Oslo, Norway
| | - Bård I Røsok
- Department of HPB Surgery, Oslo University Hospital - Rikshospitalet, Oslo, Norway
| | - Sheraz Yaqub
- Institute of Clinical Medicine, Medical Faculty, University of Oslo, Oslo, Norway
- Department of HPB Surgery, Oslo University Hospital - Rikshospitalet, Oslo, Norway
| | - Rafael Palomar
- The Intervention Centre, Oslo University Hospital - Rikshospitalet, 0027, Oslo, Norway
- Department of Computer Science, Norwegian University of Science and Technology, Gjøvik, Norway
| | - Bjørn Edwin
- The Intervention Centre, Oslo University Hospital - Rikshospitalet, 0027, Oslo, Norway
- Institute of Clinical Medicine, Medical Faculty, University of Oslo, Oslo, Norway
- Department of HPB Surgery, Oslo University Hospital - Rikshospitalet, Oslo, Norway
| |
Collapse
|
8
|
Antonelli M, Lucignani M, Parrillo C, Grassi F, Figà Talamanca L, Rossi Espagnet MC, Gandolfo C, Secinaro A, Pasquini L, De Benedictis A, Placidi E, De Palma L, Marras CE, Marasi A, Napolitano A. Magnetic resonance imaging based neurosurgical planning on hololens 2: A feasibility study in a paediatric hospital. Digit Health 2023; 9:20552076231214066. [PMID: 38025111 PMCID: PMC10656794 DOI: 10.1177/20552076231214066] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 01/31/2023] [Indexed: 12/01/2023] Open
Abstract
Objective The goal of this work is to show how to implement a mixed reality application (app) for neurosurgery planning based on neuroimaging data, highlighting the strengths and weaknesses of its design. Methods Our workflow explains how to handle neuroimaging data, including how to load morphological, functional and diffusion tensor imaging data into a mixed reality environment, thus creating a first guide of this kind. Brain magnetic resonance imaging data from a paediatric patient were acquired using a 3 T Siemens Magnetom Skyra scanner. Initially, this raw data underwent specific software pre-processing and were subsequently transformed to ensure seamless integration with the mixed reality app. After that, we created three-dimensional models of brain structures and the mixed reality environment using Unity™ engine together with Microsoft® HoloLens 2™ device. To get an evaluation of the app we submitted a questionnaire to four neurosurgeons. To collect data concerning the performance of a user session we used Unity Performance Profiler. Results The use of the interactive features, such as rotating, scaling and moving models and browsing through menus, provided by the app had high scores in the questionnaire, and their use can still be improved as suggested by the performance data collected. The questionnaire's average scores were high, so the overall experiences of using our mixed reality app were positive. Conclusion We have successfully created a valuable and easy-to-use neuroimaging data mixed reality app, laying the foundation for more future clinical uses, as more models and data derived from various biomedical images can be imported.
Collapse
Affiliation(s)
- Martina Antonelli
- Medical Physics Department, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
| | - Martina Lucignani
- Medical Physics Department, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
| | - Chiara Parrillo
- Medical Physics Department, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
| | - Francesco Grassi
- Medical Physics Department, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
| | - Lorenzo Figà Talamanca
- Neuroradiology Unit, Imaging Department, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
| | - Maria C Rossi Espagnet
- Neuroradiology Unit, Imaging Department, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
- Department of Neuroscience, Mental Health and Sensory Organs (NESMOS), Sant’Andrea Hospital, Sapienza University, Roma, Italy
| | - Carlo Gandolfo
- Neuroradiology Unit, Imaging Department, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
| | - Aurelio Secinaro
- Advanced Cardiovascular Imaging Unit, Department of Imaging, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
| | - Luca Pasquini
- Neuroradiology Unit, Imaging Department, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
- Department of Neuroscience, Mental Health and Sensory Organs (NESMOS), Sant’Andrea Hospital, Sapienza University, Roma, Italy
| | - Alessandro De Benedictis
- Pediatric Neurosurgery Unit, Department of Neuroscience and Neurorehabilitation, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
| | - Elisa Placidi
- Medical Physics UOC, Fondazione Policlinico Universitario Agostino Gemelli, IRCCS, Roma, Italy
| | - Luca De Palma
- Rare and Complex Epilepsies, Department of Neuroscience and Neurorehabilitation, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
| | - Carlo E Marras
- Pediatric Neurosurgery Unit, Department of Neuroscience and Neurorehabilitation, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
| | - Alessandra Marasi
- Pediatric Neurosurgery Unit, Department of Neuroscience and Neurorehabilitation, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
| | - Antonio Napolitano
- Medical Physics Department, Bambino Gesù Children's Hospital, IRCCS, Roma, Italy
| |
Collapse
|
9
|
Minimally invasive and invasive liver surgery based on augmented reality training: a review of the literature. J Robot Surg 2022; 17:753-763. [DOI: 10.1007/s11701-022-01499-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Accepted: 11/14/2022] [Indexed: 11/29/2022]
|
10
|
Huber T, Huettl F, Hanke LI, Vradelis L, Heinrich S, Hansen C, Boedecker C, Lang H. Leberchirurgie 4.0 - OP-Planung, Volumetrie, Navigation und Virtuelle
Realität. Zentralbl Chir 2022; 147:361-368. [DOI: 10.1055/a-1844-0549] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
ZusammenfassungDurch die Optimierung der konservativen Behandlung, die Verbesserung der
bildgebenden Verfahren und die Weiterentwicklung der Operationstechniken haben
sich das operative Spektrum sowie der Maßstab für die Resektabilität in Bezug
auf die Leberchirurgie in den letzten Jahrzehnten deutlich verändert.Dank zahlreicher technischer Entwicklungen, insbesondere der 3-dimensionalen
Segmentierung, kann heutzutage die präoperative Planung und die Orientierung
während der Operation selbst, vor allem bei komplexen Eingriffen, unter
Berücksichtigung der patientenspezifischen Anatomie erleichtert werden.Neue Technologien wie 3-D-Druck, virtuelle und augmentierte Realität bieten
zusätzliche Darstellungsmöglichkeiten für die individuelle Anatomie.
Verschiedene intraoperative Navigationsmöglichkeiten sollen die präoperative
Planung im Operationssaal verfügbar machen, um so die Patientensicherheit zu
erhöhen.Dieser Übersichtsartikel soll einen Überblick über den gegenwärtigen Stand der
verfügbaren Technologien sowie einen Ausblick in den Operationssaal der Zukunft
geben.
Collapse
Affiliation(s)
- Tobias Huber
- Klinik für Allgemein-, Viszeral- und Transplantationschirurgie,
Universitätsmedizin Mainz, Mainz, Deutschland
| | - Florentine Huettl
- Klinik für Allgemein-, Viszeral- und Transplantationschirurgie,
Universitätsmedizin Mainz, Mainz, Deutschland
| | - Laura Isabel Hanke
- Klinik für Allgemein-, Viszeral- und Transplantationschirurgie,
Universitätsmedizin Mainz, Mainz, Deutschland
| | - Lukas Vradelis
- Klinik für Allgemein-, Viszeral- und Transplantationschirurgie,
Universitätsmedizin Mainz, Mainz, Deutschland
| | - Stefan Heinrich
- Klinik für Allgemein-, Viszeral- und Transplantationschirurgie,
Universitätsmedizin Mainz, Mainz, Deutschland
| | - Christian Hansen
- Fakultät für Informatik, Otto von Guericke Universität
Magdeburg, Magdeburg, Deutschland
| | - Christian Boedecker
- Klinik für Allgemein-, Viszeral- und Transplantationschirurgie,
Universitätsmedizin Mainz, Mainz, Deutschland
| | - Hauke Lang
- Klinik für Allgemein-, Viszeral- und Transplantationschirurgie,
Universitätsmedizin Mainz, Mainz, Deutschland
| |
Collapse
|
11
|
Doughty M, Ghugre NR, Wright GA. Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery. J Imaging 2022; 8:jimaging8070203. [PMID: 35877647 PMCID: PMC9318659 DOI: 10.3390/jimaging8070203] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2022] [Revised: 07/15/2022] [Accepted: 07/18/2022] [Indexed: 02/01/2023] Open
Abstract
We conducted a systematic review of recent literature to understand the current challenges in the use of optical see-through head-mounted displays (OST-HMDs) for augmented reality (AR) assisted surgery. Using Google Scholar, 57 relevant articles from 1 January 2021 through 18 March 2022 were identified. Selected articles were then categorized based on a taxonomy that described the required components of an effective AR-based navigation system: data, processing, overlay, view, and validation. Our findings indicated a focus on orthopedic (n=20) and maxillofacial surgeries (n=8). For preoperative input data, computed tomography (CT) (n=34), and surface rendered models (n=39) were most commonly used to represent image information. Virtual content was commonly directly superimposed with the target site (n=47); this was achieved by surface tracking of fiducials (n=30), external tracking (n=16), or manual placement (n=11). Microsoft HoloLens devices (n=24 in 2021, n=7 in 2022) were the most frequently used OST-HMDs; gestures and/or voice (n=32) served as the preferred interaction paradigm. Though promising system accuracy in the order of 2–5 mm has been demonstrated in phantom models, several human factors and technical challenges—perception, ease of use, context, interaction, and occlusion—remain to be addressed prior to widespread adoption of OST-HMD led surgical navigation.
Collapse
Affiliation(s)
- Mitchell Doughty
- Department of Medical Biophysics, University of Toronto, Toronto, ON M5S 1A1, Canada; (N.R.G.); (G.A.W.)
- Schulich Heart Program, Sunnybrook Health Sciences Centre, Toronto, ON M4N 3M5, Canada
- Correspondence:
| | - Nilesh R. Ghugre
- Department of Medical Biophysics, University of Toronto, Toronto, ON M5S 1A1, Canada; (N.R.G.); (G.A.W.)
- Schulich Heart Program, Sunnybrook Health Sciences Centre, Toronto, ON M4N 3M5, Canada
- Physical Sciences Platform, Sunnybrook Research Institute, Toronto, ON M4N 3M5, Canada
| | - Graham A. Wright
- Department of Medical Biophysics, University of Toronto, Toronto, ON M5S 1A1, Canada; (N.R.G.); (G.A.W.)
- Schulich Heart Program, Sunnybrook Health Sciences Centre, Toronto, ON M4N 3M5, Canada
- Physical Sciences Platform, Sunnybrook Research Institute, Toronto, ON M4N 3M5, Canada
| |
Collapse
|
12
|
McBain KA, Habib R, Laggis G, Quaiattini A, M Ventura N, Noel GPJC. Scoping review: The use of augmented reality in clinical anatomical education and its assessment tools. ANATOMICAL SCIENCES EDUCATION 2022; 15:765-796. [PMID: 34800073 DOI: 10.1002/ase.2155] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/26/2019] [Revised: 11/15/2021] [Accepted: 11/17/2021] [Indexed: 06/13/2023]
Abstract
The purpose of this review was to identify the different augmented reality (AR) modalities used to teach anatomy to students, health professional trainees, and surgeons, and to examine the assessment tools used to evaluate the performance of various AR modalities. A scoping review of four databases was performed using variations of: (1) AR, (2) medical or anatomical teaching/education/training, and (3) anatomy or radiology or cadaver. Scientific articles were identified and screened for the inclusion and exclusion criteria as per Preferred Reporting Items for Systematic Reviews and Meta-Analyses with extension for scoping reviews guidelines. Virtual reality was an exclusion criterion. From this scoping review, data were extracted from a total of 54 articles and the following four AR modalities were identified: head-mounted display, projection, instrument and screen, and mobile device. The usability, feasibility, and acceptability of these AR modalities were evaluated using a variety of quantitative and qualitative assessment tools. Within more recent years of AR integration into anatomy education, the assessment of visuospatial ability, cognitive load, time on task, and increasing academic achievement outcomes are variables of interest, which continue to warrant more exploration. Sufficiently powered studies using validated assessment tools must be conducted to better understand the role of AR in anatomical education.
Collapse
Affiliation(s)
- Kimberly A McBain
- School of Physical and Occupational Therapy, McGill University, Montreal, Quebec, Canada
| | - Rami Habib
- School of Medicine and Health Sciences, McGill University, Montreal, Quebec, Canada
| | - George Laggis
- School of Physical and Occupational Therapy, McGill University, Montreal, Quebec, Canada
| | - Andrea Quaiattini
- Schulich Library of Physical Sciences, Life Sciences, and Engineering, McGill University, Montreal, Quebec, Canada
- Institute of Health Sciences Education, Faculty of Medicine, McGill University, Montreal, Quebec, Canada
| | - Nicole M Ventura
- Institute of Health Sciences Education, Faculty of Medicine, McGill University, Montreal, Quebec, Canada
- Division of Anatomical Sciences, Department of Anatomy and Cell Biology, McGill University, Montreal, Quebec, Canada
| | - Geoffroy P J C Noel
- Institute of Health Sciences Education, Faculty of Medicine, McGill University, Montreal, Quebec, Canada
- Division of Anatomical Sciences, Department of Anatomy and Cell Biology, McGill University, Montreal, Quebec, Canada
- Division of Anatomy, Department of Surgery, University of California San Diego, San Diego, California, USA
| |
Collapse
|
13
|
Gavriilidis P, Edwin B, Pelanis E, Hidalgo E, de'Angelis N, Memeo R, Aldrighetti L, Sutcliffe RP. Navigated liver surgery: State of the art and future perspectives. Hepatobiliary Pancreat Dis Int 2022; 21:226-233. [PMID: 34544668 DOI: 10.1016/j.hbpd.2021.09.002] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/23/2021] [Accepted: 08/27/2021] [Indexed: 02/08/2023]
Abstract
BACKGROUND In recent years, the development of digital imaging technology has had a significant influence in liver surgery. The ability to obtain a 3-dimensional (3D) visualization of the liver anatomy has provided surgery with virtual reality of simulation 3D computer models, 3D printing models and more recently holograms and augmented reality (when virtual reality knowledge is superimposed onto reality). In addition, the utilization of real-time fluorescent imaging techniques based on indocyanine green (ICG) uptake allows clinicians to precisely delineate the liver anatomy and/or tumors within the parenchyma, applying the knowledge obtained preoperatively through digital imaging. The combination of both has transformed the abstract thinking until now based on 2D imaging into a 3D preoperative conception (virtual reality), enhanced with real-time visualization of the fluorescent liver structures, effectively facilitating intraoperative navigated liver surgery (augmented reality). DATA SOURCES A literature search was performed from inception until January 2021 in MEDLINE (PubMed), Embase, Cochrane library and database for systematic reviews (CDSR), Google Scholar, and National Institute for Health and Clinical Excellence (NICE) databases. RESULTS Fifty-one pertinent articles were retrieved and included. The different types of digital imaging technologies and the real-time navigated liver surgery were estimated and compared. CONCLUSIONS ICG fluorescent imaging techniques can contribute essentially to the real-time definition of liver segments; as a result, precise hepatic resection can be guided by the presence of fluorescence. Furthermore, 3D models can help essentially to further advancing of precision in hepatic surgery by permitting estimation of liver volume and functional liver remnant, delineation of resection lines along the liver segments and evaluation of tumor margins. In liver transplantation and especially in living donor liver transplantation (LDLT), 3D printed models of the donor's liver and models of the recipient's hilar anatomy can contribute further to improving the results. In particular, pediatric LDLT abdominal cavity models can help to manage the largest challenge of this procedure, namely large-for-size syndrome.
Collapse
Affiliation(s)
- Paschalis Gavriilidis
- Department of Hepato-Pancreato-Biliary and Liver Transplant Surgery, Queen Elizabeth University Hospitals Birmingham NHS Foundation Trust, B15 2TH, UK.
| | - Bjørn Edwin
- The Intervention Centre and Department of HPB Surgery, Oslo University Hospital and Faculty of Medicine, University of Oslo, Oslo, Norway
| | - Egidijus Pelanis
- The Intervention Centre and Department of HPB Surgery, Oslo University Hospital and Faculty of Medicine, University of Oslo, Oslo, Norway
| | - Ernest Hidalgo
- Department of Hepato-Pancreatico-Biliary Surgery and Transplantation, Hospital Universitari Vall d'Hebron, Barcelona, Spain
| | - Nicola de'Angelis
- Department of Digestive Surgery, University Hospital Henri Mondor (AP-HP), 94010 Créteil and University of Paris Est, Créteil, France
| | - Riccardo Memeo
- Department of Hepatobiliary and Pancreatic Surgery, Miulli Hospital, Acquaviva delle Fonti, Bari 70021, Italy
| | - Luca Aldrighetti
- Division of Hepatobiliary Surgery, San Raffaele Hospital, Via Olgettina 60, Milan 20132, Italy
| | - Robert P Sutcliffe
- Department of Hepato-Pancreato-Biliary and Liver Transplant Surgery, Queen Elizabeth University Hospitals Birmingham NHS Foundation Trust, B15 2TH, UK
| |
Collapse
|
14
|
A Fast Method for Whole Liver- and Colorectal Liver Metastasis Segmentations from MRI Using 3D FCNN Networks. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12105145] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Abstract
The liver is the most frequent organ for metastasis from colorectal cancer, one of the most common tumor types with a poor prognosis. Despite reducing surgical planning time and providing better spatial representation, current methods of 3D modeling of patient-specific liver anatomy are extremely time-consuming. The purpose of this study was to develop a deep learning model trained on an in-house dataset of 84 MRI volumes to rapidly provide fully automated whole liver and liver lesions segmentation from volumetric MRI series. A cascade approach was utilized to address the problem of class imbalance. The trained model achieved an average Dice score for whole liver segmentation of 0.944 ± 0.009 and 0.780 ± 0.119 for liver lesion segmentation. Furthermore, applying this method to a not-annotated dataset creates a complete 3D segmentation in less than 6 s per MRI volume, with a mean segmentation Dice score of 0.994 ± 0.003 for the liver and 0.709 ± 0.171 for tumors compared to manual corrections applied after the inference was achieved. Availability and integration of our method in clinical practice may improve diagnosis and treatment planning in patients with colorectal liver metastasis and open new possibilities for research into liver tumors.
Collapse
|
15
|
Sivananthan A, Gueroult A, Zijlstra G, Martin G, Baheerathan A, Pratt P, Darzi A, Patel N, Kinross J. Using Mixed Reality Headsets to Deliver Remote Bedside Teaching During the COVID-19 Pandemic: Feasibility Trial of HoloLens 2. JMIR Form Res 2022; 6:e35674. [PMID: 35389347 PMCID: PMC9116455 DOI: 10.2196/35674] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Revised: 03/07/2022] [Accepted: 03/25/2022] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND COVID-19 has had a catastrophic impact in terms of human lives lost. Medical education has also been impacted as appropriately stringent infection control policies precluded medical trainees from attending clinical teaching. Lecture-based education has been easily transferred to a digital platform, but bedside teaching has not. OBJECTIVE This study aims to assess the feasibility of using a mixed reality (MR) headset to deliver remote bedside teaching. METHODS Two MR sessions were led by senior doctors wearing the HoloLens headset. The trainers selected patients requiring their specialist input. The headset allowed bidirectional audiovisual communication between the trainer and trainee doctors. Trainee doctor conceptions of bedside teaching, impact of the COVID-19 pandemic on bedside teaching, and the MR sessions were evaluated using pre- and postround questionnaires, using Likert scales. Data related to clinician exposure to at-risk patients and use of personal protective equipment (PPE) were collected. RESULTS Prequestionnaire respondents (n=24) strongly agreed that bedside teaching is key to educating clinicians (median 7, IQR 6-7). Postsession questionnaires showed that, overall, users subjectively agreed the MR session was helpful to their learning (median 6, IQR 5.25-7) and that it was worthwhile (median 6, IQR 5.25-7). Mixed reality versus in-person teaching led to a 79.5% reduction in cumulative clinician exposure time and 83.3% reduction in PPE use. CONCLUSIONS This study is proof of principle that HoloLens can be used effectively to deliver clinical bedside teaching. This novel format confers significant advantages in terms of minimizing exposure of trainees to COVID-19, reducing PPE use, enabling larger attendance, and delivering convenient and accessible real-time clinical training.
Collapse
Affiliation(s)
- Arun Sivananthan
- Institute of Global Health Innovation, Imperial College London, London, United Kingdom
- Department of Gastroenterology and Hepatology, Imperial College Healthcare NHS Trust, London, United Kingdom
| | - Aurelien Gueroult
- Department of Gastroenterology and Hepatology, Imperial College Healthcare NHS Trust, London, United Kingdom
| | - Geiske Zijlstra
- Department of Academic Public Health, Imperial College London, London, United Kingdom
| | - Guy Martin
- Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | | | - Philip Pratt
- Medical iSight Corporation, London, United Kingdom
| | - Ara Darzi
- Institute of Global Health Innovation, Imperial College London, London, United Kingdom
| | - Nisha Patel
- Institute of Global Health Innovation, Imperial College London, London, United Kingdom
- Department of Gastroenterology and Hepatology, Imperial College Healthcare NHS Trust, London, United Kingdom
| | - James Kinross
- Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| |
Collapse
|
16
|
Birlo M, Edwards PJE, Clarkson M, Stoyanov D. Utility of optical see-through head mounted displays in augmented reality-assisted surgery: A systematic review. Med Image Anal 2022; 77:102361. [PMID: 35168103 PMCID: PMC10466024 DOI: 10.1016/j.media.2022.102361] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2021] [Revised: 11/17/2021] [Accepted: 01/10/2022] [Indexed: 12/11/2022]
Abstract
This article presents a systematic review of optical see-through head mounted display (OST-HMD) usage in augmented reality (AR) surgery applications from 2013 to 2020. Articles were categorised by: OST-HMD device, surgical speciality, surgical application context, visualisation content, experimental design and evaluation, accuracy and human factors of human-computer interaction. 91 articles fulfilled all inclusion criteria. Some clear trends emerge. The Microsoft HoloLens increasingly dominates the field, with orthopaedic surgery being the most popular application (28.6%). By far the most common surgical context is surgical guidance (n=58) and segmented preoperative models dominate visualisation (n=40). Experiments mainly involve phantoms (n=43) or system setup (n=21), with patient case studies ranking third (n=19), reflecting the comparative infancy of the field. Experiments cover issues from registration to perception with very different accuracy results. Human factors emerge as significant to OST-HMD utility. Some factors are addressed by the systems proposed, such as attention shift away from the surgical site and mental mapping of 2D images to 3D patient anatomy. Other persistent human factors remain or are caused by OST-HMD solutions, including ease of use, comfort and spatial perception issues. The significant upward trend in published articles is clear, but such devices are not yet established in the operating room and clinical studies showing benefit are lacking. A focused effort addressing technical registration and perceptual factors in the lab coupled with design that incorporates human factors considerations to solve clear clinical problems should ensure that the significant current research efforts will succeed.
Collapse
Affiliation(s)
- Manuel Birlo
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London (UCL), Charles Bell House, 43-45 Foley Street, London W1W 7TS, UK.
| | - P J Eddie Edwards
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London (UCL), Charles Bell House, 43-45 Foley Street, London W1W 7TS, UK
| | - Matthew Clarkson
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London (UCL), Charles Bell House, 43-45 Foley Street, London W1W 7TS, UK
| | - Danail Stoyanov
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London (UCL), Charles Bell House, 43-45 Foley Street, London W1W 7TS, UK
| |
Collapse
|
17
|
Saito Y, Shimada M, Morine Y, Yamada S, Sugimoto M. Essential updates 2020/2021: Current topics of simulation and navigation in hepatectomy. Ann Gastroenterol Surg 2022; 6:190-196. [PMID: 35261944 PMCID: PMC8889864 DOI: 10.1002/ags3.12542] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/19/2021] [Revised: 11/26/2021] [Accepted: 12/14/2021] [Indexed: 01/01/2023] Open
Abstract
With the development of three-dimensional (3D) simulation software, preoperative simulation technology is almost completely established. The remaining issue is how to recognize anatomy three-dimensionally. Extended reality is a newly developed technology with several merits for surgical application: no requirement for a sterilized display monitor, better spatial awareness, and the ability to share 3D images among all surgeons. Various technology or devices for intraoperative navigation have also been developed to support the safety and certainty of liver surgery. Consensus recommendations regarding indocyanine green fluorescence were determined in 2021. Extended reality has also been applied to intraoperative navigation, and artificial intelligence (AI) is one of the topics of real-time navigation. AI might overcome the problem of liver deformity with automatic registration. Including the issues described above, this article focuses on recent advances in simulation and navigation in liver surgery from 2020 to 2021.
Collapse
Affiliation(s)
- Yu Saito
- Department of SurgeryTokushima UniversityTokushimaJapan
| | | | - Yuji Morine
- Department of SurgeryTokushima UniversityTokushimaJapan
| | | | - Maki Sugimoto
- Department of SurgeryTokushima UniversityTokushimaJapan
- Okinaga Research InstituteTeikyo UniversityChiyoda‐kuJapan
| |
Collapse
|
18
|
Zhu LY, Hou JC, Yang L, Liu ZR, Tong W, Bai Y, Zhang YM. Application value of mixed reality in hepatectomy for hepatocellular carcinoma. World J Gastrointest Surg 2022; 14:36-45. [PMID: 35126861 PMCID: PMC8790326 DOI: 10.4240/wjgs.v14.i1.36] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/04/2021] [Revised: 11/29/2021] [Accepted: 12/25/2021] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND As a new digital holographic imaging technology, mixed reality (MR) technology has unique advantages in determining the liver anatomy and location of tumor lesions. With the popularization of 5G communication technology, MR shows great potential in preoperative planning and intraoperative navigation, making hepatectomy more accurate and safer.
AIM To evaluate the application value of MR technology in hepatectomy for hepatocellular carcinoma (HCC).
METHODS The clinical data of 95 patients who underwent open hepatectomy surgery for HCC between June 2018 and October 2020 at our hospital were analyzed retrospectively. We selected 95 patients with HCC according to the inclusion criteria and exclusion criteria. In 38 patients, hepatectomy was assisted by MR (Group A), and an additional 57 patients underwent traditional hepatectomy without MR (Group B). The perioperative outcomes of the two groups were collected and compared to evaluate the application value of MR in hepatectomy for patients with HCC.
RESULTS We summarized the technical process of MR-assisted hepatectomy in the treatment of HCC. Compared to traditional hepatectomy in Group B, MR-assisted hepatectomy in Group A yielded a shorter operation time (202.86 ± 46.02 min vs 229.52 ± 57.13 min, P = 0.003), less volume of bleeding (329.29 ± 97.31 mL vs 398.23 ± 159.61 mL, P = 0.028), and shorter obstructive time of the portal vein (17.71 ± 4.16 min vs 21.58 ± 5.24 min, P = 0.019). Group A had lower alanine aminotransferas and higher albumin values on the third day after the operation (119.74 ± 29.08 U/L vs 135.53 ± 36.68 U/L, P = 0.029 and 33.60 ± 3.21 g/L vs 31.80 ± 3.51 g/L, P = 0.014, respectively). The total postoperative complications and hospitalization days in Group A were significantly less than those in Group B [14 (37.84%) vs 35 (60.34%), P = 0.032 and 12.05 ± 4.04 d vs 13.78 ± 4.13 d, P = 0.049, respectively].
CONCLUSION MR has some application value in three-dimensional visualization of the liver, surgical planning, and intraoperative navigation during hepatectomy, and it significantly improves the perioperative outcomes of hepatectomy for HCC.
Collapse
Affiliation(s)
- Liu-Yang Zhu
- First Central Clinical College, Tianjin Medical University, Tianjin 300070, China
| | - Jian-Cun Hou
- Department of Hepatobiliary Surgery, Tianjin First Central Hospital, Tianjin 300192, China
| | - Long Yang
- Department of Hepatobiliary Surgery, Tianjin First Central Hospital, Tianjin 300192, China
| | - Zi-Rong Liu
- Department of Hepatobiliary Surgery, Tianjin First Central Hospital, Tianjin 300192, China
| | - Wen Tong
- First Central Clinical College, Tianjin Medical University, Tianjin 300070, China
| | - Yi Bai
- Department of Hepatobiliary Surgery, Tianjin First Central Hospital, Tianjin 300192, China
| | - Ya-Min Zhang
- Department of Hepatobiliary Surgery, Tianjin First Central Hospital, Tianjin 300192, China
| |
Collapse
|
19
|
The Applications of 3D Imaging and Indocyanine Green Dye Fluorescence in Laparoscopic Liver Surgery. Diagnostics (Basel) 2021; 11:diagnostics11122169. [PMID: 34943406 PMCID: PMC8700092 DOI: 10.3390/diagnostics11122169] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2021] [Revised: 11/19/2021] [Accepted: 11/21/2021] [Indexed: 02/06/2023] Open
Abstract
Laparoscopic liver resections have gained widespread popularity among hepatobiliary surgeons and is nowadays performed for both standard and more complex hepatectomies. Given the increased technical challenges, preoperative planning and intraoperative guidance is pivotal in laparoscopic surgery to safely carry out complex and oncologically safe hepatectomies. Modern tools can help both preoperatively and intraoperatively and allow surgeons to perform more precise hepatectomies. Preoperative 3D reconstructions and printing as well as augmented reality can increase the knowledge of the specific anatomy of the case and therefore plan the surgery accordingly and tailor the procedure on the patient. Furthermore, the indocyanine green retention dye is an increasingly used tool that can nowadays improve the precision during laparoscopic hepatectomies, especially when considering anatomical resection. The use of preoperative modern imaging and intraoperative indocyanine green dye are key to successfully perform complex hepatectomies such as laparoscopic parenchymal sparing liver resections. In this narrative review, we discuss the aspects of preoperative and intraoperative tools that are nowadays increasingly used in experienced hepatobiliary centers.
Collapse
|
20
|
Sarmadi H, Muñoz-Salinas R, Álvaro Berbís M, Luna A, Medina-Carnicer R. Joint scene and object tracking for cost-Effective augmented reality guided patient positioning in radiation therapy. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 209:106296. [PMID: 34380076 DOI: 10.1016/j.cmpb.2021.106296] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2020] [Accepted: 07/17/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND AND OBJECTIVE The research is done in the field of Augmented Reality (AR) for patient positioning in radiation therapy is scarce. We propose an efficient and cost-effective algorithm for tracking the scene and the patient to interactively assist the patient's positioning process by providing visual feedback to the operator. Up to our knowledge, this is the first framework that can be employed for mobile interactive AR to guide patient positioning. METHODS We propose a pointcloud processing method that, combined with a fiducial marker-mapper algorithm and the generalized ICP algorithm, tracks the patient and the camera precisely and efficiently only using the CPU unit. The 3D reference model and body marker map alignment is calculated employing an efficient body reconstruction algorithm. RESULTS Our quantitative evaluation shows that the proposed method achieves a translational and rotational error of 4.17 mm/0.82∘ at 9 fps. Furthermore, the qualitative results demonstrate the usefulness of our algorithm in patient positioning on different human subjects. CONCLUSION Since our algorithm achieves a relatively high frame rate and accuracy employing a regular laptop (without a dedicated GPU), it is a very cost-effective AR-based patient positioning method. It also opens the way for other researchers by introducing a framework that could be improved upon for better mobile interactive AR patient positioning solutions in the future.
Collapse
Affiliation(s)
- Hamid Sarmadi
- Instituto Maimónides de Investigación en Biomedicina (IMIBIC). Avenida Menéndez Pidal s/n, Córdoba, 14004, Spain.
| | - Rafael Muñoz-Salinas
- Computing and Numerical Analysis Department, Edificio Einstein. Campus de Rabanales, Córdoba University, Córdoba, 14071, Spain; Instituto Maimónides de Investigación en Biomedicina (IMIBIC). Avenida Menéndez Pidal s/n, Córdoba, 14004, Spain.
| | - M Álvaro Berbís
- HT Médica, Hospital San Juan de Dios. Avda Brillante 106, Córdoba, 14012, Spain.
| | - Antonio Luna
- HT Médica, Clínica las Nieves, Carmelo Torres 2, Jaén, 23007, Spain.
| | - R Medina-Carnicer
- Computing and Numerical Analysis Department, Edificio Einstein. Campus de Rabanales, Córdoba University, Córdoba, 14071, Spain; Instituto Maimónides de Investigación en Biomedicina (IMIBIC). Avenida Menéndez Pidal s/n, Córdoba, 14004, Spain.
| |
Collapse
|
21
|
Wahba R, Thomas MN, Bunck AC, Bruns CJ, Stippel DL. Clinical use of augmented reality, mixed reality, three-dimensional-navigation and artificial intelligence in liver surgery. Artif Intell Gastroenterol 2021; 2:94-104. [DOI: 10.35712/aig.v2.i4.94] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/09/2021] [Revised: 07/10/2021] [Accepted: 08/27/2021] [Indexed: 02/06/2023] Open
Abstract
A precise knowledge of intra-parenchymal vascular and biliary architecture and the location of lesions in relation to the complex anatomy is indispensable to perform liver surgery. Therefore, virtual three-dimensional (3D)-reconstruction models from computed tomography/magnetic resonance imaging scans of the liver might be helpful for visualization. Augmented reality, mixed reality and 3D-navigation could transfer such 3D-image data directly into the operation theater to support the surgeon. This review examines the literature about the clinical and intraoperative use of these image guidance techniques in liver surgery and provides the reader with the opportunity to learn about these techniques. Augmented reality and mixed reality have been shown to be feasible for the use in open and minimally invasive liver surgery. 3D-navigation facilitated targeting of intraparenchymal lesions. The existing data is limited to small cohorts and description about technical details e.g., accordance between the virtual 3D-model and the real liver anatomy. Randomized controlled trials regarding clinical data or oncological outcome are not available. Up to now there is no intraoperative application of artificial intelligence in liver surgery. The usability of all these sophisticated image guidance tools has still not reached the grade of immersion which would be necessary for a widespread use in the daily surgical routine. Although there are many challenges, augmented reality, mixed reality, 3D-navigation and artificial intelligence are emerging fields in hepato-biliary surgery.
Collapse
Affiliation(s)
- Roger Wahba
- Department of General, Visceral, Cancer and Transplantation Surgery, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne 50937, Germany
| | - Michael N Thomas
- Department of General, Visceral, Cancer and Transplantation Surgery, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne 50937, Germany
| | - Alexander C Bunck
- Department of Diagnostic and Interventional Radiology, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne 50937, Germany
| | - Christiane J Bruns
- Department of General, Visceral, Cancer and Transplantation Surgery, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne 50937, Germany
| | - Dirk L Stippel
- Department of General, Visceral, Cancer and Transplantation Surgery, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne 50937, Germany
| |
Collapse
|
22
|
Hattab G, Hatzipanayioti A, Klimova A, Pfeiffer M, Klausing P, Breucha M, Bechtolsheim FV, Helmert JR, Weitz J, Pannasch S, Speidel S. Investigating the utility of VR for spatial understanding in surgical planning: evaluation of head-mounted to desktop display. Sci Rep 2021; 11:13440. [PMID: 34188080 PMCID: PMC8241863 DOI: 10.1038/s41598-021-92536-x] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Accepted: 06/11/2021] [Indexed: 02/06/2023] Open
Abstract
Recent technological advances have made Virtual Reality (VR) attractive in both research and real world applications such as training, rehabilitation, and gaming. Although these other fields benefited from VR technology, it remains unclear whether VR contributes to better spatial understanding and training in the context of surgical planning. In this study, we evaluated the use of VR by comparing the recall of spatial information in two learning conditions: a head-mounted display (HMD) and a desktop screen (DT). Specifically, we explored (a) a scene understanding and then (b) a direction estimation task using two 3D models (i.e., a liver and a pyramid). In the scene understanding task, participants had to navigate the rendered the 3D models by means of rotation, zoom and transparency in order to substantially identify the spatial relationships among its internal objects. In the subsequent direction estimation task, participants had to point at a previously identified target object, i.e., internal sphere, on a materialized 3D-printed version of the model using a tracked pointing tool. Results showed that the learning condition (HMD or DT) did not influence participants' memory and confidence ratings of the models. In contrast, the model type, that is, whether the model to be recalled was a liver or a pyramid significantly affected participants' memory about the internal structure of the model. Furthermore, localizing the internal position of the target sphere was also unaffected by participants' previous experience of the model via HMD or DT. Overall, results provide novel insights on the use of VR in a surgical planning scenario and have paramount implications in medical learning by shedding light on the mental model we make to recall spatial structures.
Collapse
Affiliation(s)
- Georges Hattab
- grid.461742.2Division of Translational Surgical Oncology (TSO), National Center for Tumor Diseases (NCT), Partner Site Dresden, 01307 Dresden, Germany
| | - Adamantini Hatzipanayioti
- grid.4488.00000 0001 2111 7257Centre for Tactile Internet with Human-in-the-Loop (CeTI), TU Dresden, 01062 Dresden, Germany ,grid.4488.00000 0001 2111 7257Unit of Lifespan Developmental Neuroscience, Faculty of Psychology, TU Dresden, 01062 Dresden, Germany
| | - Anna Klimova
- grid.4488.00000 0001 2111 7257Institute for Medical Informatics and Biometry (IMB), Faculty of Medicine, TU Dresden, 01307 Dresden, Germany ,grid.461742.2Core Unit for Data Management and Analytics, National Center for Tumor Diseases (NCT), Partner Site Dresden, 01307 Dresden, Germany
| | - Micha Pfeiffer
- grid.461742.2Division of Translational Surgical Oncology (TSO), National Center for Tumor Diseases (NCT), Partner Site Dresden, 01307 Dresden, Germany
| | - Peter Klausing
- grid.461742.2Division of Translational Surgical Oncology (TSO), National Center for Tumor Diseases (NCT), Partner Site Dresden, 01307 Dresden, Germany
| | - Michael Breucha
- grid.461742.2Division of Translational Surgical Oncology (TSO), National Center for Tumor Diseases (NCT), Partner Site Dresden, 01307 Dresden, Germany ,grid.412282.f0000 0001 1091 2917Department of Visceral, Thoracic and Vascular Surgery, Faculty of Medicine, University Hospital Carl Gustav Carus, Dresden, Germany
| | - Felix von Bechtolsheim
- grid.4488.00000 0001 2111 7257Centre for Tactile Internet with Human-in-the-Loop (CeTI), TU Dresden, 01062 Dresden, Germany ,grid.412282.f0000 0001 1091 2917Department of Visceral, Thoracic and Vascular Surgery, Faculty of Medicine, University Hospital Carl Gustav Carus, Dresden, Germany
| | - Jens R. Helmert
- grid.4488.00000 0001 2111 7257Centre for Tactile Internet with Human-in-the-Loop (CeTI), TU Dresden, 01062 Dresden, Germany ,grid.4488.00000 0001 2111 7257Engineering Psychology and Applied Cognitive Research, Faculty of Psychology, TU Dresden, 01062 Dresden, Germany
| | - Jürgen Weitz
- grid.4488.00000 0001 2111 7257Centre for Tactile Internet with Human-in-the-Loop (CeTI), TU Dresden, 01062 Dresden, Germany ,grid.412282.f0000 0001 1091 2917Department of Visceral, Thoracic and Vascular Surgery, Faculty of Medicine, University Hospital Carl Gustav Carus, Dresden, Germany
| | - Sebastian Pannasch
- grid.4488.00000 0001 2111 7257Centre for Tactile Internet with Human-in-the-Loop (CeTI), TU Dresden, 01062 Dresden, Germany ,grid.4488.00000 0001 2111 7257Engineering Psychology and Applied Cognitive Research, Faculty of Psychology, TU Dresden, 01062 Dresden, Germany
| | - Stefanie Speidel
- grid.461742.2Division of Translational Surgical Oncology (TSO), National Center for Tumor Diseases (NCT), Partner Site Dresden, 01307 Dresden, Germany ,grid.4488.00000 0001 2111 7257Centre for Tactile Internet with Human-in-the-Loop (CeTI), TU Dresden, 01062 Dresden, Germany
| |
Collapse
|
23
|
Iizuka K, Sato Y, Imaizumi Y, Mizutani T. Potential Efficacy of Multimodal Mixed Reality in Epilepsy Surgery. Oper Neurosurg (Hagerstown) 2021; 20:276-281. [PMID: 33382064 DOI: 10.1093/ons/opaa341] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2020] [Accepted: 08/25/2020] [Indexed: 11/12/2022] Open
Abstract
BACKGROUND Mixed reality (MR) technology, which can fuse things in real and virtual space in real time, has been used mainly for simulation in neurosurgical training. OBJECTIVE To develop MR technology into multimodal MR for intraoperative guidance during epilepsy surgery. METHODS A 33-yr-old male patient suffered from intractable general tonic seizures due to left temporal meningoencephalocele. Preoperative scalp electroencephalograms localized interictal epileptic activity on the left temporal lobe. Iomazenil single photon emission tomography revealed temporal lobe lateralization. Magnetic resonance imaging (MRI) demonstrated left basal temporal meningoencephalocele extending into the pterygopalatine fossa through a bone defect at the base of the greater sphenoid wing. A 3-dimensional model was created for MR based on multimodal data including computed tomography, MRI tractography, and digital subtraction angiography, which enabled 3-dimensional visualization of abnormal subcortical fiber connections between the meningoencephalocele and the epileptic focus. RESULTS By using intraoperative multimodal MR, we were able to safely remove the meningoencephalocele and perform epileptic focus resection. The patient was seizure-free postoperatively, and no adverse effects were noted. CONCLUSION Intraoperative multimodal MR was a feasible and effective technique, and it can be applied for a wide range of epilepsy surgeries.
Collapse
Affiliation(s)
- Kazuki Iizuka
- Department of Neurosurgery, AOI Universal Hospital, Kawasaki, Kanagawa, Japan.,Department of Neurosurgery, Showa University of Medicine, Shinagawa, Tokyo, Japan
| | - Yosuke Sato
- Department of Neurosurgery, Showa University of Medicine, Shinagawa, Tokyo, Japan
| | - Yohichi Imaizumi
- Department of Neurosurgery, AOI Universal Hospital, Kawasaki, Kanagawa, Japan.,Department of Neurosurgery, Showa University of Medicine, Shinagawa, Tokyo, Japan
| | - Tohru Mizutani
- Department of Neurosurgery, Showa University of Medicine, Shinagawa, Tokyo, Japan
| |
Collapse
|
24
|
Teatini A, Kumar RP, Elle OJ, Wiig O. Mixed reality as a novel tool for diagnostic and surgical navigation in orthopaedics. Int J Comput Assist Radiol Surg 2021; 16:407-414. [PMID: 33555563 PMCID: PMC7946663 DOI: 10.1007/s11548-020-02302-z] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2020] [Accepted: 12/14/2020] [Indexed: 12/15/2022]
Abstract
Purpose This study presents a novel surgical navigation tool developed in mixed reality environment for orthopaedic surgery. Joint and skeletal deformities affect all age groups and greatly reduce the range of motion of the joints. These deformities are notoriously difficult to diagnose and to correct through surgery. Method We have developed a surgical tool which integrates surgical instrument tracking and augmented reality through a head mounted display. This allows the surgeon to visualise bones with the illusion of possessing “X-ray” vision. The studies presented below aim to assess the accuracy of the surgical navigation tool in tracking a location at the tip of the surgical instrument in holographic space. Results Results show that the average accuracy provided by the navigation tool is around 8 mm, and qualitative assessment by the orthopaedic surgeons provided positive feedback in terms of the capabilities for diagnostic use. Conclusions More improvements are necessary for the navigation tool to be accurate enough for surgical applications, however, this new tool has the potential to improve diagnostic accuracy and allow for safer and more precise surgeries, as well as provide for better learning conditions for orthopaedic surgeons in training.
Collapse
Affiliation(s)
- Andrea Teatini
- The Intervention Centre, Oslo University Hospital, Oslo, Norway.
- Department of Informatics, University of Oslo, Oslo, Norway.
| | - Rahul P Kumar
- The Intervention Centre, Oslo University Hospital, Oslo, Norway
| | - Ole Jakob Elle
- The Intervention Centre, Oslo University Hospital, Oslo, Norway
- Department of Informatics, University of Oslo, Oslo, Norway
| | - Ola Wiig
- Department of Orthopaedic Surgery, Oslo University Hospital, Oslo, Norway
| |
Collapse
|
25
|
Guo ZY, Ding ZF, Miao C, Li CJ, Tang XF, Zhang Z. [Application of mixed reality in oromaxillofacial head and neck oncology surgery: a preliminary study]. HUA XI KOU QIANG YI XUE ZA ZHI = HUAXI KOUQIANG YIXUE ZAZHI = WEST CHINA JOURNAL OF STOMATOLOGY 2020; 38:470-474. [PMID: 32865371 DOI: 10.7518/hxkq.2020.04.021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
Mixed reality (MR), characterized by the ability to integrate digital data into human real feeling, is a new technique in medical imaging and surgical navigation. MR has tremendous value in surgery, but its application in oromaxillofacial head and neck oncology surgery is not yet reported. This paper reports the application of MR in oromaxillofacial head and neck oncology surgery. The merits, demerits, and present research situations and prospects of MR are further discussed.
Collapse
Affiliation(s)
- Zhi-Yong Guo
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases & Dept. of Head and Neck Oncology, West China Hospital of Stomatology, Sichuan University, Chengdu 610041, China
| | - Zhang-Fan Ding
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases & Dept. of Head and Neck Oncology, West China Hospital of Stomatology, Sichuan University, Chengdu 610041, China
| | - Cheng Miao
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases & Dept. of Head and Neck Oncology, West China Hospital of Stomatology, Sichuan University, Chengdu 610041, China
| | - Chun-Jie Li
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases & Dept. of Head and Neck Oncology, West China Hospital of Stomatology, Sichuan University, Chengdu 610041, China
| | - Xiu-Fa Tang
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases & Dept. of Head and Neck Oncology, West China Hospital of Stomatology, Sichuan University, Chengdu 610041, China
| | - Zhuang Zhang
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases & Dept. of Head and Neck Oncology, West China Hospital of Stomatology, Sichuan University, Chengdu 610041, China
| |
Collapse
|
26
|
Ultrasound in augmented reality: a mixed-methods evaluation of head-mounted displays in image-guided interventions. Int J Comput Assist Radiol Surg 2020; 15:1895-1905. [PMID: 32725398 PMCID: PMC8332636 DOI: 10.1007/s11548-020-02236-6] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2020] [Accepted: 07/14/2020] [Indexed: 11/24/2022]
Abstract
Purpose Augmented reality (AR) and head-mounted displays (HMD) in medical practice are current research topics. A commonly proposed use case of AR-HMDs is to display data in image-guided interventions. Although technical feasibility has been thoroughly shown, effects of AR-HMDs on interventions are not yet well researched, hampering clinical applicability. Therefore, the goal of this study is to better understand the benefits and limitations of this technology in ultrasound-guided interventions. Methods We used an AR-HMD system (based on the first-generation Microsoft Hololens) which overlays live ultrasound images spatially correctly at the location of the ultrasound transducer. We chose ultrasound-guided needle placements as a representative task for image-guided interventions. To examine the effects of the AR-HMD, we used mixed methods and conducted two studies in a lab setting: (1) In a randomized crossover study, we asked participants to place needles into a training model and evaluated task duration and accuracy with the AR-HMD as compared to the standard procedure without visual overlay and (2) in a qualitative study, we analyzed the user experience with AR-HMD using think-aloud protocols during ultrasound examinations and semi-structured interviews after the task. Results Participants (n = 20) placed needles more accurately (mean error of 7.4 mm vs. 4.9 mm, p = 0.022) but not significantly faster (mean task duration of 74.4 s vs. 66.4 s, p = 0.211) with the AR-HMD. All participants in the qualitative study (n = 6) reported limitations of and unfamiliarity with the AR-HMD, yet all but one also clearly noted benefits and/or that they would like to test the technology in practice. Conclusion We present additional, though still preliminary, evidence that AR-HMDs provide benefits in image-guided procedures. Our data also contribute insights into potential causes underlying the benefits, such as improved spatial perception. Still, more comprehensive studies are needed to ascertain benefits for clinical applications and to clarify mechanisms underlying these benefits. Electronic supplementary material The online version of this article (10.1007/s11548-020-02236-6) contains supplementary material, which is available to authorized users.
Collapse
|
27
|
Teatini A, Pelanis E, Aghayan DL, Kumar RP, Palomar R, Fretland ÅA, Edwin B, Elle OJ. The effect of intraoperative imaging on surgical navigation for laparoscopic liver resection surgery. Sci Rep 2019; 9:18687. [PMID: 31822701 PMCID: PMC6904553 DOI: 10.1038/s41598-019-54915-3] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2019] [Accepted: 11/21/2019] [Indexed: 12/14/2022] Open
Abstract
Conventional surgical navigation systems rely on preoperative imaging to provide guidance. In laparoscopic liver surgery, insufflation of the abdomen (pneumoperitoneum) can cause deformations on the liver, introducing inaccuracies in the correspondence between the preoperative images and the intraoperative reality. This study evaluates the improvements provided by intraoperative imaging for laparoscopic liver surgical navigation, when displayed as augmented reality (AR). Significant differences were found in terms of accuracy of the AR, in favor of intraoperative imaging. In addition, results showed an effect of user-induced error: image-to-patient registration based on annotations performed by clinicians caused 33% more inaccuracy as compared to image-to-patient registration algorithms that do not depend on user annotations. Hence, to achieve accurate surgical navigation for laparoscopic liver surgery, intraoperative imaging is recommendable to compensate for deformation. Moreover, user annotation errors may lead to inaccuracies in registration processes.
Collapse
Affiliation(s)
- Andrea Teatini
- The Intervention Centre, Oslo University Hospital, Oslo, Norway.
- Department of Informatics, University of Oslo, Oslo, Norway.
| | - Egidijus Pelanis
- The Intervention Centre, Oslo University Hospital, Oslo, Norway
- Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Davit L Aghayan
- The Intervention Centre, Oslo University Hospital, Oslo, Norway
- Institute of Clinical Medicine, University of Oslo, Oslo, Norway
- Department of Surgery N1, Yerevan State Medical University, Yerevan, Armenia
| | | | - Rafael Palomar
- The Intervention Centre, Oslo University Hospital, Oslo, Norway
- Department of Computer Science, NTNU, Gjøvik, Norway
| | - Åsmund Avdem Fretland
- The Intervention Centre, Oslo University Hospital, Oslo, Norway
- Institute of Clinical Medicine, University of Oslo, Oslo, Norway
- Department of Hepato-Pancreatic-Biliary surgery, Oslo University Hospital, Oslo, Norway
| | - Bjørn Edwin
- The Intervention Centre, Oslo University Hospital, Oslo, Norway
- Institute of Clinical Medicine, University of Oslo, Oslo, Norway
- Department of Hepato-Pancreatic-Biliary surgery, Oslo University Hospital, Oslo, Norway
| | - Ole Jakob Elle
- The Intervention Centre, Oslo University Hospital, Oslo, Norway
- Department of Informatics, University of Oslo, Oslo, Norway
| |
Collapse
|