Intraoperative support with three-dimensional holographic cholangiography in hepatobiliary surgery

Mixed Reality in the Operating Room: A Systematic Review

Mixed Reality is a technology that has gained attention due to its unique capabilities for accessing and visualizing information. When integrated with voice control mechanisms, gestures and even iris movement, it becomes a valuable tool for medicine. These features are particularly appealing for the operating room and surgical learning, where access to information and freedom of hand operation are fundamental. This study examines the most significant research on mixed reality in the operating room over the past five years, to identify the trends, use cases, its applications and limitations. A systematic review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines to answer the research questions established using the PICO (Population, Intervention, Comparator and Outcome) framework. Although implementation of Mixed Reality applications in the operations room presents some challenges, when used appropriately, it can yield remarkable results. It can make learning easier, flatten the learning curve for several procedures, and facilitate various aspects of the surgical processes. The articles' conclusions highlight the potential benefits of these innovations in surgical practice while acknowledging the challenges that must be addressed. Technical complexity, equipment costs, and steep learning curves present significant obstacles to the widespread adoption of Mixed Reality and computer-assisted evaluation. The need for more flexible approaches and comprehensive studies is underscored by the specificity of procedures and limited samples sizes. The integration of imaging modalities and innovative functionalities holds promise for clinical applications. However, it is important to consider issues related to usability, bias, and statistical analyses. Mixed Reality offers significant benefits, but there are still open challenges such as ergonomic issues, limited field of view, and battery autonomy that must be addressed to ensure widespread acceptance.

Hepatobiliary surgery based on intelligent image segmentation technology

Liver disease is an important disease that seriously threatens human health. It accounts for the highest proportion in various malignant tumors, and its incidence rate and mortality are on the rise, seriously affecting human health. Modern imaging has developed rapidly, but the application of image segmentation in liver tumor surgery is still rare. The application of image processing technology represented by artificial intelligence (AI) in surgery can greatly improve the efficiency of surgery, reduce surgical complications, and reduce the cost of surgery. Hepatocellular carcinoma is the most common malignant tumor in the world, and its mortality is second only to lung cancer. The resection rate of liver cancer surgery is high, and it is a multidisciplinary surgery, so it is necessary to explore the possibility of effective switching between different disciplines. Resection of hepatobiliary and pancreatic tumors is one of the most challenging and lethal surgical procedures. The operation requires a high level of doctors' experience and understanding of anatomical structures. The surgical segmentation is slow and there may be obvious complications. Therefore, the surgical system needs to make full use of the relevant functions of AI technology and computer vision analysis software, and combine the processing strategy based on image processing algorithm and computer vision analysis model. Intelligent optimization algorithm, also known as modern heuristic algorithm, is an algorithm with global optimization performance, strong universality, and suitable for parallel processing. This algorithm generally has a strict theoretical basis, rather than relying solely on expert experience. In theory, the optimal solution or approximate optimal solution can be found in a certain time. This work studies the hepatobiliary surgery through intelligent image segmentation technology, and analyzes them through intelligent optimization algorithm. The research results showed that when other conditions were the same, there were three patients who had adverse reactions in hepatobiliary surgery through intelligent image segmentation technology, accounting for 10%. The number of patients with adverse reactions in hepatobiliary surgery by conventional methods was nine, accounting for 30%, which was significantly higher than the former, indicating a positive relationship between intelligent image segmentation technology and hepatobiliary surgery.

The HoloLens in medicine: A systematic review and taxonomy

The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.

Towards Wearable Augmented Reality in Healthcare: A Comparative Survey and Analysis of Head-Mounted Displays

Head-mounted displays (HMDs) have the potential to greatly impact the surgical field by maintaining sterile conditions in healthcare environments. Google Glass (GG) and Microsoft HoloLens (MH) are examples of optical HMDs. In this comparative survey related to wearable augmented reality (AR) technology in the medical field, we examine the current developments in wearable AR technology, as well as the medical aspects, with a specific emphasis on smart glasses and HoloLens. The authors searched recent articles (between 2017 and 2022) in the PubMed, Web of Science, Scopus, and ScienceDirect databases and a total of 37 relevant studies were considered for this analysis. The selected studies were divided into two main groups; 15 of the studies (around 41%) focused on smart glasses (e.g., Google Glass) and 22 (59%) focused on Microsoft HoloLens. Google Glass was used in various surgical specialities and preoperative settings, namely dermatology visits and nursing skill training. Moreover, Microsoft HoloLens was used in telepresence applications and holographic navigation of shoulder and gait impairment rehabilitation, among others. However, some limitations were associated with their use, such as low battery life, limited memory size, and possible ocular pain. Promising results were obtained by different studies regarding the feasibility, usability, and acceptability of using both Google Glass and Microsoft HoloLens in patient-centric settings as well as medical education and training. Further work and development of rigorous research designs are required to evaluate the efficacy and cost-effectiveness of wearable AR devices in the future.

Essential updates 2020/2021: Current topics of simulation and navigation in hepatectomy

With the development of three-dimensional (3D) simulation software, preoperative simulation technology is almost completely established. The remaining issue is how to recognize anatomy three-dimensionally. Extended reality is a newly developed technology with several merits for surgical application: no requirement for a sterilized display monitor, better spatial awareness, and the ability to share 3D images among all surgeons. Various technology or devices for intraoperative navigation have also been developed to support the safety and certainty of liver surgery. Consensus recommendations regarding indocyanine green fluorescence were determined in 2021. Extended reality has also been applied to intraoperative navigation, and artificial intelligence (AI) is one of the topics of real-time navigation. AI might overcome the problem of liver deformity with automatic registration. Including the issues described above, this article focuses on recent advances in simulation and navigation in liver surgery from 2020 to 2021.