First-in-human real-time AI-assisted instrument deocclusion during augmented reality robotic surgery

Indocyanine Blue (ICB) as a Functional Alternative to Indocyanine Green (ICG) for Enhanced 700 nm NIR Imaging

Despite significant advancements in bioimaging technology, only a limited number of fluorophores are currently approved for clinical applications. Indocyanine green (ICG) is the first FDA-approved near-infrared (NIR) fluorophore and has significantly advanced clinical interventions over the past three decades. However, its single-channel imaging at 800 nm emission is often insufficient for capturing comprehensive diagnostic information during surgery. In this study, we evaluate indocyanine blue (ICB), an ICG analog with a shorter polymethine bridge, as a promising candidate for multi-channel NIR imaging. ICB demonstrated peak absorption and emission approximately 100 nm shorter than ICG in aqueous solutions, placing it within the 700 nm range of the NIR window. Furthermore, ICB exhibited favorable solubility and optical properties in aqueous environments, supporting its potential for in vivo imaging applications. Notably, ICB shows rapid systemic clearance, likely due to its lower molecular weight, which facilitates clear visualization in angiography, cholangiography, and lymph node mapping with minimal background interference. Additionally, dual-channel imaging of tumors and lymph nodes was achieved using a tumor-targeting fluorophore in conjunction with ICB, illustrating the potential for enhanced intraoperative guidance. ICB emitting at 700 nm, therefore, can be useful in NIR imaging, broadening the possibilities for improved diagnostic accuracy and therapeutic outcomes in clinical settings.

Metaverse in surgery - origins and future potential

The metaverse refers to a collective virtual space that combines physical and digital realities to create immersive, interactive environments. This space is powered by technologies such as augmented reality (AR), virtual reality (VR), artificial intelligence (AI) and blockchain. In healthcare, the metaverse can offer many applications. Specifically in surgery, potential uses of the metaverse include the possibility of conducting immersive surgical training in a VR or AR setting, and enhancing surgical planning through the adoption of three-dimensional virtual models and simulated procedures. At the intraoperative level, AR-guided surgery can assist the surgeon in real time to increase surgical precision in tumour identification and selective management of vessels. In post-operative care, potential uses of the metaverse include recovery monitoring and patient education. In urology, AR and VR have been widely explored in the past decade, mainly for surgical navigation in prostate and kidney cancer surgery, whereas only anecdotal metaverse experiences have been reported to date, specifically in partial nephrectomy. In the future, further integration of AI will improve the metaverse experience, potentially increasing the possibility of carrying out surgical navigation, data collection and virtual trials within the metaverse. However, challenges concerning data security and regulatory compliance must be addressed before the metaverse can be used to improve patient care.

Evolution of Three-Dimensional Computed Tomography Imaging in Thoracic Surgery

Radiologic reconstruction technology allows the wide use of three-dimensional (3D) computed tomography (CT) images in thoracic surgery. A minimally invasive surgery has become one of the standard therapies in thoracic surgery, and therefore, the need for preoperative and intraoperative simulations has increased. Three-dimensional CT images have been extensively used, and various types of software have been developed to reconstruct 3D-CT images for surgical simulation worldwide. Several software types have been commercialized and widely used by not only radiologists and technicians, but also thoracic surgeons. Three-dimensional CT images are helpful surgical guides; however, in almost all cases, they provide only static images, different from the intraoperative views. Lungs are soft and variable organs that can easily change shape by intraoperative inflation/deflation and surgical procedures. To address this issue, we have developed a novel software called the Resection Process Map (RPM), which creates variable virtual 3D images. Herein, we introduce the RPM and its development by tracking the history of 3D CT imaging in thoracic surgery. The RPM could help develop a real-time and accurate surgical navigation system for thoracic surgery.

A System for Mixed-Reality Holographic Overlays of Real-Time Rendered 3D-Reconstructed Imaging Using a Video Pass-through Head-Mounted Display-A Pathway to Future Navigation in Chest Wall Surgery

Background: Three-dimensional reconstructions of state-of-the-art high-resolution imaging are progressively being used more for preprocedural assessment in thoracic surgery. It is a promising tool that aims to improve patient-specific treatment planning, for example, for minimally invasive or robotic-assisted lung resections. Increasingly available mixed-reality hardware based on video pass-through technology enables the projection of image data as a hologram onto the patient. We describe the novel method of real-time 3D surgical planning in a mixed-reality setting by presenting three representative cases utilizing volume rendering. Materials: A mixed-reality system was set up using a high-performance workstation running a video pass-through-based head-mounted display. Image data from computer tomography were imported and volume-rendered in real-time to be customized through live editing. The image-based hologram was projected onto the patient, highlighting the regions of interest. Results: Three oncological cases were selected to explore the potentials of the mixed-reality system. Two of them presented large tumor masses in the thoracic cavity, while a third case presented an unclear lesion of the chest wall. We aligned real-time rendered 3D holographic image data onto the patient allowing us to investigate the relationship between anatomical structures and their respective body position. Conclusions: The exploration of holographic overlay has proven to be promising in improving preprocedural surgical planning, particularly for complex oncological tasks in the thoracic surgical field. Further studies on outcome-related surgical planning and navigation should therefore be conducted. Ongoing technological progress of extended reality hardware and intelligent software features will most likely enhance applicability and the range of use in surgical fields within the near future.