A Patient-Specific Mixed-Reality Visualization Tool for Thoracic Surgical Planning

Optimal planning and management strategies for minimally invasive lung segmentectomies: an international Delphi consensus report

OBJECTIVES

CALGB140503/JCOG0802 RCTs comparing lobectomy with sublobar resection in stage IA NSCLC have confirmed the non-inferiority of segmentectomy. Additional insight is needed to improve preoperative work-up and intraoperative strategies to increase safety and promote the dissemination of minimally invasive segmentectomy (MIS). A Delphi panel study assessed the level of consensus among surgeons for the planning and management of MIS.

METHODS

Twenty-one expert lung surgeons represented academic institutions, major teaching hospitals, and community hospitals from Europe, North America, and Asia. A 3-round Delphi methodology was used to analyse the answers of each panellist. Recognizing that questions with fewer response options have a higher consensus probability due to limited variability, weighted consensus thresholds were modified based on the number of response options.

RESULTS

The 21 panellists responded to all 3 rounds of questions. Based on the most robust consensus (94.4%), 3D chest CT reconstructions are recommended only when planning complex segmentectomies. Surgeons should perform 3D reconstructions chest CT scans (consensus = 83.3%). The most effective and safest technique is image-guided VATS in a hybrid operating room (consensus = 83.3%). Dyes with intravenous administration are the safest technique for identifying the intersegmental plane during MIS (consensus = 72.2%). Augmented/mixed reality will probably not immediately help reduce perioperative complications (consensus = 72.2%).

CONCLUSIONS

This Delphi consensus supports 3D reconstructions and preoperative pulmonary nodule localization before complex MIS. These recommendations should be considered when allocating resources to improve MIS's safety and oncologic efficacy for patients with small, early-stage lung cancers.

Application of image recognition-based tracker-less augmented reality navigation system in a series of sawbone trials

BACKGROUND

This study introduced an Augmented Reality (AR) navigation system to address limitations in conventional high tibial osteotomy (HTO). The objective was to enhance precision and efficiency in HTO procedures, overcoming challenges such as inconsistent postoperative alignment and potential neurovascular damage.

METHODS

The AR-MR (Mixed Reality) navigation system, comprising HoloLens, Unity Engine, and Vuforia software, was employed for pre-clinical trials using tibial sawbone models. CT images generated 3D anatomical models, projected via HoloLens, allowing surgeons to interact through intuitive hand gestures. The critical procedure of target tracking, essential for aligning virtual and real objects, was facilitated by Vuforia's feature detection algorithm.

RESULTS

In trials, the AR-MR system demonstrated significant reductions in both preoperative planning and intraoperative times compared to conventional navigation and metal 3D-printed surgical guides. The AR system, while exhibiting lower accuracy, exhibited efficiency, making it a promising option for HTO procedures. The preoperative planning time for the AR system was notably shorter (4 min) compared to conventional navigation (30.5 min) and metal guides (75.5 min). Intraoperative time for AR lasted 8.5 min, considerably faster than that of conventional navigation (31.5 min) and metal guides (10.5 min).

CONCLUSIONS

The AR navigation system presents a transformative approach to HTO, offering a trade-off between accuracy and efficiency. Ongoing improvements, such as the incorporation of two-stage registration and pointing devices, could further enhance precision. While the system may be less accurate, its efficiency renders it a potential breakthrough in orthopedic surgery, particularly for reducing unnecessary harm and streamlining surgical procedures.

3D printing technology and its revolutionary role in stent implementation in cardiovascular disease

Cardiovascular disease (CVD), exemplified by coronary artery disease (CAD), is a global health concern, escalating in prevalence and burden. The etiology of CAD is intricate, involving different risk factors. CVD remains a significant cause of mortality, driving the need for innovative interventions like percutaneous coronary intervention and vascular stents. These stents aim to minimize restenosis, thrombosis, and neointimal hyperplasia while providing mechanical support. Notably, the challenges of achieving ideal stent characteristics persist. An emerging avenue to address this involves enhancing the mechanical performance of polymeric bioresorbable stents using additive manufacturing techniques And Three-dimensional (3D) printing, encompassing various manufacturing technologies, has transcended its initial concept to become a tangible reality in the medical field. The technology's evolution presents a significant opportunity for pharmaceutical and medical industries, enabling the creation of targeted drugs and swift production of medical implants. It revolutionizes medical procedures, transforming the strategies of doctors and surgeons. Patient-specific 3D-printed anatomical models are now pivotal in precision medicine and personalized treatment approaches. Despite its ongoing development, additive manufacturing in healthcare is already integrated into various medical applications, offering substantial benefits to a sector under pressure for performance and cost reduction. In this review primarily emphasizes stent technology, different types of stents, highlighting its application with some potential complications. Here we also address their benefits, potential issues, effectiveness, indications, and contraindications. In future it can potentially reduce complications and help in improving patients' outcomes. 3DP technology offers the promise to customize solutions for complex CVD conditions and help or fostering a new era of precision medicine in cardiology.

Lung Cancer Surgery in Octogenarians: Implications and Advantages of Artificial Intelligence in the Preoperative Assessment

The general world population is aging and patients are often diagnosed with early-stage lung cancer at an advanced age. Several studies have shown that age is not itself a contraindication for lung cancer surgery, and therefore, more and more octogenarians with early-stage lung cancer are undergoing surgery with curative intent. However, octogenarians present some peculiarities that make surgical treatment more challenging, so an accurate preoperative selection is mandatory. In recent years, new artificial intelligence techniques have spread worldwide in the diagnosis, treatment, and therapy of lung cancer, with increasing clinical applications. However, there is still no evidence coming out from trials specifically designed to assess the potential of artificial intelligence in the preoperative evaluation of octogenarian patients. The aim of this narrative review is to investigate, through the analysis of the available international literature, the advantages and implications that these tools may have in the preoperative assessment of this particular category of frail patients. In fact, these tools could represent an important support in the decision-making process, especially in octogenarian patients in whom the diagnostic and therapeutic options are often questionable. However, these technologies are still developing, and a strict human-led process is mandatory.

What's new in minimally invasive thoracic surgery? Clinical application of augmented reality and learning opportunities in surgical simulation

Lung cancer represents the most lethal cancer worldwide. Surgery is the treatment of choice for early-stage non-small cell lung cancer, with an overall survival that can reach 90% at 5 years, but its detection is difficult to achieve due to the lack of symptoms. Screening programs are crucial to identify small cancer. Minimally invasive surgery has modified the therapeutical approach of these tumors, becoming the standard of care, with an important clinical yield in terms of reduction of postoperative pain and length of hospital stay. The aim of this mini-review is to explore and describe two important and innovative aspects in the context of "growing opportunities in minimally invasive thoracic surgery": the clinical application of augmented reality and its advantages for patient and surgeon, and the pedagogical issue through simulation-based training.

The HoloLens in medicine: A systematic review and taxonomy

The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.

Mixed Reality in Modern Surgical and Interventional Practice: Narrative Review of the Literature

BACKGROUND

Mixed reality (MR) and its potential applications have gained increasing interest within the medical community over the recent years. The ability to integrate virtual objects into a real-world environment within a single video-see-through display is a topic that sparks imagination. Given these characteristics, MR could facilitate preoperative and preinterventional planning, provide intraoperative and intrainterventional guidance, and aid in education and training, thereby improving the skills and merits of surgeons and residents alike.

OBJECTIVE

In this narrative review, we provide a broad overview of the different applications of MR within the entire spectrum of surgical and interventional practice and elucidate on potential future directions.

METHODS

A targeted literature search within the PubMed, Embase, and Cochrane databases was performed regarding the application of MR within surgical and interventional practice. Studies were included if they met the criteria for technological readiness level 5, and as such, had to be validated in a relevant environment.

RESULTS

A total of 57 studies were included and divided into studies regarding preoperative and interventional planning, intraoperative and interventional guidance, as well as training and education.

CONCLUSIONS

The overall experience with MR is positive. The main benefits of MR seem to be related to improved efficiency. Limitations primarily seem to be related to constraints associated with head-mounted display. Future directions should be aimed at improving head-mounted display technology as well as incorporation of MR within surgical microscopes, robots, and design of trials to prove superiority.

State of the Art in Lung Nodule Localization

Lung nodule and ground-glass opacity localization for diagnostic and therapeutic purposes is often a challenge for thoracic surgeons. While there are several adjuncts and techniques in the surgeon's armamentarium that can be helpful, accurate localization persists as a problem without a perfect solution. The last several decades have seen tremendous improvement in our ability to perform major operations with minimally invasive procedures and resulting lower morbidity. However, technological advances have not been as widely realized for lung nodule localization to complement minimally invasive surgery. This review describes the latest advances in lung nodule localization technology while also demonstrating that more efforts in this area are needed.

Artificial intelligence assisted display in thoracic surgery: development and possibilities

In this golden age of rapid development of artificial intelligence (AI), researchers and surgeons realized that AI could contribute to healthcare in all aspects, especially in surgery. The popularity of low-dose computed tomography (LDCT) and the improvement of the video-assisted thoracoscopic surgery (VATS) not only bring opportunities for thoracic surgery but also bring challenges on the way forward. Preoperatively localizing lung nodules precisely, intraoperatively identifying anatomical structures accurately, and avoiding complications requires a visual display of individuals’ specific anatomy for surgical simulation and assistance. With the advance of AI-assisted display technologies, including 3D reconstruction/3D printing, virtual reality (VR), augmented reality (AR), and mixed reality (MR), computer tomography (CT) imaging in thoracic surgery has been fully utilized for transforming 2D images to 3D model, which facilitates surgical teaching, planning, and simulation. AI-assisted display based on surgical videos is a new surgical application, which is still in its infancy. Notably, it has potential applications in thoracic surgery education, surgical quality evaluation, intraoperative assistance, and postoperative analysis. In this review, we illustrated the current AI-assisted display applications based on CT in thoracic surgery; focused on the emerging AI applications in thoracic surgery based on surgical videos by reviewing its relevant researches in other surgical fields and anticipate its potential development in thoracic surgery.

Augmented reality-assisted localization of solitary pulmonary nodules for precise sublobar lung resection: a preliminary study using an animal model

Background

Accurate localization of early lung cancer, manifested as solitary pulmonary nodules (SPNs) on computed tomography (CT), is critical in sublobar lung resection. The AR-assisted localization of SPNs was evaluated using a pig animal model.

Methods

A Microsoft HoloLens AR system was used. First, a plastic thoracic model was used for the pilot study. Three female 12 months 45 kg Danish Landrace Pigs were then used for the animal study. Thirty natural pulmonary structures, such as lymphonodus and bifurcated bronchioles or bronchial vessels, were chosen as simulated SPNs. The average angle between the actual puncturing needle and the expected path, the average distance between the puncture point and the plan point, and the difference between the actual puncturing depth and expected depth were recorded, and the accuracy rate was calculated.

Results

The point selected in the plastic thoracic model could be hit accurately with the assistance from the AR system in the pilot study. Moreover, the average angle between the actual puncturing needle and the expected path was 14.52°±6.04°. Meanwhile, the average distance between the puncture point and the expected point was 8.74±5.07 mm, and the difference between the actual and expected depths was 9.42±7.95 mm. Puncturing within a 1 cm³ area around the SPN using a hook-wire was considered a successful hit. The puncture accuracy was calculated. The average hit rate within a spherical area with a diameter of 1 cm range was 76.67%, and within a diameter of 2 cm range was 100%.

Conclusions

The HoloLens AR-assisted localization of SPNs may become a promising technique to improve the surgical treatment of early-stage lung cancer. Here, we evaluated its feasibility in an animal model. Nevertheless, its safety and effectiveness require further investigation in clinical trials.

Feasibility of the application of mixed reality in mandible reconstruction with fibula flap: A cadaveric specimen study

BACKGROUND

In recent years, a new technology, mixed reality (MR), has emerged and surpassed the limitations of augmented reality (AR) with its inability to interact with hologram. This study aimed to investigate the feasibility of the application of MR in mandible reconstruction with fibula flap.

METHODS

Computed tomography (CT) examination was performed for one cadaveric mandible and ten fibula bones. Using professional software Proplan CMF 3.0 (Materialize, Leuven, Belgium), we created a defected mandibular model and simulated the reconstruction design with these 10 fibula bones. The surgical plans were transferred to the HoloLens. We used HoloLens to guide the osteotomy and shaping of the fibular bone. After fixing the fibular segments using the Ti template, all segments underwent a CT examination. Before and after objects were compared for measurements of the location of fibular osteotomies, angular deviation of fibular segments, and intergonial angle distances.

RESULTS

The mean location of the fibular osteotomies, angular deviation of the fibular segments, and intergonial angle distances were 2.11 ± 1.31 mm, 2.85°± 1.97°, and 7.24 ± 3.42 mm, respectively.

CONCLUSION

The experimental results revealed that slight deviations remained in the accuracy of fibular osteotomy. With the further development of technology, it has the potential to improve the efficiency and precision of the reconstructive surgery.

Current and Future Applications of Virtual, Augmented, and Mixed Reality in Cardiothoracic Surgery

BACKGROUND

This review aims to examine the existing literature to address currently used virtual, augmented, and mixed reality modalities in the areas of preoperative surgical planning, intraoperative guidance, and postoperative management in the field of cardiothoracic surgery. In addition, this innovative technology provides future perspectives and potential benefits for cardiothoracic surgeons, trainees, and patients.

METHODS

A targeted, non-systematic literature assessment was performed within the Medline and Google Scholar databases to help identify current trends and to provide better understanding of the current state-of-the-art extended reality (XR) modalities in cardiothoracic surgery. Related articles published up to July 2020, are included in the review.

RESULTS

XR is a novel technique gaining increasing application in cardiothoracic surgery. It provides a three-dimensional (3D) and realistic view of structures and environments and offers the user the ability to interact with digital projections of surgical targets. Recent studies showed the validity and benefits of XR applications in cardiothoracic surgery. Examples include XR-guided pre-operative planning, intraoperative guidance and navigation, post-operative pain and rehabilitation management, surgical simulation, and patient education.

CONCLUSIONS

XR is gaining interest in the field of cardiothoracic surgery. In particular, there are promising roles for XR applications in televirtuality, surgical planning, surgical simulation, and perioperative management. However, future refinement and research is needed to further implement XR in the aforementioned settings within cardiothoracic surgery.