Artificial intelligence-assisted augmented reality robotic lung surgery: Navigating the future of thoracic surgery

The role of artificial intelligence in the diagnosis, imaging, and treatment of thoracic empyema

PURPOSE OF REVIEW

The management of thoracic empyema is often complicated by diagnostic delays, recurrence, treatment failures and infections with antibiotic resistant bacteria. The emergence of artificial intelligence (AI) in healthcare, particularly in clinical decision support, imaging, and diagnostic microbiology raises great expectations in addressing these challenges.

RECENT FINDINGS

Machine learning (ML) and AI models have been applied to CT scans and chest X-rays to identify and classify pleural effusions and empyema with greater accuracy. AI-based analyses can identify complex imaging features that are often missed by the human eye, improving diagnostic precision. AI-driven decision-support algorithms could reduce time to diagnosis, improve antibiotic stewardship, and enhance more precise and less invasive surgical therapy, significantly improving clinical outcomes and reducing inpatient hospital stays.

SUMMARY

ML and AI can analyse large datasets and recognize complex patterns and thus have the potential to enhance diagnostic accuracy, preop planning for thoracic surgery, and optimize surgical treatment strategies, antibiotic therapy, antibiotic stewardship, monitoring complications, and long-term patient management outcomes.

A novel 3D image registration technique for augmented reality vision in minimally invasive thoracoscopic pulmonary segmentectomy

PURPOSE

In this feasibility study, we aimed to create a dedicated pulmonary augmented reality (AR) workflow to enable a semi-automated intraoperative overlay of the pulmonary anatomy during video-assisted thoracoscopic surgery (VATS) or robot-assisted thoracoscopic surgery (RATS).

METHODS

Initially, the stereoscopic cameras were calibrated to obtain the intrinsic camera parameters. Intraoperatively, stereoscopic images were recorded and a 3D point cloud was generated from these images. By manually selecting the bifurcation key points, the 3D segmentation (from the diagnostic CT scan) was registered onto the intraoperative 3D point cloud.

RESULTS

Image reprojection errors were 0.34 and 0.22 pixels for the VATS and RATS cameras, respectively. We created disparity maps and point clouds for all eight patients. Time for creation of the 3D AR overlay was 5 min. Validation of the point clouds was performed, resulting in a median absolute error of 0.20 mm [IQR 0.10-0.54]. We were able to visualize the AR overlay and identify the arterial bifurcations adequately for five patients. In addition to creating AR overlays of the visible or invisible structures intraoperatively, we successfully visualized branch labels and altered the transparency of the overlays.

CONCLUSION

An algorithm was developed transforming the operative field into a 3D point cloud surface. This allowed for an accurate registration and visualization of preoperative 3D models. Using this system, surgeons can navigate through the patient's anatomy intraoperatively, especially during crucial moments, by visualizing otherwise invisible structures. This proposed registration method lays the groundwork for automated intraoperative AR navigation during minimally invasive pulmonary resections.

Evolution of Three-Dimensional Computed Tomography Imaging in Thoracic Surgery

Radiologic reconstruction technology allows the wide use of three-dimensional (3D) computed tomography (CT) images in thoracic surgery. A minimally invasive surgery has become one of the standard therapies in thoracic surgery, and therefore, the need for preoperative and intraoperative simulations has increased. Three-dimensional CT images have been extensively used, and various types of software have been developed to reconstruct 3D-CT images for surgical simulation worldwide. Several software types have been commercialized and widely used by not only radiologists and technicians, but also thoracic surgeons. Three-dimensional CT images are helpful surgical guides; however, in almost all cases, they provide only static images, different from the intraoperative views. Lungs are soft and variable organs that can easily change shape by intraoperative inflation/deflation and surgical procedures. To address this issue, we have developed a novel software called the Resection Process Map (RPM), which creates variable virtual 3D images. Herein, we introduce the RPM and its development by tracking the history of 3D CT imaging in thoracic surgery. The RPM could help develop a real-time and accurate surgical navigation system for thoracic surgery.