Progressive internal landmark registration for surgical navigation in laparoscopic gastrectomy for gastric cancer

Computer-assisted preoperative planning system with an automated registration method in prosthesis-driven oral implantology

PURPOSE

Dentition defect including edentulism is a problem that deserves attention, which requires precise preoperative planning. The trajectories of the implants can be determined using a pre-made radiographic template, which is adopted for prosthesis-driven oral implantology. However, existing solutions for the registration between the radiographic template and the patient's CBCT still require manual operation and cause inadequate accuracy. In this study, a pre-operative planning system for prosthesis-driven oral implantology is developed with a novel automated registration method.

METHODS

Based on threshold segmentation and morphological feature filtering, the potential feature points on two sets of CBCTs are, respectively, recognized. The distance features of the point sets are used to predict the optimal solution for point pair matching, after which the automated registration is implemented. The prosthesis-driven planning can be completed according to the results of registration and multi-planar reconstruction. Then, the surgical templates can be designed and fabricated using 3D printing technology based on the planning results and finally used for intra-operative guidance during implant placement.

RESULTS

Verification of the proposed method was conducted on three clinical cases. The mean Fiducial Registration Error of 0.13 ± 0.01mm was achieved with great efficiency. The average time was 0.15 s for the automatic registration algorithm, and 15.64 s for the whole procedure.

CONCLUSIONS

The proposed method proved to be accurate and robust. The results indicate that it can achieve higher efficiency while maintaining a low error level, which will have great potential clinical applications in the future.

The value of Augmented Reality in surgery - A usability study on laparoscopic liver surgery

Augmented Reality (AR) is considered to be a promising technology for the guidance of laparoscopic liver surgery. By overlaying pre-operative 3D information of the liver and internal blood vessels on the laparoscopic view, surgeons can better understand the location of critical structures. In an effort to enable AR, several authors have focused on the development of methods to obtain an accurate alignment between the laparoscopic video image and the pre-operative 3D data of the liver, without assessing the benefit that the resulting overlay can provide during surgery. In this paper, we present a study that aims to assess quantitatively and qualitatively the value of an AR overlay in laparoscopic surgery during a simulated surgical task on a phantom setup. We design a study where participants are asked to physically localise pre-operative tumours in a liver phantom using three image guidance conditions - a baseline condition without any image guidance, a condition where the 3D surfaces of the liver are aligned to the video and displayed on a black background, and a condition where video see-through AR is displayed on the laparoscopic video. Using data collected from a cohort of 24 participants which include 12 surgeons, we observe that compared to the baseline, AR decreases the median localisation error of surgeons on non-peripheral targets from 25.8 mm to 9.2 mm. Using subjective feedback, we also identify that AR introduces usability improvements in the surgical task and increases the perceived confidence of the users. Between the two tested displays, the majority of participants preferred to use the AR overlay instead of navigated view of the 3D surfaces on a separate screen. We conclude that AR has the potential to improve performance and decision making in laparoscopic surgery, and that improvements in overlay alignment accuracy and depth perception should be pursued in the future.

Database-driven patient-specific registration error compensation method for image-guided laparoscopic surgery

PURPOSE

A surgical navigation system helps surgeons understand anatomical structures in the operative field during surgery. Patient-to-image registration, which aligns coordinate systems between the CT volume and a positional tracker, is vital for accurate surgical navigation. Although a point-based rigid registration method using fiducials on the body surface is often utilized for laparoscopic surgery navigation, precise registration is difficult due to such factors as soft tissue deformation. We propose a method that compensates a transformation matrix computed using fiducials on the body surface based on the analysis of positional information in the database.

METHODS

We built our database by measuring the positional information of the fiducials and the guidance targets in both the CT volume and positional tracker coordinate systems through previous surgeries. We computed two transformation matrices: using only the fiducials and using only the guidance targets in all the data in the database. We calculated the differences between the two transformation matrices in each piece of data. The compensation transformation matrix was computed by averaging these difference matrices. In this step, we selected the data from the database based on the similarity of the fiducials and the configuration of the guidance targets.

RESULTS

We evaluated our proposed method using 20 pieces of data acquired during laparoscopic gastrectomy for gastric cancer. The locations of blood vessels were used as guidance targets for computing target registration error. The mean target registration errors significantly decreased from 33.0 to 17.1 mm before and after the compensation.

CONCLUSION

This paper described a registration error compensation method using a database for image-guided laparoscopic surgery. Since our proposed method reduced registration error without additional intraoperative measurements during surgery, it increases the accuracy of surgical navigation for laparoscopic surgery.

Stereotactic navigation using registration based on intra-abdominal landmarks in robotic-assisted lateral pelvic lymph node dissection

BACKGROUND

We carried out robot-assisted lateral pelvic lymph node dissection (LPLND) for rectal cancer with a stereotactic navigation system. The purpose of this study was to evaluate the accuracy and feasibility of the system.

METHODS

We constructed a navigation system based on the Polaris Spectra optical tracking device (Northern Digital Inc., Canada) and the open-source software 3D Slicer (version 3.8.1; http://www.slicer.org ). We used the landmark-based registration method for patient-to-image registration. Body surface landmarks and intra-abdominal landmarks were used. We evaluated the time required for registration and target registration error (TRE; the distance between corresponding points after registration) for the root of the superior gluteal artery the root of the obturator or superior vesical artery, and the obturator foramen during minimally invasive LPLND for rectal cancer. Five patients who had LPLND for rectal cancer at the University of Tokyo Hospital between September 2020 and May 2021 were enrolled.

RESULTS

The mean time required for registration was 49 s with the body surface landmarks and 88 s with the intra-abdominal landmarks. The mean TRE improved markedly when the registration was performed using intra-abdominal landmarks. The mean TRE of the root of the superior gluteal artery, the root of the obturator or superior vesical artery, and the obturator foramen were 55.8 mm, 53.4 mm, and 55.2 mm with the body surface landmarks and 11.8 mm, 10.0 mm, and 12.6 mm with the intra-abdominal landmarks, respectively. There were no adverse events related to the registration process.

CONCLUSIONS

When stereotactic navigation systems are used for minimally invasive LPLND, the use of intra-abdominal landmarks for registration is feasible and may allow simpler and more accurate navigation than the use of body surface landmarks.

Augmented reality navigation for liver resection with a stereoscopic laparoscope

OBJECTIVE

Understanding the three-dimensional (3D) spatial position and orientation of vessels and tumor(s) is vital in laparoscopic liver resection procedures. Augmented reality (AR) techniques can help surgeons see the patient's internal anatomy in conjunction with laparoscopic video images.

METHOD

In this paper, we present an AR-assisted navigation system for liver resection based on a rigid stereoscopic laparoscope. The stereo image pairs from the laparoscope are used by an unsupervised convolutional network (CNN) framework to estimate depth and generate an intraoperative 3D liver surface. Meanwhile, 3D models of the patient's surgical field are segmented from preoperative CT images using V-Net architecture for volumetric image data in an end-to-end predictive style. A globally optimal iterative closest point (Go-ICP) algorithm is adopted to register the pre- and intraoperative models into a unified coordinate space; then, the preoperative 3D models are superimposed on the live laparoscopic images to provide the surgeon with detailed information about the subsurface of the patient's anatomy, including tumors, their resection margins and vessels.

RESULTS

The proposed navigation system is tested on four laboratory ex vivo porcine livers and five operating theatre in vivo porcine experiments to validate its accuracy. The ex vivo and in vivo reprojection errors (RPE) are 6.04 ± 1.85 mm and 8.73 ± 2.43 mm, respectively.

CONCLUSION AND SIGNIFICANCE

Both the qualitative and quantitative results indicate that our AR-assisted navigation system shows promise and has the potential to be highly useful in clinical practice.

Application of image-guided navigation system for laparoscopic hepatobiliary surgery

BACKGROUND

To achieve safety of the operation, preoperative simulation became a routine practice for hepatobiliary and pancreatic (HBP) surgery. The use of intraoperative ultrasonography (IOUS) is essential in HBP surgery. There is a limitation in the use of IOUS in laparoscopic surgery (LS), for which a new intraoperative system is expected. We have developed an image-guided navigation system (IG-NS) for open HBP surgery since 2006, and we have applied our system to LS. The aim of this study is to evaluate the results of clinical application of IG-NS in LS.

MATERIALS AND METHODS

Eight patients underwent LS using IG-NS; LS consisted of cholecystectomy and hepatectomy in four patients each. After registration, the 3D models were superimposed on the surgical field. We performed LS while observing the navigation image. Moreover, we developed a support system for operations.

RESULTS

The average registration error was 8.8 mm for LS. Repeated registration was effective for organ deformation and improved the precision of IG-NS. By using various countermeasures, identification of the tumor's position and the setting of the resection line became easy.

CONCLUSION

As IG-NS provided real-time detailed and intuitive information, this intraoperative assist system may be an effective tool in LS.

A deformable model for navigated laparoscopic gastrectomy based on finite elemental method

Background: Accurate registration for surgical navigation of laparoscopic surgery is highly challenging due to vessel deformation. Here, we describe the design of a deformable model with improved matching accuracy by applying the finite element method (FEM).Material and methods: ANSYS software was used to simulate an FEM model of the vessel after pull-up based on laparoscopic gastrectomy requirements. The central line of the FEM model and the central line of the ground truth were drawn and compared. Based on the material and parameters determined from the animal experiment, a perigastric vessel FEM model of a gastric cancer patient was created, and its accuracy in a laparoscopic gastrectomy surgical scene was evaluated.Results: In the animal experiment, the FEM model created with Ogden foam material exhibited better results. The average distance between the two central lines was 6.5mm, and the average distance between their closest points was 3.8 mm. In the laparoscopic gastrectomy surgical scene, the FEM model and the true artery deformation demonstrated good coincidence.Conclusion: In this study, a deformable vessel model based on FEM was constructed using preoperative CT images to improve matching accuracy and to supply a reference for further research on deformation matching to facilitate laparoscopic gastrectomy navigation.

Latest technology in minimally invasive thoracic surgery

From the introduction of video-assisted thoracoscopic surgery (VATS) in the 1990s, to performing major lung resections using a uniportal VATS approach, technology has paved the way for the development of minimally invasive thoracic surgery. Natural orifice access to achieve a 'no port' approach, is also on the rise, with advancements in bronchoscopic techniques for diagnosis and therapy, as well as development of soft robotics to achieve desired flexibility, dexterity and stability in future platforms, which may involve in vivo deployment to bring the surgeon totally inside the body. Development of haptic feedback in robotic platforms to enhance the surgical experience is also a major goal, with vibrotactile and mechanical feedback generation, to replicate the traditional touch. In addition, the aid of technology in the form of procedural guidance mechanisms, like augmented reality, will further improve the safety and accuracy of future operations.

In vivo estimation of target registration errors during augmented reality laparoscopic surgery

PURPOSE

Successful use of augmented reality for laparoscopic surgery requires that the surgeon has a thorough understanding of the likely accuracy of any overlay. Whilst the accuracy of such systems can be estimated in the laboratory, it is difficult to extend such methods to the in vivo clinical setting. Herein we describe a novel method that enables the surgeon to estimate in vivo errors during use. We show that the method enables quantitative evaluation of in vivo data gathered with the SmartLiver image guidance system.

METHODS

The SmartLiver system utilises an intuitive display to enable the surgeon to compare the positions of landmarks visible in both a projected model and in the live video stream. From this the surgeon can estimate the system accuracy when using the system to locate subsurface targets not visible in the live video. Visible landmarks may be either point or line features. We test the validity of the algorithm using an anatomically representative liver phantom, applying simulated perturbations to achieve clinically realistic overlay errors. We then apply the algorithm to in vivo data.

RESULTS

The phantom results show that using projected errors of surface features provides a reliable predictor of subsurface target registration error for a representative human liver shape. Applying the algorithm to in vivo data gathered with the SmartLiver image-guided surgery system shows that the system is capable of accuracies around 12 mm; however, achieving this reliably remains a significant challenge.

CONCLUSION

We present an in vivo quantitative evaluation of the SmartLiver image-guided surgery system, together with a validation of the evaluation algorithm. This is the first quantitative in vivo analysis of an augmented reality system for laparoscopic surgery.

Recent Development of Augmented Reality in Surgery: A Review

Introduction

The development augmented reality devices allow physicians to incorporate data visualization into diagnostic and treatment procedures to improve work efficiency, safety, and cost and to enhance surgical training. However, the awareness of possibilities of augmented reality is generally low. This review evaluates whether augmented reality can presently improve the results of surgical procedures.

Methods

We performed a review of available literature dating from 2010 to November 2016 by searching PubMed and Scopus using the terms “augmented reality” and “surgery.” Results. The initial search yielded 808 studies. After removing duplicates and including only journal articles, a total of 417 studies were identified. By reading of abstracts, 91 relevant studies were chosen to be included. 11 references were gathered by cross-referencing. A total of 102 studies were included in this review.

Conclusions

The present literature suggest an increasing interest of surgeons regarding employing augmented reality into surgery leading to improved safety and efficacy of surgical procedures. Many studies showed that the performance of newly devised augmented reality systems is comparable to traditional techniques. However, several problems need to be addressed before augmented reality is implemented into the routine practice.

Three-Dimensional Liver Surgery Simulation: Computer-Assisted Surgical Planning with Three-Dimensional Simulation Software and Three-Dimensional Printing<sup/>

To perform accurate hepatectomy without injury, it is necessary to understand the anatomical relationship among the branches of Glisson's sheath, hepatic veins, and tumor. In Japan, three-dimensional (3D) preoperative simulation for liver surgery is becoming increasingly common, and liver 3D modeling and 3D hepatectomy simulation by 3D analysis software for liver surgery have been covered by universal healthcare insurance since 2012. Herein, we review the history of virtual hepatectomy using computer-assisted surgery (CAS) and our research to date, and we discuss the future prospects of CAS. We have used the SYNAPSE VINCENT medical imaging system (Fujifilm Medical, Tokyo, Japan) for 3D visualization and virtual resection of the liver since 2010. We developed a novel fusion imaging technique combining 3D computed tomography (CT) with magnetic resonance imaging (MRI). The fusion image enables us to easily visualize anatomic relationships among the hepatic arteries, portal veins, bile duct, and tumor in the hepatic hilum. In 2013, we developed an original software, called Liversim, which enables real-time deformation of the liver using physical simulation, and a randomized control trial has recently been conducted to evaluate the use of Liversim and SYNAPSE VINCENT for preoperative simulation and planning. Furthermore, we developed a novel hollow 3D-printed liver model whose surface is covered with frames. This model is useful for safe liver resection, has better visibility, and the production cost is reduced to one-third of a previous model. Preoperative simulation and navigation with CAS in liver resection are expected to help planning and conducting a surgery and surgical education. Thus, a novel CAS system will contribute to not only the performance of reliable hepatectomy but also to surgical education.