Short rigid scope and stereo-scope designed specifically for open abdominal navigation surgery: clinical application for hepatobiliary and pancreatic surgery

Augmented reality-assisted navigation system contributes to better intraoperative and short-time outcomes of laparoscopic pancreaticoduodenectomy: a retrospective cohort study

BACKGROUND

Augmented reality (AR)-assisted navigation system are currently good techniques for hepatectomy; however, its application and efficacy for laparoscopic pancreatoduodenectomy have not been reported. This study sought to focus on and evaluate the advantages of laparoscopic pancreatoduodenectomy guided by the AR-assisted navigation system in intraoperative and short-time outcomes.

METHODS

Eighty-two patients who underwent laparoscopic pancreatoduodenectomy from January 2018 to May 2022 were enrolled and divided into the AR and non-AR groups. Clinical baseline features, operation time, intraoperative blood loss, blood transfusion rate, perioperative complications, and mortality were analyzed.

RESULTS

AR-guided laparoscopic pancreaticoduodenectomy was performed in the AR group ( n =41), whereas laparoscopic pancreatoduodenectomy was carried out routinely in the non-AR group ( n =41). There was no significant difference in baseline data between the two groups ( P >0.05); Although the operation time of the AR group was longer than that of the non-AR group (420.15±94.38 vs. 348.98±76.15, P <0.001), the AR group had a less intraoperative blood loss (219.51±167.03 vs. 312.20±195.51, P =0.023), lower blood transfusion rate (24.4 vs. 65.9%, P <0.001), lower occurrence rates of postoperative pancreatic fistula (12.2 vs. 46.3%, P =0.002) and bile leakage (0 vs. 14.6%, P =0.026), and shorter postoperative hospital stay (11.29±2.78 vs. 20.04±11.22, P <0.001) compared with the non-AR group.

CONCLUSION

AR-guided laparoscopic pancreatoduodenectomy has significant advantages in identifying important vascular structures, minimizing intraoperative damage, and reducing postoperative complications, suggesting that it is a safe, feasible method with a bright future in the clinical setting.

Augmented reality in liver surgery

INTRODUCTION

During an operation, augmented reality (AR) enables surgeons to enrich their vision of the operating field by means of digital imagery, particularly as regards tumors and anatomical structures. While in some specialties, this type of technology is routinely ustilized, in liver surgery due to the complexity of modeling organ deformities in real time, its applications remain limited. At present, numerous teams are attempting to find a solution applicable to current practice, the objective being to overcome difficulties of intraoperative navigation in an opaque organ.

OBJECTIVE

To identify, itemize and analyze series reporting AR techniques tested in liver surgery, the objectives being to establish a state of the art and to provide indications of perspectives for the future.

METHODS

In compliance with the PRISMA guidelines and availing ourselves of the PubMed, Embase and Cochrane databases, we identified English-language articles published between January 2020 and January 2022 corresponding to the following keywords: augmented reality, hepatic surgery, liver and hepatectomy.

RESULTS

Initially, 102 titles, studies and summaries were preselected. Twenty-eight corresponding to the inclusion criteria were included, reporting on 183patients operated with the help of AR by laparotomy (n=31) or laparoscopy (n=152). Several techniques of acquisition and visualization were reported. Anatomical precision was the main assessment criterion in 19 articles, with values ranging from 3mm to 14mm, followed by time of acquisition and clinical feasibility.

CONCLUSION

While several AR technologies are presently being developed, due to insufficient anatomical precision their clinical applications have remained limited. That much said, numerous teams are currently working toward their optimization, and it is highly likely that in the short term, the application of AR in liver surgery will have become more frequent and effective. As for its clinical impact, notably in oncology, it remains to be assessed.

Minimally invasive gastrointestinal surgery: From past to the future

The improvement of the science and art of surgery began over 150 years ago. Surgical core tasks, “cutting and sewing” with hand and direct contact with the organs, have remained the same. However, in the 21st century, there has been a shifting paradigm in the methodology of surgery. The joint union between innovators, engineers, industry, and patient demands resulted in minimally invasive surgery (MIS). This method has influenced the techniques in every aspect of abdominal surgery, such as surgeons are not required to direct contact or see the structures on which they operate. Advances in the endoscope, imaging, and improved instrumentations convert the essential open surgery into the endoscopic method. Furthermore, computers and robotics show a promising future to facilitate complex procedures, enhance accuracy in microscale operations, and develop a simulation to improve the ability to face sophisticated approaches. MIS has been replacing open surgery due to improved survival, fewer complications, and rapid recoveries in recent years. Minimally invasive surgery's further research in diagnostic and therapeutic modalities is under investigation to achieve genuinely “noninvasive” surgery. Thus, MIS has gained interest in recent days and has been improving with promising outcomes.

•

Minimally invasive surgery has interfered with multiple aspects of the surgical approach.

•

Advancement in the endoscope, imaging, and other instrumentations shifting the current methodological conventional surgery.

•

The benefit over risk is the promising primary outcome to achieve an exceptional quality of life.

Current trends in three-dimensional visualization and real-time navigation as well as robot-assisted technologies in hepatobiliary surgery

With the continuous development of digital medicine, minimally invasive precision and safety have become the primary development trends in hepatobiliary surgery. Due to the specificity and complexity of hepatobiliary surgery, traditional preoperative imaging techniques such as computed tomography and magnetic resonance imaging cannot meet the need for identification of fine anatomical regions. Imaging-based three-dimensional (3D) reconstruction, virtual simulation of surgery and 3D printing optimize the surgical plan through preoperative assessment, improving the controllability and safety of intraoperative operations, and in difficult-to-reach areas of the posterior and superior liver, assistive robots reproduce the surgeon’s natural movements with stable cameras, reducing natural vibrations. Electromagnetic navigation in abdominal surgery solves the problem of conventional surgery still relying on direct visual observation or preoperative image assessment. We summarize and compare these recent trends in digital medical solutions for the future development and refinement of digital medicine in hepatobiliary surgery.

Clinical use of augmented reality, mixed reality, three-dimensional-navigation and artificial intelligence in liver surgery

A precise knowledge of intra-parenchymal vascular and biliary architecture and the location of lesions in relation to the complex anatomy is indispensable to perform liver surgery. Therefore, virtual three-dimensional (3D)-reconstruction models from computed tomography/magnetic resonance imaging scans of the liver might be helpful for visualization. Augmented reality, mixed reality and 3D-navigation could transfer such 3D-image data directly into the operation theater to support the surgeon. This review examines the literature about the clinical and intraoperative use of these image guidance techniques in liver surgery and provides the reader with the opportunity to learn about these techniques. Augmented reality and mixed reality have been shown to be feasible for the use in open and minimally invasive liver surgery. 3D-navigation facilitated targeting of intraparenchymal lesions. The existing data is limited to small cohorts and description about technical details e.g., accordance between the virtual 3D-model and the real liver anatomy. Randomized controlled trials regarding clinical data or oncological outcome are not available. Up to now there is no intraoperative application of artificial intelligence in liver surgery. The usability of all these sophisticated image guidance tools has still not reached the grade of immersion which would be necessary for a widespread use in the daily surgical routine. Although there are many challenges, augmented reality, mixed reality, 3D-navigation and artificial intelligence are emerging fields in hepato-biliary surgery.

Development of recognised position-guided navigation system

BACKGROUND

Previously, we developed an image-guided navigation system (IG-NS) incorporating augmented reality technology. Nevertheless, the system could still only aid the operator by presenting imagery and was short of achieving the goal of developing a real navigation system. Therefore, we developed a recognised position-guided navigation system (RP-NS) and herein reported the functionality and usefulness of this system in a phantom model for clinical applications.

METHODS

We developed RP-NS which was reconstructed by adding the positional recognition and instruction functions with the cautions by displaying the images on the monitor with a voice to the IG-NS. We evaluated accuracy of positional recognition and instruction functions using phantom model. By utilising the chronological recording of the tip position of the surgical apparatus, the surgical precision of the operators was assessed. Finally, the feasibility of improvements in surgical precision using this system was evaluated.

RESULTS

The RP-NS indicated an accuracy of the position recognition functions with an error of 2.7 mm. The surgeons could perform partial hepatectomies within mean value of 7.5% error as compared with calculated volume according to the instruction. Improvements in surgical precision using this system were obtained on the surgeons with different levels.

CONCLUSIONS

The RP-NS was highly effective as a navigation system owing to precise positional recognition and adequate instruction functions. Therefore, these results indicate that the use of this system may complement differences in proficiency, and numerically evaluate surgical skills and analyse tendencies of surgeons.

[Systematic review of current trends in preoperative planning of surgery for liver tumors]

The purpose of the study was a systematic review of current trends in preoperative planning of surgery for liver tumors. These data will be valuable to determine the advantages and disadvantages of 3D modeling, augmented reality technology and 3D printing in preoperative planning of surgery for focal liver lesions.

Augmented Reality during Open Liver Surgery Using a Markerless Non-rigid Registration System

INTRODUCTION

Intraoperative navigation during liver resection remains difficult and requires high radiologic skills because liver anatomy is complex and patient-specific. Augmented reality (AR) during open liver surgery could be helpful to guide hepatectomies and optimize resection margins but faces many challenges when large parenchymal deformations take place. We aimed to experiment a new vision-based AR to assess its clinical feasibility and anatomical accuracy.

PATIENTS AND METHODS

Based on preoperative CT scan 3-D segmentations, we applied a non-rigid registration method, integrating a physics-based elastic model of the liver, computed in real time using an efficient finite element method. To fit the actual deformations, the model was driven by data provided by a single RGB-D camera. Five livers were considered in this experiment. In vivo AR was performed during hepatectomy (n = 4), with manual handling of the livers resulting in large realistic deformations. Ex vivo experiment (n = 1) consisted in repeated CT scans of explanted whole organ carrying internal metallic landmarks, in fixed deformations, and allowed us to analyze our estimated deformations and quantify spatial errors.

RESULTS

In vivo AR tests were successfully achieved in all patients with a fast and agile setup installation (< 10 min) and real-time overlay of the virtual anatomy onto the surgical field displayed on an external screen. In addition, an ex vivo quantification demonstrated a 7.9 mm root mean square error for the registration of internal landmarks.

CONCLUSION

These first experiments of a markerless AR provided promising results, requiring very little equipment and setup time, yet providing real-time AR with satisfactory 3D accuracy. These results must be confirmed in a larger prospective study to definitively assess the impact of such minimally invasive technology on pathological margins and oncological outcomes.

Role of artificial intelligence in hepatobiliary and pancreatic surgery

Over the past decade, enhanced preoperative imaging and visualization, improved delineation of the complex anatomical structures of the liver and pancreas, and intra-operative technological advances have helped deliver the liver and pancreatic surgery with increased safety and better postoperative outcomes. Artificial intelligence (AI) has a major role to play in 3D visualization, virtual simulation, augmented reality that helps in the training of surgeons and the future delivery of conventional, laparoscopic, and robotic hepatobiliary and pancreatic (HPB) surgery; artificial neural networks and machine learning has the potential to revolutionize individualized patient care during the preoperative imaging, and postoperative surveillance. In this paper, we reviewed the existing evidence and outlined the potential for applying AI in the perioperative care of patients undergoing HPB surgery.

Image-guided minimally invasive endopancreatic surgery using a computer-assisted navigation system

BACKGROUND

Minimally invasive endopancreatic surgery (EPS), performing a pancreatic resection from inside the pancreatic duct, has been proposed as an experimental alternative to duodenum-preserving pancreatic head resection in benign diseases such as chronic pancreatitis, but is complicated by difficult spatial orientation when trying to reach structures of interest. This study assessed the feasibility and potential benefits of image-guided EPS using a computer-assisted navigation system in artificial pancreas silicon model.

METHODS

A surgical navigation system displayed a 3D reconstruction of the original computed tomography (CT) scan and the endoscope in relation to a selected target structure. In a first step, different surface landmark (LM)-based and intraparenchymal LM-based approaches for image-to-physical space registration were evaluated. The accuracy of registration was measured as fiducial registration error (FRE). Subsequently, intrapancreatic lesions (n = 8) that were visible on preoperative imaging, but not on the endoscopic view, were targeted with a computer-assisted, image-guided endopancreatic resection technique in pancreas silicon models. After each experiment, a CT scan was obtained for measurement of the shortest distance from the resection cavity to the centre of the lesion.

RESULTS

Intraparenchymal LM registration [FRE 2.24 mm (1.40-2.85)] was more accurate than surface LM registration [FRE 3.46 mm (2.25-4.85); p = 0.035], but not more accurate than combined registration of intraparenchymal and surface LM [FRE 2.46 mm (1.60-3.35); p = 0.052]. Using image-guided EPS, six of seven lesions were successfully targeted. The median distance from the resection cavity to the centre of the lesion on CT was 1.52 mm (0-2.4). In one pancreas, a lesion could not be resected due to the fragility of the pancreas model.

CONCLUSION

Image-guided minimally invasive EPS using a computer-assisted navigation system enabled successful targeting of pancreatic lesions that were invisible on the endoscopic image, but detectable on preoperative imaging. In the clinical setting, this tool could facilitate complex minimally invasive and robotic pancreatic procedures.

Application of image-guided navigation system for laparoscopic hepatobiliary surgery

BACKGROUND

To achieve safety of the operation, preoperative simulation became a routine practice for hepatobiliary and pancreatic (HBP) surgery. The use of intraoperative ultrasonography (IOUS) is essential in HBP surgery. There is a limitation in the use of IOUS in laparoscopic surgery (LS), for which a new intraoperative system is expected. We have developed an image-guided navigation system (IG-NS) for open HBP surgery since 2006, and we have applied our system to LS. The aim of this study is to evaluate the results of clinical application of IG-NS in LS.

MATERIALS AND METHODS

Eight patients underwent LS using IG-NS; LS consisted of cholecystectomy and hepatectomy in four patients each. After registration, the 3D models were superimposed on the surgical field. We performed LS while observing the navigation image. Moreover, we developed a support system for operations.

RESULTS

The average registration error was 8.8 mm for LS. Repeated registration was effective for organ deformation and improved the precision of IG-NS. By using various countermeasures, identification of the tumor's position and the setting of the resection line became easy.

CONCLUSION

As IG-NS provided real-time detailed and intuitive information, this intraoperative assist system may be an effective tool in LS.

Virtual and Augmented Reality in Oncologic Liver Surgery

Virtual reality (VR) and augmented reality (AR) in complex surgery are evolving technologies enabling improved preoperative planning and intraoperative navigation. The basis of these technologies is a computer-based generation of a patient-specific 3-dimensional model from Digital Imaging and Communications in Medicine (DICOM) data. This article provides a state-of-the- art overview on the clinical use of this technology with a specific focus on hepatic surgery. Although VR and AR are still in an evolving stage with only some clinical application today, these technologies have the potential to become a key factor in improving preoperative and intraoperative decision making.

Augmented reality in open surgery

Augmented reality (AR) has been successfully providing surgeons an extensive visual information of surgical anatomy to assist them throughout the procedure. AR allows surgeons to view surgical field through the superimposed 3D virtual model of anatomical details. However, open surgery presents new challenges. This study provides a comprehensive overview of the available literature regarding the use of AR in open surgery, both in clinical and simulated settings. In this way, we aim to analyze the current trends and solutions to help developers and end/users discuss and understand benefits and shortcomings of these systems in open surgery. We performed a PubMed search of the available literature updated to January 2018 using the terms (1) "augmented reality" AND "open surgery", (2) "augmented reality" AND "surgery" NOT "laparoscopic" NOT "laparoscope" NOT "robotic", (3) "mixed reality" AND "open surgery", (4) "mixed reality" AND "surgery" NOT "laparoscopic" NOT "laparoscope" NOT "robotic". The aspects evaluated were the following: real data source, virtual data source, visualization processing modality, tracking modality, registration technique, and AR display type. The initial search yielded 502 studies. After removing the duplicates and by reading abstracts, a total of 13 relevant studies were chosen. In 1 out of 13 studies, in vitro experiments were performed, while the rest of the studies were carried out in a clinical setting including pancreatic, hepatobiliary, and urogenital surgeries. AR system in open surgery appears as a versatile and reliable tool in the operating room. However, some technological limitations need to be addressed before implementing it into the routine practice.

Novel navigation system by augmented reality technology using a tablet PC for hepatobiliary and pancreatic surgery

BACKGROUND

We previously developed an image-guided navigation system (IG-NS) using augmented reality technology for hepatobiliary and pancreatic (HBP) surgery. This system superimposed a 3D model onto a stereoscope-captured surgical field (i.e., the scope method). Unfortunately, this method requires an expensive stereoscope, surgeons have to shift their eyesight away from the surgical field, and the method has poor controllability. Therefore, an IG-NS using a tablet PC (i.e., the tablet method) was developed. The aim of the current study is to evaluate the efficiency of this novel method.

METHODS

We studied 9 patients, for whom a 3D model was created from computed tomography images. After registration was performed, the 3D model was superimposed onto the surgical field, which was captured by the tablet PC's camera.

RESULTS

The IG-NS could be applied with very little time lag. The visibility and controllability of the tablet method were superior to those of the scope method. It was especially useful in surgery for multiple metastatic liver carcinoma due to easy localization of the position of the carcinomas and vessels.

CONCLUSIONS

We successfully developed the tablet method and tested it in a clinical setting. This system may contribute to surgical efficacy and improve the educational effects.

Augmented reality technology for preoperative planning and intraoperative navigation during hepatobiliary surgery: A review of current methods

BACKGROUND

Augmented reality (AR) technology is used to reconstruct three-dimensional (3D) images of hepatic and biliary structures from computed tomography and magnetic resonance imaging data, and to superimpose the virtual images onto a view of the surgical field. In liver surgery, these superimposed virtual images help the surgeon to visualize intrahepatic structures and therefore, to operate precisely and to improve clinical outcomes.

DATA SOURCES

The keywords "augmented reality", "liver", "laparoscopic" and "hepatectomy" were used for searching publications in the PubMed database. The primary source of literatures was from peer-reviewed journals up to December 2016. Additional articles were identified by manual search of references found in the key articles.

RESULTS

In general, AR technology mainly includes 3D reconstruction, display, registration as well as tracking techniques and has recently been adopted gradually for liver surgeries including laparoscopy and laparotomy with video-based AR assisted laparoscopic resection as the main technical application. By applying AR technology, blood vessels and tumor structures in the liver can be displayed during surgery, which permits precise navigation during complex surgical procedures. Liver transformation and registration errors during surgery were the main factors that limit the application of AR technology.

CONCLUSIONS

With recent advances, AR technologies have the potential to improve hepatobiliary surgical procedures. However, additional clinical studies will be required to evaluate AR as a tool for reducing postoperative morbidity and mortality and for the improvement of long-term clinical outcomes. Future research is needed in the fusion of multiple imaging modalities, improving biomechanical liver modeling, and enhancing image data processing and tracking technologies to increase the accuracy of current AR methods.

Augmented reality navigation in open surgery for hilar cholangiocarcinoma resection with hemihepatectomy using video-based in situ three-dimensional anatomical modeling: A case report

RATIONALE

Patients who undergo hilar cholangiocarcinoma (HCAC) resection with concomitant hepatectomy have a high risk of postoperative morbidity and mortality due to surgical trauma to the hepatic and biliary vasculature.

PATIENT CONCERNS

A 58-year-old Chinese man with yellowing skin and sclera, abdominal distension, pruritus, and anorexia for approximately 3 weeks.

DIAGNOSES

Magnetic resonance cholangiopancreatography and enhanced computed tomography (CT) scanning revealed a mass over the biliary tree at the porta hepatis, which diagnosed to be s a hilar cholangiocarcinoma.

INTERVENTION

Three-dimensional (3D) images of the patient's hepatic and biliary structures were reconstructed preoperatively from CT data, and the 3D images were used for preoperative planning and augmented reality (AR)-assisted intraoperative navigation during open HCAC resection with hemihepatectomy. A 3D-printed model of the patient's biliary structures was also used intraoperatively as a visual reference.

OUTCOMES

No serious postoperative complications occurred, and the patient was tumor-free at the 9-month follow-up examination based on CT results.

LESSONS

AR-assisted preoperative planning and intraoperative navigation might be beneficial in other patients with HCAC patients to reduce postoperative complications and ensure disease-free survival. In our postoperative analysis, we also found that, when the3D images were superimposed 3D-printed model using a see-through integral video graphy display device, our senses of depth perception and motion parallax were improved, compared with that which we had experienced intraoperatively using the videobased AR display system.

Recent Development of Augmented Reality in Surgery: A Review

Introduction

The development augmented reality devices allow physicians to incorporate data visualization into diagnostic and treatment procedures to improve work efficiency, safety, and cost and to enhance surgical training. However, the awareness of possibilities of augmented reality is generally low. This review evaluates whether augmented reality can presently improve the results of surgical procedures.

Methods

We performed a review of available literature dating from 2010 to November 2016 by searching PubMed and Scopus using the terms “augmented reality” and “surgery.” Results. The initial search yielded 808 studies. After removing duplicates and including only journal articles, a total of 417 studies were identified. By reading of abstracts, 91 relevant studies were chosen to be included. 11 references were gathered by cross-referencing. A total of 102 studies were included in this review.

Conclusions

The present literature suggest an increasing interest of surgeons regarding employing augmented reality into surgery leading to improved safety and efficacy of surgical procedures. Many studies showed that the performance of newly devised augmented reality systems is comparable to traditional techniques. However, several problems need to be addressed before augmented reality is implemented into the routine practice.

Augmented Reality Guidance for the Resection of Missing Colorectal Liver Metastases: An Initial Experience

BACKGROUND

Modern chemotherapy achieves the shrinking of colorectal cancer liver metastases (CRLM) to such extent that they may disappear from radiological imaging. Disappearing CRLM rarely represents a complete pathological remission and have an important risk of recurrence. Augmented reality (AR) consists in the fusion of real-time patient images with a computer-generated 3D virtual patient model created from pre-operative medical imaging. The aim of this prospective pilot study is to investigate the potential of AR navigation as a tool to help locate and surgically resect missing CRLM.

METHODS

A 3D virtual anatomical model was created from thoracoabdominal CT-scans using customary software (VR RENDER(®), IRCAD). The virtual model was superimposed to the operative field using an Exoscope (VITOM(®), Karl Storz, Tüttlingen, Germany). Virtual and real images were manually registered in real-time using a video mixer, based on external anatomical landmarks with an estimated accuracy of 5 mm. This modality was tested in three patients, with four missing CRLM that had sizes from 12 to 24 mm, undergoing laparotomy after receiving pre-operative oxaliplatin-based chemotherapy.

RESULTS

AR display and fine registration was performed within 6 min. AR helped detect all four missing CRLM, and guided their resection. In all cases the planned security margin of 1 cm was clear and resections were confirmed to be R0 by pathology. There was no postoperative major morbidity or mortality. No local recurrence occurred in the follow-up period of 6-22 months.

CONCLUSIONS

This initial experience suggests that AR may be a helpful navigation tool for the resection of missing CRLM.

Augmented Reality Guidance for the Resection of Missing Colorectal Liver Metastases: An Initial Experience

BACKGROUND

Modern chemotherapy achieves the shrinking of colorectal cancer liver metastases (CRLM) to such extent that they may disappear from radiological imaging. Disappearing CRLM rarely represents a complete pathological remission and have an important risk of recurrence. Augmented reality (AR) consists in the fusion of real-time patient images with a computer-generated 3D virtual patient model created from pre-operative medical imaging. The aim of this prospective pilot study is to investigate the potential of AR navigation as a tool to help locate and surgically resect missing CRLM.

METHODS

A 3D virtual anatomical model was created from thoracoabdominal CT-scans using customary software (VR RENDER(®), IRCAD). The virtual model was superimposed to the operative field using an Exoscope (VITOM(®), Karl Storz, Tüttlingen, Germany). Virtual and real images were manually registered in real-time using a video mixer, based on external anatomical landmarks with an estimated accuracy of 5 mm. This modality was tested in three patients, with four missing CRLM that had sizes from 12 to 24 mm, undergoing laparotomy after receiving pre-operative oxaliplatin-based chemotherapy.

RESULTS

AR display and fine registration was performed within 6 min. AR helped detect all four missing CRLM, and guided their resection. In all cases the planned security margin of 1 cm was clear and resections were confirmed to be R0 by pathology. There was no postoperative major morbidity or mortality. No local recurrence occurred in the follow-up period of 6-22 months.

CONCLUSIONS

This initial experience suggests that AR may be a helpful navigation tool for the resection of missing CRLM.

Navigation surgery using an augmented reality for pancreatectomy

AIM

The aim of this study was to evaluate the utility of navigation surgery using augmented reality technology (AR-based NS) for pancreatectomy.

METHODS

The 3D reconstructed images from CT were created by segmentation. The initial registration was performed by using the optical location sensor. The reconstructed images were superimposed onto the real organs in the monitor display. Of the 19 patients who had undergone hepatobiliary and pancreatic surgery using AR-based NS, the accuracy, visualization ability, and utility of our system were assessed in five cases with pancreatectomy.

RESULTS

The position of each organ in the surface-rendering image corresponded almost to that of the actual organ. Reference to the display image allowed for safe dissection while preserving the adjacent vessels or organs. The locations of the lesions and resection line on the targeted organ were overlaid on the operating field. The initial mean registration error was improved to approximately 5 mm by our refinements. However, several problems such as registration accuracy, portability and cost still remain.

CONCLUSION

AR-based NS contributed to accurate and effective surgical resection in pancreatectomy. The pancreas appears to be a suitable organ for further investigations. This technology is promising to improve surgical quality, training, and education.

Clinical application of navigation surgery using augmented reality in the abdominal field

This article presents general principles and recent advancements in the clinical application of augmented reality-based navigation surgery (AR based NS) for abdominal procedures and includes a description of our clinical trial and subsequent outcomes. Moreover, current problems and future aspects are discussed. The development of AR-based NS in the abdomen is delayed compared with another field because of the problem of intraoperative organ deformations or the existence of established modalities. Although there are a few reports on the clinical use of AR-based NS for digestive surgery, sophisticated technologies in urology have often been reported. However, the rapid widespread use of video- or robot assisted surgeries requires this technology. We have worked to develop a system of AR-based NS for hepatobiliary and pancreatic surgery. Then we developed a short rigid scope that enables surgeons to obtain 3D view. We recently focused on pancreatic surgery, because intraoperative organ shifting is minimal. The position of each organ in overlaid image almost corresponded with that of the actual organ with about 5 mm of mean registration errors. Intraoperative information generated from this system provided us with useful navigation. However, AR-based NS has several problems to overcome such as organ deformity, evaluation of utility, portability or cost.

Identification of inferior pancreaticoduodenal artery during pancreaticoduodenectomy using augmented reality-based navigation system

BACKGROUND

In pancreaticoduodenectomy (PD), early ligation of the inferior pancreaticoduodenal artery (IPDA) before efferent veins has been advocated to decrease blood loss by congestion of the pancreatic head to be resected. In this study, we herein report the utility of early identification of the IPDA using an augmented reality (AR)-based navigation system (NS).

METHODS

Seven nonconsecutive patients underwent PD using AR-based NS. After paired-point matching registration, the reconstructed image obtained by preoperative computed tomography (CT) was fused with a real-time operative field image and displayed on 3D monitors. The vascular reconstructed images, including the superior mesenteric artery, jejunal artery, and IPDA were visualized to facilitate image-guided surgical procedures. We compared operating time and intraoperative blood loss of six patients who successfully underwent identification of IPDA using AR-based NS (group A) with nine patients who underwent early ligation of IPDA without using AR (group B) and 18 patients who underwent a conventional PD (group C).

RESULTS

The IPDA or the jejunal artery was rapidly identified and ligated in six patients. The mean operating time and intraoperative blood loss in group A was 415 min and 901 ml, respectively. There was no significant difference in operating time and intraoperative blood loss among the groups.

CONCLUSIONS

The AR-based NS provided precise anatomical information, which allowed the surgeons to rapidly identify and perform early ligation of IPDA in PD.