Development of a navigation system for minimally invasive esophagectomy

A 3D Image Registration Method for Laparoscopic Liver Surgery Navigation

At present, laparoscopic augmented reality (AR) navigation has been applied to minimally invasive abdominal surgery, which can help doctors to see the location of blood vessels and tumors in organs, so as to perform precise surgery operations. Image registration is the process of optimally mapping one or more images to the target image, and it is also the core of laparoscopic AR navigation. The key is how to shorten the registration time and optimize the registration accuracy. We have studied the three-dimensional (3D) image registration technology in laparoscopic liver surgery navigation and proposed a new registration method combining rough registration and fine registration. First, the adaptive fireworks algorithm (AFWA) is applied to rough registration, and then the optimized iterative closest point (ICP) algorithm is applied to fine registration. We proposed a method that is validated by the computed tomography (CT) dataset 3D-IRCADb-01. Experimental results show that our method is superior to other registration methods based on stochastic optimization algorithms in terms of registration time and accuracy.

Preliminary study for developing a navigation system for gastric cancer surgery using artificial intelligence

PURPOSE

We are attempting to develop a navigation system for safe and effective peripancreatic lymphadenectomy in gastric cancer surgery. As a preliminary study, we examined whether or not the peripancreatic dissection line could be learned by a machine learning model (MLM).

METHODS

Among the 41 patients with gastric cancer who underwent radical gastrectomy between April 2019 and January 2020, we selected 6 in whom the pancreatic contour was relatively easy to trace. The pancreatic contour was annotated by a trainer surgeon in 1242 images captured from the video recordings. The MLM was trained using the annotated images from five of the six patients. The pancreatic contour was then segmented by the trained MLM using images from the remaining patient. The same procedure was repeated for all six combinations.

RESULTS

The median maximum intersection over union of each image was 0.708, which was higher than the threshold (0.5). However, the pancreatic contour was misidentified in parts where fatty tissue or thin vessels overlaid the pancreas in some cases.

CONCLUSION

The contour of the pancreas could be traced relatively well using the trained MLM. Further investigations and training of the system are needed to develop a practical navigation system.

Artificial intelligence as a future in cancer surgery

Artificial intelligence (AI) is defined as the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, and decision-making. Machine learning and deep learning (DL) are subfields of AI that are able to learn from experience in order to complete tasks. AI and its subfields, in particular DL, have been applied in numerous fields of medicine, especially in the cure of cancer. Computer vision (CV) system has improved diagnostic accuracy both in histopathology analyses and radiology. In surgery, CV has been used to design navigation system and robotic-assisted surgical tools that increased the safety and efficiency of oncological surgery by minimizing human error. By learning the basis of AI, surgeons can take part in this revolution to optimize surgical care of oncologic disease.

Artificial intelligence in perioperative management of major gastrointestinal surgeries

Artificial intelligence (AI) demonstrated by machines is based on reinforcement learning and revolves around the usage of algorithms. The purpose of this review was to summarize concepts, the scope, applications, and limitations in major gastrointestinal surgery. This is a narrative review of the available literature on the key capabilities of AI to help anesthesiologists, surgeons, and other physicians to understand and critically evaluate ongoing and new AI applications in perioperative management. AI uses available databases called “big data” to formulate an algorithm. Analysis of other data based on these algorithms can help in early diagnosis, accurate risk assessment, intraoperative management, automated drug delivery, predicting anesthesia and surgical complications and postoperative outcomes and can thus lead to effective perioperative management as well as to reduce the cost of treatment. Perioperative physicians, anesthesiologists, and surgeons are well-positioned to help integrate AI into modern surgical practice. We all need to partner and collaborate with data scientists to collect and analyze data across all phases of perioperative care to provide clinical scenarios and context. Careful implementation and use of AI along with real-time human interpretation will revolutionize perioperative care, and is the way forward in future perioperative management of major surgery.

Augmented Reality Navigation for Stereoscopic Laparoscopic Anatomical Hepatectomy of Primary Liver Cancer: Preliminary Experience

Background

Accurate determination of intrahepatic anatomy remains challenging for laparoscopic anatomical hepatectomy (LAH). Laparoscopic augmented reality navigation (LARN) is expected to facilitate LAH of primary liver cancer (PLC) by identifying the exact location of tumors and vessels. The study was to evaluate the safety and effectiveness of our independently developed LARN system in LAH of PLC.

Methods

From May 2018 to July 2020, the study included 85 PLC patients who underwent three-dimensional (3D) LAH. According to whether LARN was performed during the operation, the patients were divided into the intraoperative navigation (IN) group and the non-intraoperative navigation (NIN) group. We compared the preoperative data, perioperative results and postoperative complications between the two groups, and introduced our preliminary experience of this novel technology in LAH.

Results

There were 44 and 41 PLC patients in the IN group and the NIN group, respectively. No significant differences were found in preoperative characteristics and any of the resection-related complications between the two groups (All P > 0.05). Compared with the NIN group, the IN group had significantly less operative bleeding (P = 0.002), lower delta Hb% (P = 0.039), lower blood transfusion rate (P < 0.001), and reduced postoperative hospital stay (P = 0.003). For the IN group, the successful fusion of simulated surgical planning and operative scene helped to determine the extent of resection.

Conclusions

The LARN contributed to the identification of important anatomical structures during LAH of PLC. It reduced vascular injury and accelerated postoperative recovery, showing a potential application prospects in liver surgery.

Effects of laparoscopy, laparotomy, and respiratory phase on liver volume in a live porcine model for liver resection

Background

Hepatectomy, living donor liver transplantations and other major hepatic interventions rely on precise calculation of the total, remnant and graft liver volume. However, liver volume might differ between the pre- and intraoperative situation. To model liver volume changes and develop and validate such pre- and intraoperative assistance systems, exact information about the influence of lung ventilation and intraoperative surgical state on liver volume is essential.

Methods

This study assessed the effects of respiratory phase, pneumoperitoneum for laparoscopy, and laparotomy on liver volume in a live porcine model. Nine CT scans were conducted per pig (N = 10), each for all possible combinations of the three operative (native, pneumoperitoneum and laparotomy) and respiratory states (expiration, middle inspiration and deep inspiration). Manual segmentations of the liver were generated and converted to a mesh model, and the corresponding liver volumes were calculated.

Results

With pneumoperitoneum the liver volume decreased on average by 13.2% (112.7 ml ± 63.8 ml, p < 0.0001) and after laparotomy by 7.3% (62.0 ml ± 65.7 ml, p = 0.0001) compared to native state. From expiration to middle inspiration the liver volume increased on average by 4.1% (31.1 ml ± 55.8 ml, p = 0.166) and from expiration to deep inspiration by 7.2% (54.7 ml ± 51.8 ml, p = 0.007).

Conclusions

Considerable changes in liver volume change were caused by pneumoperitoneum, laparotomy and respiration. These findings provide knowledge for the refinement of available preoperative simulation and operation planning and help to adjust preoperative imaging parameters to best suit the intraoperative situation.

Real-Time Multi-Modal Sensing and Feedback for Catheterization in Porcine Tissue

Objective: In this study, we introduce a multi-modal sensing and feedback framework aimed at assisting clinicians during endovascular surgeries and catheterization procedures. This framework utilizes state-of-the-art imaging and sensing sub-systems to produce a 3D visualization of an endovascular catheter and surrounding vasculature without the need for intra-operative X-rays. Methods: The catheterization experiments within this study are conducted inside a porcine limb undergoing motions. A hybrid position-force controller of a robotically-actuated ultrasound (US) transducer for uneven porcine tissue surfaces is introduced. The tissue, vasculature, and catheter are visualized by integrated real-time US images, 3D surface imaging, and Fiber Bragg Grating (FBG) sensors. Results: During externally-induced limb motions, the vasculature and catheter can be reliably reconstructed at mean accuracies of 1.9±0.3 mm and 0.82±0.21 mm, respectively. Conclusions: The conventional use of intra-operative X-ray imaging to visualize instruments and vasculature in the human body can be reduced by employing improved diagnostic technologies that do not operate via ionizing radiation or nephrotoxic contrast agents. Significance: The presented multi-modal framework enables the radiation-free and accurate reconstruction of significant tissues and instruments involved in catheterization procedures.

Mobile, real-time, and point-of-care augmented reality is robust, accurate, and feasible: a prospective pilot study

BACKGROUND

Augmented reality (AR) systems are currently being explored by a broad spectrum of industries, mainly for improving point-of-care access to data and images. Especially in surgery and especially for timely decisions in emergency cases, a fast and comprehensive access to images at the patient bedside is mandatory. Currently, imaging data are accessed at a distance from the patient both in time and space, i.e., at a specific workstation. Mobile technology and 3-dimensional (3D) visualization of radiological imaging data promise to overcome these restrictions by making bedside AR feasible.

METHODS

In this project, AR was realized in a surgical setting by fusing a 3D-representation of structures of interest with live camera images on a tablet computer using marker-based registration. The intent of this study was to focus on a thorough evaluation of AR. Feasibility, robustness, and accuracy were thus evaluated consecutively in a phantom model and a porcine model. Additionally feasibility was evaluated in one male volunteer.

RESULTS

In the phantom model (n = 10), AR visualization was feasible in 84% of the visualization space with high accuracy (mean reprojection error ± standard deviation (SD): 2.8 ± 2.7 mm; 95th percentile = 6.7 mm). In a porcine model (n = 5), AR visualization was feasible in 79% with high accuracy (mean reprojection error ± SD: 3.5 ± 3.0 mm; 95th percentile = 9.5 mm). Furthermore, AR was successfully used and proved feasible within a male volunteer.

CONCLUSIONS

Mobile, real-time, and point-of-care AR for clinical purposes proved feasible, robust, and accurate in the phantom, animal, and single-trial human model shown in this study. Consequently, AR following similar implementation proved robust and accurate enough to be evaluated in clinical trials assessing accuracy, robustness in clinical reality, as well as integration into the clinical workflow. If these further studies prove successful, AR might revolutionize data access at patient bedside.

Computer-assisted abdominal surgery: new technologies

BACKGROUND

Computer-assisted surgery is a wide field of technologies with the potential to enable the surgeon to improve efficiency and efficacy of diagnosis, treatment, and clinical management.

PURPOSE

This review provides an overview of the most important new technologies and their applications.

METHODS

A MEDLINE database search was performed revealing a total of 1702 references. All references were considered for information on six main topics, namely image guidance and navigation, robot-assisted surgery, human-machine interface, surgical processes and clinical pathways, computer-assisted surgical training, and clinical decision support. Further references were obtained through cross-referencing the bibliography cited in each work. Based on their respective field of expertise, the authors chose 64 publications relevant for the purpose of this review.

CONCLUSION

Computer-assisted systems are increasingly used not only in experimental studies but also in clinical studies. Although computer-assisted abdominal surgery is still in its infancy, the number of studies is constantly increasing, and clinical studies start showing the benefits of computers used not only as tools of documentation and accounting but also for directly assisting surgeons during diagnosis and treatment of patients. Further developments in the field of clinical decision support even have the potential of causing a paradigm shift in how patients are diagnosed and treated.

Real-time image guidance in laparoscopic liver surgery: first clinical experience with a guidance system based on intraoperative CT imaging

BACKGROUND

Laparoscopic liver surgery is particularly challenging owing to restricted access, risk of bleeding, and lack of haptic feedback. Navigation systems have the potential to improve information on the exact position of intrahepatic tumors, and thus facilitate oncological resection. This study aims to evaluate the feasibility of a commercially available augmented reality (AR) guidance system employing intraoperative robotic C-arm cone-beam computed tomography (CBCT) for laparoscopic liver surgery.

METHODS

A human liver-like phantom with 16 target fiducials was used to evaluate the Syngo iPilot(®) AR system. Subsequently, the system was used for the laparoscopic resection of a hepatocellular carcinoma in segment 7 of a 50-year-old male patient.

RESULTS

In the phantom experiment, the AR system showed a mean target registration error of 0.96 ± 0.52 mm, with a maximum error of 2.49 mm. The patient successfully underwent the operation and showed no postoperative complications.

CONCLUSION

The use of intraoperative CBCT and AR for laparoscopic liver resection is feasible and could be considered an option for future liver surgery in complex cases.

Magnetic tracking in the operation room using the da Vinci(®) telemanipulator is feasible

In recent years, robotic assistance for surgical procedures has grown on a worldwide scale, particularly for use in more complex operations. Such operations usually require meticulous handling of tissue, involve a narrow working space and limit the surgeon’s sense of orientation in the human body. Improvement in both tissue handling and working within a narrow working space might be achieved through the use of robotic assistance. Soft tissue navigation might improve orientation by visualizing important target and risk structures intraoperatively, thereby possibly improving patient outcome. Prerequisites for navigation are its integration into the surgical workflow and accurate localization of both the instruments and patient. Magnetic tracking allows for good integration but is susceptible to distortion through metal or electro-magnetic interference, which may be caused by the operation table or a robotic system. We have investigated whether magnetic tracking can be used in combination with the da Vinci^® (DV) telemanipulator in terms of stability and precision. We used a common magnetic tracking system (Aurora^®, NDI Inc.) with the DV in a typical operation setup. Magnetic field distortion was evaluated using a measuring facility, with the following reference system: without any metal (R), operation table alone (T), DV in standby (D) and DV in motion (Dm). The maximum error of the entire tracking volume for R, T, D and Dm was 9.9, 32.8, 37.9 and 37.2 mm, respectively. Limiting the tracking volume to 190 mm (from cranial to caudal) resulted in a maximum error of 4.0, 8.3, 8.5 and 8.9 mm, respectively. When used in the operation room, magnetic tracking shows high errors, mainly due to the operation table. The target area should be limited to increase accuracy, which is possible for most surgical applications. The use of the da Vinci^® telemanipulator only slightly aggravates the distortion and can thus be used in combination with magnetic tracking systems.

[Organization of clinical research: in general and visceral surgery]

The structural organization of research facilities within a surgical university center should aim at strengthening the department's research output and likewise provide opportunities for the scientific education of academic surgeons. We suggest a model in which several independent research groups within a surgical department engage in research projects covering various aspects of surgically relevant basic, translational or clinical research. In order to enhance the translational aspects of surgical research, a permanent link needs to be established between the department's scientific research projects and its chief interests in clinical patient care. Importantly, a focus needs to be placed on obtaining evidence-based data to judge the efficacy of novel diagnostic and treatment concepts. Integration of modern technologies from the fields of physics, computer science and molecular medicine into surgical research necessitates cooperation with external research facilities, which can be strengthened by coordinated support programs offered by research funding institutions.