Towards real time 2D to 3D registration for ultrasound-guided endoscopic and laparoscopic procedures

CBCT-based navigation system for open liver surgery: Accurate guidance toward mobile and deformable targets with a semi-rigid organ approximation and electromagnetic tracking of the liver

Purpose

The surgical navigation system that provides guidance throughout the surgery can facilitate safer and more radical liver resections, but such a system should also be able to handle organ motion. This work investigates the accuracy of intraoperative surgical guidance during open liver resection, with a semi‐rigid organ approximation and electromagnetic tracking of the target area.

Methods

The suggested navigation technique incorporates a preoperative 3D liver model based on diagnostic 4D MRI scan, intraoperative contrast‐enhanced CBCT imaging and electromagnetic (EM) tracking of the liver surface, as well as surgical instruments, by means of six degrees‐of‐freedom micro‐EM sensors.

Results

The system was evaluated during surgeries with 35 patients and resulted in an accurate and intuitive real‐time visualization of liver anatomy and tumor's location, confirmed by intraoperative checks on visible anatomical landmarks. Based on accuracy measurements verified by intraoperative CBCT, the system’s average accuracy was 4.0 ± 3.0 mm, while the total surgical delay due to navigation stayed below 20 min.

Conclusions

The electromagnetic navigation system for open liver surgery developed in this work allows for accurate localization of liver lesions and critical anatomical structures surrounding the resection area, even when the liver was manipulated. However, further clinically integrating the method requires shortening the guidance‐related surgical delay, which can be achieved by shifting to faster intraoperative imaging like ultrasound. Our approach is adaptable to navigation on other mobile and deformable organs, and therefore may benefit various clinical applications.

Augmented Reality Guided Needle Biopsy of Soft Tissue: A Pilot Study

Percutaneous biopsies are popular for extracting suspicious tissue formations (primarily for cancer diagnosis purposes) due to the: relatively low cost, minimal invasiveness, quick procedure times, and low risk for the patient. Despite the advantages provided by percutaneous biopsies, poor needle and tumor visualization is a problem that can result in the clinicians classifying the tumor as benign when it was malignant (false negative). The system developed by the authors aims to address the concern of poor needle and tumor visualization through two virtualization setups. This system is designed to track and visualize the needle and tumor in three-dimensional space using an electromagnetic tracking system. User trials were conducted in which the 10 participants, who were not medically trained, performed a total of 6 tests, each guiding the biopsy needle to the desired location. The users guided the biopsy needle to the desired point on an artificial spherical tumor (diameters of 30, 20, and 10 mm) using the 3D augmented reality (AR) overlay for three trials and a projection on a second monitor (TV) for the other three trials. From the randomized trials, it was found that the participants were able to guide the needle tip 6.5 ± 3.3 mm away from the desired position with an angle deviation of 1.96 ± 1.10° in the AR trials, compared to values of 4.5 ± 2.3 mm and 2.70 ± 1.67° in the TV trials. The results indicate that for simple stationary surgical procedures, an AR display is non-inferior a TV display.

Clinical Study of Three Dimensional Endoscopic Technique in the Treatment of Thyroid Neoplasm

OBJECTIVE

To evaluate the safety and efficacy of three-dimensional (3D) endoscopic technique in clinical practice.

METHODS

A retrospective analysis of the clinical data of 180 cases of endoscopic thyroid surgery, 88 cases by umbrella 3D endoscopic technique (observation group), 92 cases with common endoscopic technique (control group), comparing two groups of patients with age, surgery related indicators, postoperative complications, recovery condition and others.

RESULTS

There was no significant difference in the average age, tumor diameter, intraoperative bleeding, postoperative drainage, hospitalization time, and pain scores after surgery between the to groups (P > .05); the operation time of the observation group was shorter than the control group [(93.8 + 33.7) minutes versus (135.5 + 37.5) minutes, P < .05]. There was no hoarseness in the observation group, and 2 cases in the control group had transient hoarseness and no permanent hoarseness.

CONCLUSION

The surgical application of 3D endoscopic technique can shorten the time of endoscopic thyroid surgery, reduce the risk of recurrent laryngeal nerve injury, reduce the operation difficulty and operation pressure, greatly reduce the risk of surgical patients, and increase the safety of surgery.

Virtual and Augmented Reality Systems for Renal Interventions: A Systematic Review

PURPOSE

Many virtual and augmented reality systems have been proposed to support renal interventions. This paper reviews such systems employed in the treatment of renal cell carcinoma and renal stones.

METHODS

A systematic literature search was performed. Inclusion criteria were virtual and augmented reality systems for radical or partial nephrectomy and renal stone treatment, excluding systems solely developed or evaluated for training purposes.

RESULTS

In total, 52 research papers were identified and analyzed. Most of the identified literature (87%) deals with systems for renal cell carcinoma treatment. About 44% of the systems have already been employed in clinical practice, but only 20% in studies with ten or more patients. Main challenges remaining for future research include the consideration of organ movement and deformation, human factor issues, and the conduction of large clinical studies.

CONCLUSION

Augmented and virtual reality systems have the potential to improve safety and outcomes of renal interventions. In the last ten years, many technical advances have led to more sophisticated systems, which are already applied in clinical practice. Further research is required to cope with current limitations of virtual and augmented reality assistance in clinical environments.

Continuous lung region segmentation from endoscopic images for intra-operative navigation

Although preoperative Computed tomography images are widely used in intraoperative navigation, they can not provide precise information for organs such as the lungs, which deform severely during surgery because of deflation. By segmenting lung regions using intraoperative endoscopic images, a more accurate navigation can be obtained because endoscopic images directly provide real-time organ descriptions. However, satisfactory segmentation is rarely achieved with the algorithms in the literature due to the high deformability of the lungs and similarity between the background and object. This article addresses these problems by describing a novel approach for lung region segmentation based on endoscopic images. The proposed method leverages both GrabCut and optical flow for continuous segmentation. It also introduces a novel technique for quick user interaction, in which users are required to quickly provide a rough curve that shows the possible area of the boundary, and then a much more precise segmentation is deduced based on the rough curve. The effectiveness of the proposed approach was demonstrated by comparing it with conventional algorithms. The results show that the average F-measure of the proposed method is more than 97%. The position, size, and boundary of the lungs obtained by the proposed method can provide useful intraoperative navigation for lung resection surgeries.

Simultaneous Multi-Structure Segmentation and 3D Nonrigid Pose Estimation in Image-Guided Robotic Surgery

In image-guided robotic surgery, segmenting the endoscopic video stream into meaningful parts provides important contextual information that surgeons can exploit to enhance their perception of the surgical scene. This information provides surgeons with real-time decision-making guidance before initiating critical tasks such as tissue cutting. Segmenting endoscopic video is a challenging problem due to a variety of complications including significant noise attributed to bleeding and smoke from cutting, poor appearance contrast between different tissue types, occluding surgical tools, and limited visibility of the objects' geometries on the projected camera views. In this paper, we propose a multi-modal approach to segmentation where preoperative 3D computed tomography scans and intraoperative stereo-endoscopic video data are jointly analyzed. The idea is to segment multiple poorly visible structures in the stereo/multichannel endoscopic videos by fusing reliable prior knowledge captured from the preoperative 3D scans. More specifically, we estimate and track the pose of the preoperative models in 3D and consider the models' non-rigid deformations to match with corresponding visual cues in multi-channel endoscopic video and segment the objects of interest. Further, contrary to most augmented reality frameworks in endoscopic surgery that assume known camera parameters, an assumption that is often violated during surgery due to non-optimal camera calibration and changes in camera focus/zoom, our method embeds these parameters into the optimization hence correcting the calibration parameters within the segmentation process. We evaluate our technique on synthetic data, ex vivo lamb kidney datasets, and in vivo clinical partial nephrectomy surgery with results demonstrating high accuracy and robustness.

Real-time image fusion involving diagnostic ultrasound

OBJECTIVE

The aim of our article is to give an overview of the current and future possibilities of real-time image fusion involving ultrasound. We present a review of the existing English-language peer-reviewed literature assessing this technique, which covers technical solutions (for ultrasound and endoscopic ultrasound), image fusion in several anatomic regions, and electromagnetic needle tracking.

CONCLUSION

The recent progress of real-time ultrasound in image fusion may provide several new possibilities, including diagnosis, treatment, and follow-up of oncologic patients.

Beyond conventional endoscopic ultrasound: elastography, contrast enhancement and hybrid techniques

PURPOSE OF REVIEW

Endoscopic ultrasound (EUS) recently became a technique with a major clinical impact in digestive diseases. EUS determines a change in the diagnosis and management of more than half of examined patients. This review summarizes recent advances in the complementary EUS examination modalities such as elastography and microbubble contrast enhancement.

RECENT FINDINGS

EUS elastography is a well documented method that allows characterization and differentiation of pancreatic cancer and chronic pancreatitis. Quantitative elastography methods, especially based on computer analyses, retrieve numeric values and possibly eliminate the human bias. The use of contrast-enhanced EUS also allows a better visualization and differentiation of focal pancreatic lesions. A hypoenhanced mass as compared with the surrounding pancreatic parenchyma is highly suggestive for pancreatic adenocarcinoma, whereas a hyperenhanced lesion indicates an inflammatory mass. Furthermore, hybrid EUS imaging techniques (in combination with computed tomography or magnetic resonance) might be useful for an increased diagnostic confidence.

SUMMARY

Despite its advantages in assessing the organs situated near the gastrointestinal tract, EUS is still an operator-dependent technique. The new EUS examination modalities incorporated in modern ultrasound systems allow a highly accurate diagnosis.

The role of a computed tomography-based image registered navigation system for natural orifice transluminal endoscopic surgery: a comparative study in a porcine model

BACKGROUND AND STUDY AIMS

Most natural orifice transluminal endoscopic surgery (NOTES) procedures have been performed in animal models through the anterior stomach wall, but this approach does not provide efficient access to all anatomic areas of interest. Moreover, injury of the adjacent structures has been reported when using a blind access. The aim of the current study was to assess the utility of a CT-based (CT: computed tomography) image registered navigation system in identifying safe gastrointestinal access sites for NOTES and identifying intraperitoneal structures.

METHODS

A total of 30 access procedures were performed in 30 pigs: anterior gastric wall (n = 10), posterior gastric wall (n = 10), and anterior rectal wall (n = 10). Of these, 15 procedures used image registered guidance (IR-NOTES) and 15 procedures used a blind access (NOTES only). Timed abdominal exploration was performed with identification of 11 organs. The location of the endoscopic tip was tracked using an electromagnetic tracking system and was recorded for each case. Necropsy was performed immediately after the procedure. The primary outcome was the rate of complications; secondary outcome variables were number of organs identified and kinematic measurements.

RESULTS

A total of 30 animals weighting a mean (± SD) of 30.2 ± 6.8 kg were included in the study. The incision point was correctly placed in 11 out of 15 animals in each group (73.3 %). The mean peritoneoscopy time and the number of properly identified organs were equivalent in the two groups. There were eight minor complications (26.7 %), two (13.3 %) in the IR-NOTES group and six (40.0 %) in the NOTES only group ( P = n. s.). Characteristics of the endoscope tip path showed a statistically significant improvement in trajectory smoothness of motion for all organs in the IR-NOTES group.

CONCLUSION

The image registered system appears to be feasible in NOTES procedures and results from this study suggest that image registered guidance might be useful for supporting navigation with an increased smoothness of motion.

An algorithm for calculi segmentation on ureteroscopic images

PURPOSE

The purpose of the study is to develop an algorithm for the segmentation of renal calculi on ureteroscopic images. In fact, renal calculi are common source of urological obstruction, and laser lithotripsy during ureteroscopy is a possible therapy. A laser-based system to sweep the calculus surface and vaporize it was developed to automate a very tedious manual task. The distal tip of the ureteroscope is directed using image guidance, and this operation is not possible without an efficient segmentation of renal calculi on the ureteroscopic images.

METHODS

We proposed and developed a region growing algorithm to segment renal calculi on ureteroscopic images. Using real video images to compute ground truth and compare our segmentation with a reference segmentation, we computed statistics on different image metrics, such as Precision, Recall, and Yasnoff Measure, for comparison with ground truth.

RESULTS

The algorithm and its parameters were established for the most likely clinical scenarii. The segmentation results are encouraging: the developed algorithm was able to correctly detect more than 90% of the surface of the calculi, according to an expert observer.

CONCLUSION

Implementation of an algorithm for the segmentation of calculi on ureteroscopic images is feasible. The next step is the integration of our algorithm in the command scheme of a motorized system to build a complete operating prototype.

Biomedical and health informatics for surgery

The application of biomedical and health informatics to surgery holds tremendous opportunities to enhance surgical care. Better use of information in surgical practice has the potential to streamline care, remove inefficiencies, and allow for improvements in surgical research. With greater EHR adoption, health care reform, and direct investment in HIT, an increasing opportunity exists for surgeons to access and use patient information more effectively. For this to happen, greater focus on the specific needs of surgeons is particularly important, alongside increasing the number of surgical informatics stakeholders.