Augmented reality in gynecologic laparoscopic surgery: development, evaluation of accuracy and clinical relevance of a device useful to identify ureters during surgery

Phantom study on surgical performance in augmented reality laparoscopy

Purpose

Only a few studies have evaluated Augmented Reality (AR) in in vivo simulations compared to traditional laparoscopy; further research is especially needed regarding the most effective AR visualization technique. This pilot study aims to determine, under controlled conditions on a 3D-printed phantom, whether an AR laparoscope improves surgical outcomes over conventional laparoscopy without augmentation.

Methods

We selected six surgical residents at a similar level of training and had them perform a laparoscopic task. The participants repeated the experiment three times, using different 3D phantoms and visualizations: Floating AR, Occlusion AR, and without any AR visualization (Control). Surgical performance was determined using objective measurements. Subjective measures, such as task load and potential application areas, were collected with questionnaires.

Results

Differences in operative time, total touching time, and SurgTLX scores showed no statistical significance ($$p>0.05$$ p > 0.05 ). However, when assessing the invasiveness of the simulated intervention, the comparison revealed a statistically significant difference ($$p=0.009$$ p = 0.009 ). Participants felt AR could be useful for various surgeries, especially for liver, sigmoid, and pancreatic resections (100%). Almost all participants agreed that AR could potentially lead to improved surgical parameters, such as operative time (83%), complication rate (83%), and identifying risk structures (83%).

Conclusion

According to our results, AR may have great potential in visceral surgery and based on the objective measures of the study, may improve surgeons' performance in terms of an atraumatic approach. In this pilot study, participants consistently took more time to complete the task, had more contact with the vascular tree, were significantly more invasive, and scored higher on the SurgTLX survey than with AR.

Robotically assisted augmented reality system for identification of targeted lymph nodes in laparoscopic gynecological surgery: a first step toward the identification of sentinel node : Augmented reality in gynecological surgery

BACKGROUND

To prove feasibility of multimodal and temporal fusion of laparoscopic images with preoperative computed tomography scans for a real-time in vivo-targeted lymph node (TLN) detection during minimally invasive pelvic lymphadenectomy and to validate and enable such guidance for safe and accurate sentinel lymph node dissection, including anatomical landmarks in an experimental model.

METHODS

A measurement campaign determined the most accurate tracking system (UR5-Cobot versus NDI Polaris). The subsequent interventions on two pigs consisted of an identification of artificial TLN and anatomical landmarks without and with augmented reality (AR) assistance. The AR overlay on target structures was quantitatively evaluated. The clinical relevance of our system was assessed via a questionnaire completed by experienced and trainee surgeons.

RESULTS

An AR-based robotic assistance system that performed real-time multimodal and temporal fusion of laparoscopic images with preoperative medical images was developed and tested. It enabled the detection of TLN and their surrounding anatomical structures during pelvic lymphadenectomy. Accuracy of the CT overlay was > 90%, with overflow rates < 6%. When comparing AR to direct vision, we found that scores were significatively higher in AR for all target structures. AR aided both experienced surgeons and trainees, whether it was for TLN, ureter, or vessel identification.

CONCLUSION

This computer-assisted system was reliable, safe, and accurate, and the present achievements represent a first step toward a clinical study.

Surgical data science - from concepts toward clinical translation

Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.

A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery

Background: Research proves that the apprenticeship model, which is the gold standard for training surgical residents, is obsolete. For that reason, there is a continuing effort toward the development of high-fidelity surgical simulators to replace the apprenticeship model. Applying Virtual Reality Augmented Reality (AR) and Mixed Reality (MR) in surgical simulators increases the fidelity, level of immersion and overall experience of these simulators.Areas covered: The objective of this review is to provide a comprehensive overview of the application of VR, AR and MR for distinct surgical disciplines, including maxillofacial surgery and neurosurgery. The current developments in these areas, as well as potential future directions, are discussed.Expert opinion: The key components for incorporating VR into surgical simulators are visual and haptic rendering. These components ensure that the user is completely immersed in the virtual environment and can interact in the same way as in the physical world. The key components for the application of AR and MR into surgical simulators include the tracking system as well as the visual rendering. The advantages of these surgical simulators are the ability to perform user evaluations and increase the training frequency of surgical residents.

Concept for Markerless 6D Tracking Employing Volumetric Optical Coherence Tomography

Optical tracking systems are widely used, for example, to navigate medical interventions. Typically, they require the presence of known geometrical structures, the placement of artificial markers, or a prominent texture on the target’s surface. In this work, we propose a 6D tracking approach employing volumetric optical coherence tomography (OCT) images. OCT has a micrometer-scale resolution and employs near-infrared light to penetrate few millimeters into, for example, tissue. Thereby, it provides sub-surface information which we use to track arbitrary targets, even with poorly structured surfaces, without requiring markers. Our proposed system can shift the OCT’s field-of-view in space and uses an adaptive correlation filter to estimate the motion at multiple locations on the target. This allows one to estimate the target’s position and orientation. We show that our approach is able to track translational motion with root-mean-squared errors below 0.25 mm and in-plane rotations with errors below 0.3°. For out-of-plane rotations, our prototypical system can achieve errors around 0.6°.