Improving vision for surgeons during laparoscopy: the Enhanced Laparoscopic Vision System (ELViS)

Poor usability of computer-assisted navigation for hip fracture surgery

INTRODUCTION

The STRYKER ADAPT computer-assisted navigation system provides intraoperative feedback to the surgeon regarding implant placement of the Gamma3 nail. The usability of the ADAPT system has not been evaluated. The aim of the study was to investigate the perceived usability of the ADAPT system.

MATERIALS AND METHODS

This was a descriptive study with prospectively collected data. ADAPT was introduced at Aarhus University Hospital in February 2021. Prior to introduction, surgeons at the department attended a general introduction to the system. ADAPT was introduced to the surgical nurses and was on display at the surgical ward at more than one occasion, where personal introduction to the system was possible. After introduction, it was mandatory to use ADAPT when using the Gamma3 nail to treat intertrochanteric femur fractures. After each procedure, primary and an eventual supervisor answered a questionnaire, which encompassed the System Usability Scale (SUS) questionnaire. The SUS is a ten-item questionnaire regarding the perceived usability of a system. SUS scores were translated to adjectives, describing user experience on a 7-point adjective scale (worst imaginable, awful, poor, ok, good, excellent, best imaginable). User acceptability, defined as "not acceptable", "marginal" or "acceptable", was also used to interpret the SUS scores.

RESULTS

ADAPT was used in 50 procedures by 29 different surgeons, with varying skill-level. Median SUS-score after first-time use of ADAPT for all 29 surgeons was 43 (range: 5-60), which translated to "poor" or "not acceptable". For surgeons who performed ≥ 3 ADAPT-assisted procedures, there were no statistically significant difference in their first to latest SUS-score (median difference: 4.3, p = 0.5). In free text comments ADAPT was positively described as helpful in placement of K-wire and providing educational opportunities for inexperienced surgeons and negatively as inconsistent, slow, time consuming, and causing excessive fluoroscopy.

CONCLUSIONS

Usability and acceptability of ADAPT was rated as "poor" or "not acceptable" by the majority of operating surgeons. ADAPT has not been used at our institution based on these findings. The System Usability Scale may be used in further research exploring usability and acceptability of novel computer-assisted navigation systems for orthopaedic surgery.

Development and validation of a deep learning-based laparoscopic system for improving video quality

PURPOSE

A clear surgical field of view is a prerequisite for successful laparoscopic surgery. Surgical smoke, image blur, and lens fogging can affect the clarity of laparoscopic imaging. We aimed to develop a real-time assistance system (namely LVQIS) for removing these interfering factors during laparoscopic surgery, thereby improving laparoscopic video quality.

METHODS

LVQIS was developed with generative adversarial networks (GAN) and transfer learning, which included two classification models (ResNet-50), a motion blur removal model (MPRNet), and a smoke/fog removal model (GAN). 136 laparoscopic surgery videos were retrospectively collected in a tripartite dataset for training and validation. A synthetic dataset was simulated using the image enhancement library Albumentations and the 3D rendering software Blender. The objective evaluation results were through PSNR, SSIM and FID, and the subjective evaluation includes the operation pause time and the degree of anxiety of surgeons.

RESULTS

The synthesized dataset contained 19,245 clear images, 19,245 motion blur images, and 19,245 smoke/fog images. The ResNet-50 CNN model identified whether a single laparoscopic image had motion blur and smoke/fog with an accuracy of over 0.99. The PSNR, SSIM and FID of the de-smoke model were 29.67, 0.9551 and 74.72, respectively, and the PSNR, SSIM and FID of the de-blurring model were 26.78, 0.9020 and 80.10, respectively, which were better than other advanced de-blurring and de-smoke/fog models. In a comparative study of 100 laparoscopic surgeries, the use of LVQIS significantly reduced the operation pause time (P < 0.001) and the anxiety of surgeons (P = 0.004).

CONCLUSIONS

In this study, LVQIS is an efficient and robust system that can improve the quality of laparoscopic video, reduce surgical pause time and the anxiety of surgeons, and has the potential for real-time application in real clinical settings.

A dual-view multi-resolution laparoscope for safer and more efficient minimally invasive surgery

Minimally invasive surgery (MIS) is limited in safety and efficiency by the hand-held nature and narrow fields of view of traditional laparoscopes. A multi-resolution foveated laparoscope (MRFL) was invented to address these concerns. The MRFL is a stationary dual-view imaging device with optical panning and zooming capabilities. It is designed to simultaneously capture and display a zoomed view and supplemental wide view of the surgical field. Optical zooming and panning capabilities facilitate repositioning of the zoomed view without physically moving the system. Additional MRFL features designed to improve safety and efficiency include its snub-nosed endoscope, tool-tip auto tracking, programmable focus profiles, unique selectable display modalities, foot pedal controls, and independently controlled surgeon and assistant displays. An MRFL prototype was constructed to demonstrate and test these features. Testing of the prototype validates its design architecture and confirms the functionality of its features. The current MRFL prototype functions adequately as a proof of concept, but the system features and performance require further improvement to be practical for clinical use.

Qualitative Comparison of Image Stitching Algorithms for Multi-Camera Systems in Laparoscopy

Multi-camera systems were recently introduced into laparoscopy to increase the narrow field of view of the surgeon. The video streams are stitched together to create a panorama that is easier for the surgeon to comprehend. Multi-camera prototypes for laparoscopy use quite basic algorithms and have only been evaluated on simple laparoscopic scenarios. The more recent state-of-the-art algorithms, mainly designed for the smartphone industry, have not yet been evaluated in laparoscopic conditions. We developed a simulated environment to generate a dataset of multi-view images displaying a wide range of laparoscopic situations, which is adaptable to any multi-camera system. We evaluated classical and state-of-the-art image stitching techniques used in non-medical applications on this dataset, including one unsupervised deep learning approach. We show that classical techniques that use global homography fail to provide a clinically satisfactory rendering and that even the most recent techniques, despite providing high quality panorama images in non-medical situations, may suffer from poor alignment or severe distortions in simulated laparoscopic scenarios. We highlight the main advantages and flaws of each algorithm within a laparoscopic context, identify the main remaining challenges that are specific to laparoscopy, and propose methods to improve these approaches. We provide public access to the simulated environment and dataset.