A novel gaze-controlled flexible robotized endoscope; preliminary trial and report

Multimodal human-computer interaction in interventional radiology and surgery: a systematic literature review

PURPOSE

As technology advances, more research dedicated to medical interactive systems emphasizes the integration of touchless and multimodal interaction (MMI). Particularly in surgical and interventional settings, this approach is advantageous because it maintains sterility and promotes a natural interaction. Past reviews have focused on investigating MMI in terms of technology and interaction with robots. However, none has put particular emphasis on analyzing these kind of interactions for surgical and interventional scenarios.

METHODS

Two databases were included in the query to search for relevant publications within the past 10 years. After identification, two screening steps followed which included eligibility criteria. A forward/backward search was added to identify more relevant publications. The analysis incorporated the clustering of references in terms of addressed medical field, input and output modalities, and challenges regarding the development and evaluation.

RESULTS

A sample of 31 references was obtained (16 journal articles, 15 conference papers). MMI was predominantly developed for laparoscopy and radiology and interaction with image viewers. The majority implemented two input modalities, with voice-hand interaction being the most common combination-voice for discrete and hand for continuous navigation tasks. The application of gaze, body, and facial control is minimal, primarily because of ergonomic concerns. Feedback was included in 81% publications, of which visual cues were most often applied.

CONCLUSION

This work systematically reviews MMI for surgical and interventional scenarios over the past decade. In future research endeavors, we propose an enhanced focus on conducting in-depth analyses of the considered use cases and the application of standardized evaluation methods. Moreover, insights from various sectors, including but not limited to the gaming sector, should be exploited.

Eye-controlled endoscopy - a benchtop trial of a novel robotic steering platform - iGAZE2

The endoscopic control system has remained similar in design for many decades The remit of advanced therapeutic endoscopy continues to expand requiring precision control and high cognitive workloads. Robotic systems are emerging, but all still require bimanual control and expensive and large new systems. Eye tracking is an exciting area that can be used as an endoscope control system. This is a study to establish the feasibility of an eye-controlled endoscope and compare its performance and cognitive demand to use of a conventional endoscope. An eye gaze-control system consisting of eye-tracking glasses, customised software and a small motor unit was built and attached to a conventional endoscope. Twelve non-endoscopists used both the eye gaze system and a conventional endoscope to complete a benchtop task in a simulated oesophagus and stomach. Completion of tasks was timed. Subjective feedback was collected from each participant on task load using the NASA Task Load Index. Participants were significantly quicker completing the task using iGAZE2 vs a conventional endoscope (65.02 ± 16.34s vs 104.21 ± 51.31s, p = 0.013) Participants were also significantly quicker completing retroflexion using iGAZE2 vs a conventional endoscope (8.48 ± 3.08 vs 11.38 ± 5.36s, p = 0.036). Participants reported a significantly lower workload (raw NASA-TLX score) when using iGAZE2 vs the conventional endoscope (152.1 ± 63.4 vs 319.6 ± 81.6, p = 0.0001) (Fig. 7). Users found iGAZE2 to have a significantly lower temporal demand, mental demand, effort, mental demand, physical demand, and frustration level. The eye gaze system is an exciting, small, and retrofittable system to any endoscope. The system shows exciting potential as a novel endoscopic control system with a significantly lower workload and better performance in novices suggesting a more intuitive control system.

A Framework for the Evaluation of Human Machine Interfaces of Robot-Assisted Colonoscopy

The Human Machine Interface (HMI) of intraluminal robots has a crucial impact on the clinician's performance. It increases or decreases the difficulty of the tasks, and is connected to the users' physical and mental stress.

OBJECTIVE

This article presents a framework to compare and evaluate different HMIs for robotic colonoscopy, with the objective of identifying the optimal HMI that minimises the clinician's effort and maximises the clinical outcomes.

METHODS

The framework comprises a 1) a virtual simulator (clinically validated), 2) wearable sensors measuring the cognitive load, 3) a data collection unit of metrics correlated to the clinical performance, and 4) questionnaires exploring the users' impressions and perceived stress. The framework was tested with 42 clinicians investigating the optimal device for tele-operated control of robotic colonoscopes. Two control devices were selected and compared: a haptic serial-kinematic device and a standard videogame joypad.

RESULTS

The haptic device was preferred by the endoscopists, but the joypad enabled better clinical performance and reduced cognitive and physical load.

CONCLUSION

The framework can be used to evaluate different aspects of a HMI, both hardware and software, and determine the optimal HMI that can reduce the burden on clinicians while improving the clinical outcome.

SIGNIFICANCE

The findings of this study, and of future studies performed with this framework, can inform the design and development of HMIs for intraluminal robots, leading to improved clinical performance, reduced physical and mental stress for clinicians, and ultimately better patient outcomes.

A generic scope actuation system for flexible endoscopes

BACKGROUND

A scope actuation system assists a surgeon in steering a scope for navigating an operative field during an interventional or diagnostic procedure. Each system is tailored for a specific surgical procedure. The development of a generic scope actuation system could assist various laparoscopic and endoscopic procedures. This has the potential to reduce the deployment and maintenance costs for a hospital, making it more accessible for clinical usage.

METHODS

A modular actuation system (for maneuvering rigid laparoscopes) was adapted to enable incorporation of flexible endoscopes. The design simplifies the installation and disassembly processes. User studies were conducted to assess the ability of the system to focus onto a diagnostic area, and to navigate during a simulated esophagogastroduodenoscopy procedure. During the studies, the endoscope was maneuvered with (robotic mode) and without (manual mode) the actuation system to navigate the endoscope's focus on a predefined track.

RESULTS

Results show that the robotic mode performed better than the manual mode on all the measured performance parameters including (a) the total duration to traverse a track, (b) the percentage of time spent outside a track while traversing, and (c) the number of times the scope focus shifts outside the track. Additionally, robotic mode also reduced the perceived workload based on the NASA-TLX scale.

CONCLUSIONS

The proposed scope actuation system enhances the maneuverability of flexible endoscopes. It also lays the groundwork for future development of modular and generic scope assistant systems that can be used in both laparoscopic and endoscopic procedures.

Eye-tracking in surgery: a systematic review

BACKGROUND

Surgery is constantly evolving with the assistance of rapidly developing novel technology. Eye-tracking devices provide opportunities to monitor the acquisition of surgical skills, gain insight into performance, and enhance surgical practice. The aim of this review was to consolidate the available evidence for the use of eye-tracking in the surgical disciplines.

METHODS

A systematic literature review was conducted in accordance with PRISMA guidelines. A search of OVID Medline, EMBASE, Cochrane library, Scopus, and Science Direct was conducted January 2000 until December 2022. Studies involving eye-tracking in surgical training, assessment and technical innovation were included in the review. Non-surgical procedures, animal studies, and studies not involving surgical participants were excluded from the review.

RESULTS

The search returned a total of 12 054 articles, 80 of which were included in the final analysis and review. Seventeen studies involved eye-tracking in surgical training, 48 surgical assessment, and 20 were focussing on technical aspects of this technology. Twenty-six different eye-tracking devices were used in the included studies. Metrics such as the number of fixations, duration of fixations, dwell time, and cognitive workload were able to differentiate between novice and expert performance. Eight studies demonstrated the effectiveness of gaze-training for improving surgical skill.

CONCLUSION

The current literature shows a broad range of utility for a variety of eye-tracking devices in surgery. There remains a lack of standardization for metric parameters and gaze analysis techniques. Further research is required to validate its use to establish reliability and create uniform practices.

User interfaces for actuated scope maneuvering in surgical systems: a scoping review

Background

A variety of human computer interfaces are used by robotic surgical systems to control and actuate camera scopes during minimally invasive surgery. The purpose of this review is to examine the different user interfaces used in both commercial systems and research prototypes.

Methods

A comprehensive scoping review of scientific literature was conducted using PubMed and IEEE Xplore databases to identify user interfaces used in commercial products and research prototypes of robotic surgical systems and robotic scope holders. Papers related to actuated scopes with human–computer interfaces were included. Several aspects of user interfaces for scope manipulation in commercial and research systems were reviewed.

Results

Scope assistance was classified into robotic surgical systems (for multiple port, single port, and natural orifice) and robotic scope holders (for rigid, articulated, and flexible endoscopes). Benefits and drawbacks of control by different user interfaces such as foot, hand, voice, head, eye, and tool tracking were outlined. In the review, it was observed that hand control, with its familiarity and intuitiveness, is the most used interface in commercially available systems. Control by foot, head tracking, and tool tracking are increasingly used to address limitations, such as interruptions to surgical workflow, caused by using a hand interface.

Conclusion

Integrating a combination of different user interfaces for scope manipulation may provide maximum benefit for the surgeons. However, smooth transition between interfaces might pose a challenge while combining controls.

Eye tracking technology in endoscopy: Looking to the future

The visual patterns of an endoscopist, that is, what the endoscopist is looking at during luminal endoscopy, is an interesting area with an evolving evidence base. The tools required for gaze analysis have become cheaper and more easily accessible. A comprehensive literature search was undertaken identifying 19 relevant papers. Gaze analysis has been used to identify certain visual patterns associated with higher polyp detection rates. There have also been increasing applications of gaze analysis as an objective study tool to compare the effectiveness of endoscopic imaging technologies. Gaze analysis also has the potential to be incorporated into endoscopic training. Eye movements have been used to control and steer a robotic endoscope. This review presents the current evidence available in this novel and evolving field of endoscopic research.

Eye Tracking and Motion Data Predict Endoscopic Sinus Surgery Skill

OBJECTIVE

Endoscopic surgery has a considerable learning curve due to dissociation of the visual-motor axes, coupled with decreased tactile feedback and mobility. In particular, endoscopic sinus surgery (ESS) lacks objective skill assessment metrics to provide specific feedback to trainees. This study aims to identify summary metrics from eye tracking, endoscope motion, and tool motion to objectively assess surgeons' ESS skill.

METHODS

In this cross-sectional study, expert and novice surgeons performed ESS tasks of inserting an endoscope and tool into a cadaveric nose, touching an anatomical landmark, and withdrawing the endoscope and tool out of the nose. Tool and endoscope motion were collected using an electromagnetic tracker, and eye gaze was tracked using an infrared camera. Three expert surgeons provided binary assessments of low/high skill. 20 summary statistics were calculated for eye, tool, and endoscope motion and used in logistic regression models to predict surgical skill.

RESULTS

14 metrics (10 eye gaze, 2 tool motion, and 2 endoscope motion) were significantly different between surgeons with low and high skill. Models to predict skill for 6/9 ESS tasks had an AUC >0.95. A combined model of all tasks (AUC 0.95, PPV 0.93, NPV 0.89) included metrics from eye tracking data and endoscope motion, indicating that these metrics are transferable across tasks.

CONCLUSIONS

Eye gaze, endoscope, and tool motion data can provide an objective and accurate measurement of ESS surgical performance. Incorporation of these algorithmic techniques intraoperatively could allow for automated skill assessment for trainees learning endoscopic surgery.

LEVEL OF EVIDENCE

N/A Laryngoscope, 133:500-505, 2023.

Scope actuation system for articulated laparoscopes

BACKGROUND

An articulated laparoscope comprises a rigid shaft with an articulated distal end to change the viewing direction. The articulation provides improved navigation of the operating field in confined spaces. Furthermore, incorporation of an actuation system tends to enhance the control of an articulated laparoscope.

METHODS

A preliminary prototype of a scope actuation system to maneuver an off-the-shelf articulated laparoscope (EndoCAMaleon by Karl Storz, Germany) was developed. A user study was conducted to evaluate this prototype for the surgical paradigm of video-assisted thoracic surgery. In the study, the subjects maneuvered an articulated scope under two modes of operation: (a) actuated mode where an operating surgeon maneuvers the scope using the developed prototype and (b) manual mode where a surgical assistant directly maneuvers the scope. The actuated mode was further assessed for multiple configurations based on the orientation of the articulated scope at the incision.

RESULTS

The data show the actuated mode scored better than the manual mode on all the measured performance parameters including (a) total duration to visualize a marked region, (a) duration for which scope focus shifts outside a predefined visualization region, and (c) number of times for which scope focus shifts outside a predefined visualization region. Among the different configurations tested using the actuated mode, no significant difference was observed.

CONCLUSIONS

The proposed articulated scope actuation system facilitates better navigation of an operative field as compared to a human assistant. Secondly, irrespective of the orientation in which an articulated scope's shaft is inserted through an incision, the proposed actuation system can navigate and visualize the operative field.

Perioperative Nursing of Vitrectomy for Ocular Trauma under the Guidance of Ophthalmoscope

To explore the perioperative nursing methods and clinical effects of vitrectomy under ophthalmoscope in the treatment of severe ocular rupture, this study reviews the clinical effects of vitrectomy in the treatment of severe ocular trauma in China, analyzes the perioperative nursing of vitrectomy for ocular trauma under the guidance of an ophthalmoscope, and analyzes the importance of stage I debridement and suture, the choice of operation time, and the advantages of vitrectomy and perioperative nursing care. The retrospective analysis of these data shows that the clinical effect of peri-vitrectomy for ocular trauma under the guidance of an ophthalmoscope is accurate, and surgical treatment should be carried out as soon as possible according to the patient's condition, which can reduce the complications of suppurative ophthalmitis, eyeball atrophy, and vitreous rebleeding. Perioperative nursing intervention is beneficial to the recovery of visual acuity in patients with severe ocular rupture treated by vitrectomy under the ophthalmoscope, which is worthy of clinical promotion.