Movement Strategy Discovery during Training via Haptic Guidance

Adaptive Surgical Robotic Training Using Real-Time Stylistic Behavior Feedback Through Haptic Cues

Surgical skill directly affects surgical procedure outcomes; thus, effective training is needed to ensure satisfactory results. Many objective assessment metrics have been developed that provide the trainee with descriptive feedback about their performance however, often lack feedback on how to improve performance. The most effective training method is one that is intuitive, easy to understand, personalized to the user,and provided in a timely manner. We propose a framework to enable user-adaptive training using near real-time detection of performance, based on intuitive styles of surgical movements, and design a haptic feedback framework to assist with correcting styles of movement. We evaluate the ability of three types of force feedback (spring, damping, and spring plus damping feedback), computed based on prior user positions, to improve different stylistic behaviors of the user during kinematically constrained reaching movement tasks. The results indicate that five out of six styles studied here were improved using at least one of the three types of force feedback. Task performance metrics were compared in the presence of the three types of feedback. Task time was statistically significantly lower when applying spring feedback, compared to the other two types of feedback. Path straightness and targeting error were statistically significantly improved when using spring-damping feedback compared to the other two types of feedback. This study presents a groundwork for adaptive training in robotic surgery based on near real-time human-centric models of surgical behavior.

Rethinking Autonomous Surgery: Focusing on Enhancement over Autonomy

CONTEXT

As robot-assisted surgery is increasingly used in surgical care, the engineering research effort towards surgical automation has also increased significantly. Automation promises to enhance surgical outcomes, offload mundane or repetitive tasks, and improve workflow. However, we must ask an important question: should autonomous surgery be our long-term goal?

OBJECTIVE

To provide an overview of the engineering requirements for automating control systems, summarize technical challenges in automated robotic surgery, and review sensing and modeling techniques to capture real-time human behaviors for integration into the robotic control loop for enhanced shared or collaborative control.

EVIDENCE ACQUISITION

We performed a nonsystematic search of the English language literature up to March 25, 2021. We included original studies related to automation in robot-assisted laparoscopic surgery and human-centered sensing and modeling.

EVIDENCE SYNTHESIS

We identified four comprehensive review papers that present techniques for automating portions of surgical tasks. Sixteen studies relate to human-centered sensing technologies and 23 to computer vision and/or advanced artificial intelligence or machine learning methods for skill assessment. Twenty-two studies evaluate or review the role of haptic or adaptive guidance during some learning task, with only a few applied to robotic surgery. Finally, only three studies discuss the role of some form of training in patient outcomes and none evaluated the effects of full or semi-autonomy on patient outcomes.

CONCLUSIONS

Rather than focusing on autonomy, which eliminates the surgeon from the loop, research centered on more fully understanding the surgeon's behaviors, goals, and limitations could facilitate a superior class of collaborative surgical robots that could be more effective and intelligent than automation alone.

PATIENT SUMMARY

We reviewed the literature for studies on automation in surgical robotics and on modeling of human behavior in human-machine interaction. The main application is to enhance the ability of surgical robotic systems to collaborate more effectively and intelligently with human surgeon operators.

Haptic Assistance Improves Tele-Manipulation With Two Asymmetric Slaves

Tele-manipulation of heavy loads typically requires the simultaneous use of two asymmetric slaves: a crane for vertical weight support and a robot for accurate lateral positioning. The industrial standard prescribes a pair of operators for such tasks (one operator to control each slave), although in principle one operator might control both slaves with a single, hybrid interface. Accurate and safe co-operative handling of the expensive and fragile heavy components is difficult, presumably due to problems in the coordination of the subtasks and the lack of mutual awareness between the two operators. This study proposes a novel haptic assistance system to improve subtask coordination and task performance. Its novelty consists of haptically linking operators/interfaces through the joint task environment. The system's efficacy is evaluated with 15 pairs of co-operators and 15 individual uni-manual operators who maneuvered a heavy load through a bounded path in Virtual Reality. Haptic assistance improves task completion time for both groups. It also reduces control activity and self-reported workload without affecting a number of critical errors made by the operators. Moreover, without haptic assistance, uni-manual operators perform worse than co-operators, but this difference between the interfaces was not found with haptic assistance.

Haptic Guidance on Demand: A Grip-Force Based Scheduling of Guidance Forces

In haptic shared control systems (HSC), a fixed strength of guidance force equates to a fixed level of control authority, which can be insufficient for complex tasks. An adaptable control authority based on operator input can allow the HSC system to better assist the operator under varied conditions. In this paper, we experimentally investigate () an adaptable authority HSC system that provides the operator with a direct way to adjust the control authority based on applied grip force. This system can serve as an intuitive 'manual override' function in case of HSC system malfunction. In a position tracking task, we explore two opposite approaches to adapt the control authority: increasing versus decreasing guidance strength with operator grip. These approaches were compared with unassisted control and two levels of fixed-level haptic guidance. Results show that the grip-adaptable approach allowed the operators to increase performance over unassisted control and over a weak guidance. At the same time, the approach substantially reduced the operator physical control effort required to cope with HSC system disturbances. Predictions based on the formalized model of the complete human-in-the-loop system corresponded to the experimental results, implying that such validated formalization can be used for model-based analysis and design of guidance systems.