Can Training Make Three Arms Better Than Two Heads for Trimanual Coordination?

Supernumerary effectors have been proposed to enable users to perform tasks alone that normally require assistance from a partner. While various supernumerary robotic limbs have been developed in the last decade, the capability of users to operate them effectively has not yet been proven. Here we tested whether users (i) can complete a task that requires simultaneous and fine control of three effectors, and (ii) can be trained to do so with similar or superior performance as through collaboration with a human partner. As in previous studies, initial augmented capability was less than that of working with a partner. However, one hour of dedicated solo trimanual training across three days significantly increased task performance, so that participants became able to perform trimanual control alone as well as or better than they could with a new partner. This shows the viability of augmentation systems for applications such as in robotic surgery or industrial assembly, which can be further validated on real tasks with physical systems.

A virtual reality platform to evaluate the effects of supernumerary limbs' appearance

Supernumerary robot limbs (SL) can expand the ability of users by increasing the number of degrees of freedom that they control. While several SLs have been designed and tested on human participants, the effect of the limb's appearance on the user's acceptance, embodiment and device usage is not yet understood. We developed a virtual reality platform with a three-arm avatar that enabled us to systematically investigate the effect of the supernumerary limb's appearance on their perception and motion control performance. A pilot study with 14 participants exhibited similar performance, workload and preference in human-like or robot-like appearance with a trend of preference for the robotic appearance.

Redundancy Resolution in Trimanual vs. Bimanual Tracking Tasks

Supernumerary limbs promise to allow users to perform complex tasks that would otherwise require the actions of teams. However, how the user's capability for multimanual coordination compares to bimanual coordination, and how the motor system decides to configure its limb contributions given task redundancy is unclear. We conducted bimanual and trimanual (with the foot as a third-hand controller) virtual reality visuomotor tracking experiments to study how 32 healthy participants changed their limb coordination in response to uninstructed cursor mapping changes. This used a shared cursor mapped to the average limbs' position for different limb combinations. The results show that most participants correctly identified the different mappings during bimanual tracking, and accordingly minimized task-irrelevant motion. Instead during trimanual coordination, participants consistently moved all three limbs concurrently, showing weaker ipsilateral hand-foot coordination. These findings show how redundancy resolution and the resulting coordination patterns differ between similar bimanual and trimanual tasks. Further research is needed to consider the effect of learning on coordination behaviour.

Human Performance of Three Hands in Unimanual, Bimanual and Trimanual Tasks

Trimanual operation using a robotic supernumerary limb is a new and challenging mechanism for human operators that could enable a single user to perform tasks requiring more than two hands. Foot-controlled interfaces have previously proven able to be intuitively controlled, enabling simple tasks to be performed. However, the effect of going from unimanual to bimanual and then to trimanual tasks on subjects performance and coordination is not well understood. In this paper, unimanual, bimanual and trimanual teleoperation tasks were performed in a virtual reality scene to evaluate the impact of extending to trimanual actions. 15 participants were required to move their limbs together in a coordinated reaching activity. The results show that the addition of another hand resulted in an increase in operating time, where the time increased in going from unimanual to bimanual operation and then increased further when going from bimanual to trimanual. Moreover, the success rate for performing bimanual and trimanual tasks was strongly influenced by the subject's performance in ipsilateral hand-foot activities, where the ipsilateral combination had a lower success rate than contralateral limbs. The addition of a hand did not affect any two-hand coordination rate and even in some cases reduced coordination deviations. Clinical relevance - This work can contribute to build efficient training and learning framework on human multiple limbs motion control and coordination for both rehabilitation and augmentation.

Principles of human movement augmentation and the challenges in making it a reality

Augmenting the body with artificial limbs controlled concurrently to one's natural limbs has long appeared in science fiction, but recent technological and neuroscientific advances have begun to make this possible. By allowing individuals to achieve otherwise impossible actions, movement augmentation could revolutionize medical and industrial applications and profoundly change the way humans interact with the environment. Here, we construct a movement augmentation taxonomy through what is augmented and how it is achieved. With this framework, we analyze augmentation that extends the number of degrees-of-freedom, discuss critical features of effective augmentation such as physiological control signals, sensory feedback and learning as well as application scenarios, and propose a vision for the field.

Human performance in three-hands tasks

The successful completion of complex tasks like hanging a picture or laparoscopic surgery requires coordinated motion of more than two limbs. User-controlled supernumerary robotic limbs (SL) have been proposed to bypass the need for coordination with a partner in such tasks. However, neither the capability to control multiple limbs alone relative to collaborative control with partners, nor how that capability varies across different tasks, is well understood. In this work, we present an investigation of tasks requiring three-hands where the foot was used as an additional source of motor commands. We considered: (1) how does simultaneous control of three hands compare to a cooperating dyad; (2) how this relative performance was altered by the existence of constraints emanating from real or virtual physical connections (mechanical constraints) or from cognitive limits (cognitive constraints). It was found that a cooperating dyad outperformed a single user in all scenarios in terms of task score, path efficiency and motion smoothness. However, while the participants were able to reach more targets with increasing mechanical constraints/decreasing number of simultaneous goals, the relative difference in performance between a dyad and a participant performing trimanual activities decreased, suggesting further potential for SLs in this class of scenario.

A Three-Limb Teleoperated Robotic System with Foot Control for Flexible Endoscopic Surgery

Flexible endoscopy requires a lot of skill to manipulate both the endoscope and the associated instruments. In most robotic flexible endoscopic systems, the endoscope and instruments are controlled separately by two operators, which may result in communication errors and inefficient operation. Our solution is to enable the surgeon to control both the endoscope and the instruments. Here, we present a novel tele-operation robotic endoscopic system commanded by one operator using the continuous and simultaneous movements of their two hands and one foot. This 13-degree-of-freedom (DoF) system integrates a foot-controlled robotic flexible endoscope and two hand-controlled robotic endoscopic instruments, a robotic grasper and a robotic cauterizing hook. A dedicated foot-interface transfers the natural foot movements to the 4-DoF movements of the endoscope while two other commercial hand interfaces map the movements of the two hands to the two instruments individually. An ex-vivo experiment was carried out by six subjects without surgical experience, where the simultaneous control with foot and hands was compared with a sequential clutch-based hand control. The participants could successfully teleoperate the endoscope and the two instruments to cut the tissues at scattered target areas in a porcine stomach. Foot control yielded 43.7% faster task completion and required less mental effort as compared to the clutch-based hand control scheme, which proves the concept of three-limb tele-operation surgery and the developed flexible endoscopic system.