On neuromechanical approaches for the study of biological and robotic grasp and manipulation

Full Hill-type muscle model of the I1/I3 retractor muscle complex in Aplysia californica

The coordination of complex behavior requires knowledge of both neural dynamics and the mechanics of the periphery. The feeding system of Aplysia californica is an excellent model for investigating questions in soft body systems' neuromechanics because of its experimental tractability. Prior work has attempted to elucidate the mechanical properties of the periphery by using a Hill-type muscle model to characterize the force generation capabilities of the key protractor muscle responsible for moving Aplysia's grasper anteriorly, the I2 muscle. However, the I1/I3 muscle, which is the main driver of retractions of Aplysia's grasper, has not been characterized. Because of the importance of the musculature's properties in generating functional behavior, understanding the properties of muscles like the I1/I3 complex may help to create more realistic simulations of the feeding behavior of Aplysia, which can aid in greater understanding of the neuromechanics of soft-bodied systems. To bridge this gap, in this work, the I1/I3 muscle complex was characterized using force-frequency, length-tension, and force-velocity experiments and showed that a Hill-type model can accurately predict its force-generation properties. Furthermore, the muscle's peak isometric force and stiffness were found to exceed those of the I2 muscle, and these results were analyzed in the context of prior studies on the I1/I3 complex's kinematics in vivo.

NSF DARE-Transforming modeling in neurorehabilitation: Four threads for catalyzing progress

We present an overview of the Conference on Transformative Opportunities for Modeling in Neurorehabilitation held in March 2023. It was supported by the Disability and Rehabilitation Engineering (DARE) program from the National Science Foundation's Engineering Biology and Health Cluster. The conference brought together experts and trainees from around the world to discuss critical questions, challenges, and opportunities at the intersection of computational modeling and neurorehabilitation to understand, optimize, and improve clinical translation of neurorehabilitation. We organized the conference around four key, relevant, and promising Focus Areas for modeling: Adaptation & Plasticity, Personalization, Human-Device Interactions, and Modeling 'In-the-Wild'. We identified four common threads across the Focus Areas that, if addressed, can catalyze progress in the short, medium, and long terms. These were: (i) the need to capture and curate appropriate and useful data necessary to develop, validate, and deploy useful computational models (ii) the need to create multi-scale models that span the personalization spectrum from individuals to populations, and from cellular to behavioral levels (iii) the need for algorithms that extract as much information from available data, while requiring as little data as possible from each client (iv) the insistence on leveraging readily available sensors and data systems to push model-driven treatments from the lab, and into the clinic, home, workplace, and community. The conference archive can be found at (dare2023.usc.edu). These topics are also extended by three perspective papers prepared by trainees and junior faculty, clinician researchers, and federal funding agency representatives who attended the conference.

Edge Computing in Nature: Minimal pre-processing of multi-muscle ensembles of spindle signals improves discriminability of limb movements

Multiple proprioceptive signals, like those from muscle spindles, are thought to enable robust estimates of body configuration. Yet, it remains unknown whether spindle signals suffice to discriminate limb movements. Here, a simulated 4-musculotendon, 2-joint planar limb model produced repeated cycles of five end-point trajectories in forward and reverse directions, which generated spindle Ia and II afferent signals (proprioceptors for velocity and length, respectively) from each musculotendon. We find that cross-correlation of the 8D time series of raw firing rates (four Ia, four II) cannot discriminate among most movement pairs (∼ 29% accuracy). However, projecting these signals onto their 1^st and 2^nd principal components greatly improves discriminability of movement pairs (82% accuracy). We conclude that high-dimensional ensembles of muscle proprioceptors can discriminate among limb movements-but only after dimensionality reduction. This may explain the pre-processing of some afferent signals before arriving at the somatosensory cortex, such as processing of cutaneous signals at the cat's cuneate nucleus.

Human-like Endtip Stiffness Modulation Inspires Dexterous Manipulation with Robotic Hands

We present a novel method for biomechanically inspired mechanical and control design by quantifying stable manipulation regions in 3D space for tendon-driven systems. Using this method, we present an analysis of the stiffness properties for a human-like index finger and thumb. Although some studies have previously evaluated biomechanical stiffness for grasping and manipulation, no prior works have evaluated the effect of anatomical stiffness parameters throughout the reachable workspace of the index finger or thumb. The passive stiffness model of biomechanically accurate tendon-driven human-like fingers enables analysis of conservatively passive stable regions. The passive stiffness model of the index finger shows that the greatest stiffness ellipsoid volume is aligned to efficiently oppose the anatomical thumb. The thumb model reveals that the greatest stiffness aligns with abduction/adduction near the index finger and shifts to align with the flexion axes for more efficient opposition of the ring or little fingers. Based on these models, biomechanically inspired stiffness controllers that efficiently utilize the underlying stiffness properties while maximizing task criteria can be developed. Trajectory tracking tasks are experimentally tested on the index finger to show the effect of stiffness and stability boundaries on performance.

Hierarchical Human-Inspired Control Strategies for Prosthetic Hands

The abilities of the human hand have always fascinated people, and many studies have been devoted to describing and understanding a mechanism so perfect and important for human activities. Hand loss can significantly affect the level of autonomy and the capability of performing the activities of daily life. Although the technological improvements have led to the development of mechanically advanced commercial prostheses, the control strategies are rather simple (proportional or on/off control). The use of these commercial systems is unnatural and not intuitive, and therefore frequently abandoned by amputees. The components of an active prosthetic hand are the mechatronic device, the decoding system of human biological signals into gestures and the control law that translates all the inputs into desired movements. The real challenge is the development of a control law replacing human hand functions. This paper presents a literature review of the control strategies of prosthetics hands with a multiple-layer or hierarchical structure, and points out the main critical aspects of the current solutions, in terms of human's functions replicated with the prosthetic device. The paper finally provides several suggestions for designing a control strategy able to mimic the functions of the human hand.

Playing the piano with a robotic third thumb: assessing constraints of human augmentation

Contemporary robotics gives us mechatronic capabilities for augmenting human bodies with extra limbs. However, how our motor control capabilities pose limits on such augmentation is an open question. We developed a Supernumerary Robotic 3rd Thumbs (SR3T) with two degrees-of-freedom controlled by the user’s body to endow them with an extra contralateral thumb on the hand. We demonstrate that a pianist can learn to play the piano with 11 fingers within an hour. We then evaluate 6 naïve and 6 experienced piano players in their prior motor coordination and their capability in piano playing with the robotic augmentation. We show that individuals’ augmented performance with the SR3T could be explained by our new custom motor coordination assessment, the Human Augmentation Motor Coordination Assessment (HAMCA) performed pre-augmentation. Our work demonstrates how supernumerary robotics can augment humans in skilled tasks and that individual differences in their augmentation capability are explainable by their individual motor coordination abilities.

Linking Individual Movements to a Skilled Repertoire: Fast Modulation of Motor Synergies by Repetition of Stereotyped Movements

Motor skills emerge when practicing individual movements enables the motor system to extract building instructions that facilitate the generation of future diverse movements. Here we asked how practicing stereotyped movements for minutes affects motor synergies that encode human motor skills acquired over years of training. Participants trained a kinematically highly constrained combined index-finger and thumb movement. Before and after training, finger movements were evoked at rest by transcranial magnetic stimulation (TMS). Post-training, the angle between posture vectors describing TMS-evoked movements and the training movements temporarily decreased, suggesting the presence of a short-term memory for the trained movement. Principal component analysis was used to identify joint covariance patterns in TMS-evoked movements. The quality of reconstruction of training or grasping movements from linear combinations of a small subset of these TMS-derived synergies was used as an index of neural efficiency of movement generation. The reconstruction quality increased for the trained movement but remained constant for grasping movements. These findings suggest that the motor system rapidly reorganizes to enhance the coding efficiency of a difficult movement without compromising the coding efficiency of overlearned movements. Practice of individual movements may drive an unsupervised bottom-up process that ultimately shapes synergistic neuronal organization by constant competition of action memories.

A review of the neurobiomechanical processes underlying secure gripping in object manipulation

O'SHEA, H. and S. J. Redmond. A review of the neurobiomechanical processes underlying secure gripping in object manipulation. NEUROSCI BIOBEHAV REV 286-300, 2021. Humans display skilful control over the objects they manipulate, so much so that biomimetic systems have yet to emulate this remarkable behaviour. Two key control processes are assumed to facilitate such dexterity: predictive cognitive-motor processes that guide manipulation procedures by anticipating action outcomes; and reactive sensorimotor processes that provide important error-based information for movement adaptation. Notwithstanding increased interdisciplinary research interest in object manipulation behaviour, the complexity of the perceptual-sensorimotor-cognitive processes involved and the theoretical divide regarding the fundamentality of control mean that the essential mechanisms underlying manipulative action remain undetermined. In this paper, following a detailed discussion of the theoretical and empirical bases for understanding human dexterous movement, we emphasise the role of tactile-related sensory events in secure object handling, and consider the contribution of certain biophysical and biomechanical phenomena. We aim to provide an integrated account of the current state-of-art in skilled human-object interaction that bridges the literature in neuroscience, cognitive psychology, and biophysics. We also propose novel directions for future research exploration in this area.

The grand challenges of Science Robotics

One of the ambitions of Science Robotics is to deeply root robotics research in science while developing novel robotic platforms that will enable new scientific discoveries. Of our 10 grand challenges, the first 7 represent underpinning technologies that have a wider impact on all application areas of robotics. For the next two challenges, we have included social robotics and medical robotics as application-specific areas of development to highlight the substantial societal and health impacts that they will bring. Finally, the last challenge is related to responsible innovation and how ethics and security should be carefully considered as we develop the technology further.

Non-invasive analysis of motor neurons controlling the intrinsic and extrinsic muscles of the hand

OBJECTIVE

We present a non-invasive framework for investigating efferent commands to 14 extrinsic and intrinsic hand muscles. We extend previous studies (limited to a few muscles) on common synaptic input among pools of motor neurons in a large number of muscles.

APPROACH

Seven subjects performed sinusoidal isometric contractions to complete seven types of grasps, with each finger and with three combinations of fingers in opposition with the thumb. High-density surface EMG (HD-sEMG) signals (384 channels in total) recorded from the 14 muscles were decomposed into the constituent motor unit action potentials. This provided a non-invasive framework for the investigation of motor neuron discharge patterns, muscle coordination and efferent commands of the hand muscles during grasping. Moreover, during grasping tasks, it was possible to identify common neural information among pools of motor neurons innervating the investigated muscles. For this purpose, principal component analysis (PCA) was applied to the smoothed discharge rates of the decoded motor units.

MAIN RESULTS

We found that the first principal component (PC1) of the ensemble of decoded motor neuron spike trains explained a variance of (53.0 ± 10.9) % and was positively correlated with force (R = 0.67 ± 0.10 across all subjects and tasks). By grouping the pools of motor neurons from extrinsic or intrinsic muscles, the PC1 explained a proportion of variance of (57.1 ± 11.3) % and (56.9 ± 11.8) %, respectively, and was correlated with force with R = 0.63 ± 0.13 and 0.63 ± 0.13, respectively.

SIGNIFICANCE

These observations demonstrate a low dimensional control of motor neurons across multiple muscles that can be exploited for extracting control signals in neural interfacing. The proposed framework was designed for hand rehabilitation perspectives, such as post-stroke rehabilitation and hand-exoskeleton control.

Dexterous Object Manipulation Requires Context-Dependent Sensorimotor Cortical Interactions in Humans

Dexterous object manipulation is a hallmark of human evolution and a critical skill for everyday activities. A previous work has used a grasping context that predominantly elicits memory-based control of digit forces by constraining where the object should be grasped. For this "constrained" grasping context, the primary motor cortex (M1) is involved in storage and retrieval of digit forces used in previous manipulations. In contrast, when choice of digit contact points is allowed ("unconstrained" grasping), behavioral studies revealed that forces are adjusted, on a trial-to-trial basis, as a function of digit position. This suggests a role of online feedback of digit position for force control. However, despite the ubiquitous nature of unconstrained hand-object interactions in activities of daily living, the underlying neural mechanisms are unknown. Using noninvasive brain stimulation, we found the role of primary motor cortex (M1) and somatosensory cortex (S1) to be sensitive to grasping context. In constrained grasping, M1 but not S1 is involved in storing and retrieving learned digit forces and position. In contrast, in unconstrained grasping, M1 and S1 are involved in modulating digit forces to position. Our findings suggest that the relative contribution of memory and online feedback modulates sensorimotor cortical interactions for dexterous manipulation.

Inertial Measurement Unit Based Upper Extremity Motion Characterization for Action Research Arm Test and Activities of Daily Living

In practical rehabilitation robot development, it is imperative to pre-specify the critical workspace to prevent redundant structure. This study aimed to characterize the upper extremity motion during essential activities in daily living. An IMU-based wearable motion capture system was used to access arm movements. Ten healthy subjects performed the Action Research Arm Test (ARAT) and six pre-selected essential daily activities. The Euler angles of the major joints, and acceleration from wrist and hand sensors were acquired and analyzed. The size of the workspace for the ARAT was 0.53 (left-right) × 0.92 (front-back) × 0.89 (up-down) m for the dominant hand. For the daily activities, the workspace size was 0.71 × 0.70 × 0.86 m for the dominant hand, significantly larger than the non-dominant hand (p ≤ 0.011). The average range of motion (RoM) during ARAT was 109.15 ± 18.82° for elbow flexion/extension, 105.23 ± 5.38° for forearm supination/pronation, 91.99 ± 0.98° for shoulder internal/external rotation, and 82.90 ± 22.52° for wrist dorsiflexion/volarflexion, whereas the corresponding range for daily activities were 120.61 ± 23.64°, 128.09 ± 22.04°, 111.56 ± 31.88°, and 113.70 ± 18.26°. The shoulder joint was more abducted and extended during pinching compared to grasping posture (p < 0.001). Reaching from a grasping posture required approximately 70° elbow extension and 36° forearm supination from the initial position. The study results provide an important database for the workspace and RoM for essential arm movements.

Fixed muscle synergies and their potential to improve the intuitive control of myoelectric assistive technology for upper extremities

BACKGROUND

Users of myoelectric controlled assistive technology (AT) for upper extremities experience difficulties in controlling this technology in daily life, partly because the control is non-intuitive. Making the control of myoelectric AT intuitive may resolve the experienced difficulties. The present paper was inspired by the suggestion that intuitive control may be achieved if the control of myoelectric AT is based on neuromotor control principles. A significant approach within neurocomputational motor control suggests that myosignals are produced via a limited number of fixed muscle synergies. To effectively employ this approach in myoelectric AT, it is required that a limited number of muscle synergies is systematically exploited, also when muscles are used differently as required in controlling myoelectric AT. Therefore, the present study examined the systematic exploitation of muscle synergies when muscles were used differently to complete point-to-point movements with and without a rod.

METHODS

Healthy participants made multidirectional point-to-point movements with different end-effectors, i.e. with the index finger and with rods of different lengths. Myosignals were collected from 22 muscles in the arm, trunk, and back, and subsequently partitioned into muscle synergies per end-effector and for a pooled dataset including all end-effectors. The exploitation of these muscle synergies was assessed by evaluating the similarity of structure and explanatory ability of myosignals of per end-effector muscle synergies and the contribution of pooled muscle synergies across end-effectors.

RESULTS

Per end-effector, 3-5 muscle synergies could explain 73.8-81.1% of myosignal variation, whereas 6-8 muscle synergies from the pooled dataset also captured this amount of myosignal variation. Subsequent analyses showed that gradually different muscle synergies-extracted from separate end-effectors-were exploited across end-effectors. In line with this result, the order of contribution of muscle synergies extracted from the pooled dataset gradually reversed across end-effectors.

CONCLUSION

A limited number of muscle synergies was systematically exploited in the examined set of movements, indicating a potential for the fixed muscle synergy approach to improve the intuitive control of myoelectric AT. Given the gradual change in muscle synergy exploitation across end-effectors, future research should examine whether this potential can be extended to a larger range of movements and tasks.

Synergies in coordination: a comprehensive overview of neural, computational, and behavioral approaches

At face value, the term "synergy" provides a unifying concept within a fractured field that encompasses complementary neural, computational, and behavioral approaches. However, the term is not used synonymously by different researchers but has substantially different meanings depending on the research approach. With so many operational definitions for the one term, it becomes difficult to use as either a descriptive or explanatory concept, yet it remains pervasive and apparently indispensable. Here we provide a summary of different approaches that invoke synergies in a descriptive or explanatory context, summarizing progress, not within the one approach, but across the theoretical landscape. Bernstein's framework of flexible hierarchical control may provide a unifying framework here, since it can incorporate divergent ideas about synergies. In the current motor control literature, synergy may refer to conceptually different processes that could potentially operate in parallel, across different levels within the same hierarchical control scheme. There is evidence for the concurrent existence of synergies with different features, both "hard-wired" and "soft-wired," and task independent and task dependent. By providing a comprehensive overview of the multifaceted ideas about synergies, our goal is to move away from the compartmentalization and narrow the focus on one level and promote a broader perspective on the control and coordination of movement.

Feasibility Theory Reconciles and Informs Alternative Approaches to Neuromuscular Control

We present Feasibility Theory, a conceptual and computational framework to unify today's theories of neuromuscular control. We begin by describing how the musculoskeletal anatomy of the limb, the need to control individual tendons, and the physics of a motor task uniquely specify the family of all valid muscle activations that accomplish it (its ‘feasible activation space’). For our example of producing static force with a finger driven by seven muscles, computational geometry characterizes—in a complete way—the structure of feasible activation spaces as 3-dimensional polytopes embedded in 7-D. The feasible activation space for a given task is the landscape where all neuromuscular learning, control, and performance must occur. This approach unifies current theories of neuromuscular control because the structure of feasible activation spaces can be separately approximated as either low-dimensional basis functions (synergies), high-dimensional joint probability distributions (Bayesian priors), or fitness landscapes (to optimize cost functions).