Sensory integration for reaching: models of optimality in the context of behavior and the underlying neural circuits

Sensorimotor regulation of facial expression - An untouched frontier

Facial expression is a critical form of nonverbal social communication which promotes emotional exchange and affiliation among humans. Facial expressions are generated via precise contraction of the facial muscles, guided by sensory feedback. While the neural pathways underlying facial motor control are well characterized in humans and primates, it remains unknown how tactile and proprioceptive information reaches these pathways to guide facial muscle contraction. Thus, despite the importance of facial expressions for social functioning, little is known about how they are generated as a unique sensorimotor behavior. In this review, we highlight current knowledge about sensory feedback from the face and how it is distinct from other body regions. We describe connectivity between the facial sensory and motor brain systems, and call attention to the other brain systems which influence facial expression behavior, including vision, gustation, emotion, and interoception. Finally, we petition for more research on the sensory basis of facial expressions, asserting that incomplete understanding of sensorimotor mechanisms is a barrier to addressing atypical facial expressivity in clinical populations.

Express Visuomotor Responses Reflect Knowledge of Both Target Locations and Contextual Rules during Reaches of Different Amplitudes

When humans reach to visual targets, extremely rapid (∼90 ms) target-directed responses can be observed in task-relevant proximal muscles. Such express visuomotor responses are inflexibly locked in time and space to the target and have been proposed to reflect rapid visuomotor transformations conveyed subcortically via the tecto-reticulo-spinal pathway. Previously, we showed that express visuomotor responses are sensitive to explicit cue-driven information about the target, suggesting that the express pathway can be modulated by cortical signals affording contextual prestimulus expectations. Here, we show that the express visuomotor system incorporates information about the physical hand-to-target distance and contextual rules during visuospatial tasks requiring different movement amplitudes. In one experiment, we recorded the activity from two shoulder muscles as 14 participants (6 females) reached toward targets that appeared at different distances from the reaching hand. Increasing the reaching distance facilitated the generation of frequent and large express visuomotor responses. This suggests that both the direction and amplitude of veridical hand-to-target reaches are encoded along the putative subcortical express pathway. In a second experiment, we modulated the movement amplitude by asking 12 participants (4 females) to deliberately undershoot, overshoot, or stop (control) at the target. The overshoot and undershoot tasks impaired the generation of large and frequent express visuomotor responses, consistent with the inability of the express pathway to generate responses directed toward nonveridical targets as in the anti-reach task. Our findings appear to reflect strategic, cortically driven modulation of the express visuomotor circuit to facilitate rapid and effective response initiation during target-directed actions.SIGNIFICANCE STATEMENT Express (∼90 ms) arm muscle responses that are consistently tuned toward the location of visual stimuli suggest a subcortical contribution to target-directed visuomotor behavior in humans, potentially via the tecto-reticulo-spinal pathway. Here, we show that express muscle responses are modulated appropriately to reach targets at different distances, but generally suppressed when the task required nonveridical responses to overshoot/undershoot the real target. This suggests that the tecto-reticulo-spinal pathway can be exploited strategically by the cerebral cortex to facilitate rapid initiation of effective responses during a visuospatial task.

Hippocampal blood flow rapidly and preferentially increases after a bout of moderate-intensity exercise in older adults with poor cerebrovascular health

Over the course of aging, there is an early degradation of cerebrovascular health, which may be attenuated with aerobic exercise training. Yet, the acute cerebrovascular response to a single bout of exercise remains elusive, particularly within key brain regions most affected by age-related disease processes. We investigated the acute global and region-specific cerebral blood flow (CBF) response to 15 minutes of moderate-intensity aerobic exercise in older adults (≥65 years; n = 60) using arterial spin labeling magnetic resonance imaging. Within 0-6 min post-exercise, CBF decreased across all regions, an effect that was attenuated in the hippocampus. The exercise-induced CBF drop was followed by a rebound effect over the 24-minute postexercise assessment period, an effect that was most robust in the hippocampus. Individuals with low baseline perfusion demonstrated the greatest hippocampal-specific CBF effect post-exercise, showing no immediate drop and a rapid increase in CBF that exceeded baseline levels within 6-12 minutes postexercise. Gains in domain-specific cognitive performance postexercise were not associated with changes in regional CBF, suggesting dissociable effects of exercise on acute neural and vascular plasticity. Together, the present findings support a precision-medicine framework for the use of exercise to target brain health that carefully considers age-related changes in the cerebrovascular system.

Sensory substitution of elbow proprioception to improve myoelectric control of upper limb prosthesis: experiment on healthy subjects and amputees

BACKGROUND

Current myoelectric prostheses lack proprioceptive information and rely on vision for their control. Sensory substitution is increasingly developed with non-invasive vibrotactile or electrotactile feedback, but most systems are designed for grasping or object discriminations, and few were tested for online control in amputees. The objective of this work was evaluate the effect of a novel vibrotactile feedback on the accuracy of myoelectric control of a virtual elbow by healthy subjects and participants with an upper-limb amputation at humeral level.

METHODS

Sixteen, healthy participants and 7 transhumeral amputees performed myoelectric control of a virtual arm under different feedback conditions: vision alone (VIS), vibration alone (VIB), vision plus vibration (VIS + VIB), or no feedback at all (NO). Reach accuracy was evaluated by angular errors during discrete as well as back and forth movements. Healthy participants' workloads were assessed with the NASA-TLX questionnaire, and feedback conditions were ranked according to preference at the end of the experiment.

RESULTS

Reach errors were higher in NO than in VIB, indicating that our vibrotactile feedback improved performance as compared to no feedback. Conditions VIS and VIS+VIB display similar levels of performance and produced lower errors than in VIB. Vision remains therefore critical to maintain good performance, which is not ameliorated nor deteriorated by the addition of vibrotactile feedback. The workload associated with VIB was higher than for VIS and VIS+VIB, which did not differ from each other. 62.5% of healthy subjects preferred the VIS+VIB condition, and ranked VIS and VIB second and third, respectively.

CONCLUSION

Our novel vibrotactile feedback improved myoelectric control of a virtual elbow as compared to no feedback. Although vision remained critical, the addition of vibrotactile feedback did not improve nor deteriorate the control and was preferred by participants. Longer training should improve performances with VIB alone and reduce the need of vision for close-loop prosthesis control.

How Tilting the Head Interferes With Eye-Hand Coordination: The Role of Gravity in Visuo-Proprioceptive, Cross-Modal Sensory Transformations

To correctly position the hand with respect to the spatial location and orientation of an object to be reached/grasped, visual information about the target and proprioceptive information from the hand must be compared. Since visual and proprioceptive sensory modalities are inherently encoded in a retinal and musculo-skeletal reference frame, respectively, this comparison requires cross-modal sensory transformations. Previous studies have shown that lateral tilts of the head interfere with the visuo-proprioceptive transformations. It is unclear, however, whether this phenomenon is related to the neck flexion or to the head-gravity misalignment. To answer to this question, we performed three virtual reality experiments in which we compared a grasping-like movement with lateral neck flexions executed in an upright seated position and while lying supine. In the main experiment, the task requires cross-modal transformations, because the target information is visually acquired, and the hand is sensed through proprioception only. In the other two control experiments, the task is unimodal, because both target and hand are sensed through one, and the same, sensory channel (vision and proprioception, respectively), and, hence, cross-modal processing is unnecessary. The results show that lateral neck flexions have considerably different effects in the seated and supine posture, but only for the cross-modal task. More precisely, the subjects’ response variability and the importance associated to the visual encoding of the information significantly increased when supine. We show that these findings are consistent with the idea that head-gravity misalignment interferes with the visuo-proprioceptive cross-modal processing. Indeed, the principle of statistical optimality in multisensory integration predicts the observed results if the noise associated to the visuo-proprioceptive transformations is assumed to be affected by gravitational signals, and not by neck proprioceptive signals per se. This finding is also consistent with the observation of otolithic projections in the posterior parietal cortex, which is involved in the visuo-proprioceptive processing. Altogether these findings represent a clear evidence of the theorized central role of gravity in spatial perception. More precisely, otolithic signals would contribute to reciprocally align the reference frames in which the available sensory information can be encoded.

Vestibular Stimulation May Drive Multisensory Processing: Principles for Targeted Sensorimotor Therapy (TSMT)

At birth, the vestibular system is fully mature, whilst higher order sensory processing is yet to develop in the full-term neonate. The current paper lays out a theoretical framework to account for the role vestibular stimulation may have driving multisensory and sensorimotor integration. Accordingly, vestibular stimulation, by activating the parieto-insular vestibular cortex, and/or the posterior parietal cortex may provide the cortical input for multisensory neurons in the superior colliculus that is needed for multisensory processing. Furthermore, we propose that motor development, by inducing change of reference frames, may shape the receptive field of multisensory neurons. This, by leading to lack of spatial contingency between formally contingent stimuli, may cause degradation of prior motor responses. Additionally, we offer a testable hypothesis explaining the beneficial effect of sensory integration therapies regarding attentional processes. Key concepts of a sensorimotor integration therapy (e.g., targeted sensorimotor therapy (TSMT)) are also put into a neurological context. TSMT utilizes specific tools and instruments. It is administered in 8-weeks long successive treatment regimens, each gradually increasing vestibular and postural stimulation, so sensory-motor integration is facilitated, and muscle strength is increased. Empirically TSMT is indicated for various diseases. Theoretical foundations of this sensorimotor therapy are discussed.

Non-invasive Brain Stimulation of the Posterior Parietal Cortex Alters Postural Adaptation

Effective central sensory integration of visual, vestibular, and proprioceptive information is required to promote adaptability in response to changes in the environment during postural control. Patients with a lesion in the posterior parietal cortex (PPC) have an impaired ability to form an internal representation of body position, an important factor for postural control and adaptation. Suppression of PPC excitability has also been shown to decrease postural stability in some contexts. As of yet, it is unknown whether stimulation of the PPC may influence postural adaptation. This investigation aimed to identify whether transcranial direct current stimulation (tDCS) of the bilateral PPC could modulate postural adaptation in response to a bipedal incline postural adaptation task. Using young, healthy subjects, we delivered tDCS over bilateral PPC followed by bouts of inclined stance (incline-interventions). Analysis of postural after-effects identified differences between stimulation conditions for maximum lean after-effect (LAE; p = 0.005) as well as a significant interaction between condition and measurement period for the average position (p = 0.03). We identified impaired postural adaptability following both active stimulation conditions. Results reinforce the notion that the PPC is involved in motor adaptation and extend this line of research to the realm of standing posture. The results further highlight the role of the bilateral PPC in utilizing sensory feedback to update one's internal representation of verticality and demonstrates the diffuse regions of the brain that are involved in postural control and adaptation. This information improves our understanding of the role of the cortex in postural control, highlighting the potential for the PPC as a target for sensorimotor rehabilitation.

A Brief History of the Encoding of Hand Position by the Cerebral Cortex: Implications for Motor Control and Cognition

Encoding hand position by the cerebral cortex is essential not only for the neural representation of the body image but also for different actions based on eye-hand coordination. These include reaching for visual objects as well as complex movement sequences, such as tea-making, tool use, and object construction, among many others. All these functions depend on a continuous refreshing of the hand position representation, relying on both predictive signaling and afferent information. The hand position influence on neural activity in the parietofrontal system, together with eye position signals, are the basic elements of an eye-hand matrix from which all the above functions can emerge and could be regarded as key features of a network with several entry points, command nodes and outflow pathways, as confirmed by the discovery of a direct parietospinal projection for the control of hand action. The integrity of this system is crucial for daily life, as testified by the consequences of cortical lesions, spanning from severe paralysis to complex forms of apraxia. In this review, I will sketch my personal understanding of the scientific and conceptual trajectory of a line of investigation with many unexpected influences on cortical function and disease, from motor behavior to cognition.

Corticocortical Systems Underlying High-Order Motor Control

Cortical networks are characterized by the origin, destination, and reciprocity of their connections, as well as by the diameter, conduction velocity, and synaptic efficacy of their axons. The network formed by parietal and frontal areas lies at the core of cognitive-motor control because the outflow of parietofrontal signaling is conveyed to the subcortical centers and spinal cord through different parallel pathways, whose orchestration determines, not only when and how movements will be generated, but also the nature of forthcoming actions. Despite intensive studies over the last 50 years, the role of corticocortical connections in motor control and the principles whereby selected cortical networks are recruited by different task demands remain elusive. Furthermore, the synaptic integration of different cortical signals, their modulation by transthalamic loops, and the effects of conduction delays remain challenging questions that must be tackled to understand the dynamical aspects of parietofrontal operations. In this article, we evaluate results from nonhuman primate and selected rodent experiments to offer a viewpoint on how corticocortical systems contribute to learning and producing skilled actions. Addressing this subject is not only of scientific interest but also essential for interpreting the devastating consequences for motor control of lesions at different nodes of this integrated circuit. In humans, the study of corticocortical motor networks is currently based on MRI-related methods, such as resting-state connectivity and diffusion tract-tracing, which both need to be contrasted with histological studies in nonhuman primates.

Putting the "Sensory" Into Sensorimotor Control: The Role of Sensorimotor Integration in Goal-Directed Hand Movements After Stroke

Integration of sensory and motor information is one-step, among others, that underlies the successful production of goal-directed hand movements necessary for interacting with our environment. Disruption of sensorimotor integration is prevalent in many neurologic disorders, including stroke. In most stroke survivors, persistent paresis of the hand reduces function and overall quality of life. Current rehabilitative methods are based on neuroplastic principles to promote motor learning that focuses on regaining motor function lost due to paresis, but the sensory contributions to motor control and learning are often overlooked and currently understudied. There is a need to evaluate and understand the contribution of both sensory and motor function in the rehabilitation of skilled hand movements after stroke. Here, we will highlight the importance of integration of sensory and motor information to produce skilled hand movements in healthy individuals and individuals after stroke. We will then discuss how compromised sensorimotor integration influences relearning of skilled hand movements after stroke. Finally, we will propose an approach to target sensorimotor integration through manipulation of sensory input and motor output that may have therapeutic implications.

Generalization via superposition: combined effects of mixed reference frame representations for explicit and implicit learning in a visuomotor adaptation task

Studies on generalization of learned visuomotor perturbations have generally focused on whether learning is coded in extrinsic or intrinsic reference frames. This dichotomy, however, is challenged by recent findings showing that learning is represented in a mixed reference frame. Overlooked in this framework is how learning appears to consist of multiple processes, such as explicit reaiming and implicit motor adaptation. Therefore, the proposed mixed representation may simply reflect the superposition of explicit and implicit generalization functions, each represented in different reference frames. Here we characterized the individual generalization functions of explicit and implicit learning in relative isolation to determine whether their combination could predict the overall generalization function when both processes are in operation. We modified the form of feedback in a visuomotor rotation task in an attempt to isolate explicit and implicit learning and tested generalization across new limb postures to dissociate the extrinsic/intrinsic representations. We found that the amplitude of explicit generalization was reduced with postural change and was only marginally shifted, resembling an extrinsic representation. In contrast, implicit generalization maintained its amplitude but was significantly shifted, resembling a mixed representation. A linear combination of individual explicit and implicit generalization functions accounted for nearly 85% of the variance associated with the generalization function in a typical visuomotor rotation task, where both processes are in operation. This suggests that each form of learning results from a mixed representation with distinct extrinsic and intrinsic contributions and the combination of these features shapes the generalization pattern observed at novel limb postures. NEW & NOTEWORTHY Generalization following learning in visuomotor adaptation tasks can reflect how the brain represents what it learns. In this study, we isolated explicit and implicit forms of learning and showed that they are derived from a mixed reference frame representation with distinct extrinsic and intrinsic contributions. Furthermore, we showed that the overall generalization pattern at novel workspaces is due to the superposition of independent generalization effects developed by explicit and implicit learning processes.

Unilateral movement preparation causes task-specific modulation of TMS responses in the passive, opposite limb

KEY POINTS

Activity in the primary motor cortices of both hemispheres increases during unilateral movement preparation, but the functional role of ipsilateral motor cortex activity is unknown. Ipsilateral motor cortical activity could represent subliminal 'motor planning' for the passive limb. Alternatively, it could represent the state of the active limb, to support coordination between the limbs should a bimanual movement be required. Here we assessed how preparation of forces toward different directions, with the left wrist, alters evoked responses to transcranial magnetic stimulation of left motor cortex. Preparation of a unilateral movement caused excitability increases in ipsilateral motor cortex that reflected forces produced with the active limb in an intrinsic (body-centred), rather than an extrinsic (world-centred), coordinate system. These results suggest that ipsilateral motor cortical activity prior to unilateral action reflects the state of the active limb, rather than subliminal motor planning for the passive limb.

ABSTRACT

Corticospinal excitability is modulated for muscles on both sides of the body during unilateral movement preparation. For the effector, there is a progressive increase in excitability, and a shift in direction of muscle twitches evoked by transcranial magnetic stimulation (TMS) toward the impending movement. By contrast, the directional characteristics of excitability changes in the opposite (passive) limb have not been fully characterized. Here we assessed how preparation of voluntary forces towards four spatially distinct visual targets with the left wrist alters muscle twitches and motor-evoked potentials (MEPs) elicited by TMS of left motor cortex. MEPs were facilitated significantly more in muscles homologous to agonist rather than antagonist muscles in the active limb, from 120 ms prior to voluntary EMG onset. Thus, unilateral motor preparation has a directionally specific influence on pathways projecting to the opposite limb that corresponds to the active muscles rather than the direction of movement in space. The directions of TMS-evoked twitches also deviated toward the impending force direction of the active limb, according to muscle-based coordinates, following the onset of voluntary EMG. The data indicate that preparation of a unilateral movement increases task-dependent excitability in ipsilateral motor cortex, or its downstream projections, that reflects the forces applied by the active limb in an intrinsic (body-centred), rather than an extrinsic (world-centred), coordinate system. The results suggest that ipsilateral motor cortical activity prior to unilateral action reflects the state of the active limb, rather than subliminal motor planning for the passive limb.

Multimodal sensory processing in Caenorhabditis elegans

Multisensory integration is a mechanism that allows organisms to simultaneously sense and understand external stimuli from different modalities. These distinct signals are transduced into neuronal signals that converge into decision-making neuronal entities. Such decision-making centres receive information through neuromodulators regarding the organism's physiological state and accordingly trigger behavioural responses. Despite the importance of multisensory integration for efficient functioning of the nervous system, and also the implication of dysfunctional multisensory integration in the aetiology of neuropsychiatric disease, little is known about the relative molecular mechanisms. Caenorhabditis elegans is an appropriate model system to study such mechanisms and elucidate the molecular ways through which organisms understand external environments in an accurate and coherent fashion.

Vision for Prehension in the Medial Parietal Cortex

In the last 2 decades, the medial posterior parietal area V6A has been extensively studied in awake macaque monkeys for visual and somatosensory properties and for its involvement in encoding of spatial parameters for reaching, including arm movement direction and amplitude. This area also contains populations of neurons sensitive to grasping movements, such as wrist orientation and grip formation. Recent work has shown that V6A neurons also encode the shape of graspable objects and their affordance. In other words, V6A seems to encode object visual properties specifically for the purpose of action, in a dynamic sequence of visuomotor transformations that evolve in the course of reach-to-grasp action.We propose a model of cortical circuitry controlling reach-to-grasp actions, in which V6A acts as a comparator that monitors differences between current and desired hand positions and configurations. This error signal could be used to continuously update the motor output, and to correct reach direction, hand orientation, and/or grip aperture as required during the act of prehension.In contrast to the generally accepted view that the dorsomedial component of the dorsal visual stream encodes reaching, but not grasping, the functional properties of V6A neurons strongly suggest the view that this area is involved in encoding all phases of prehension, including grasping.

Distinct coordinate systems for adaptations of movement direction and extent

Learned compensations for perturbed visual feedback of movement extent and direction generalize differently to unpracticed movement directions, which suggests different underlying neural mechanisms. Here we investigated whether gain and rotation adaptations are consistent with representation in different coordinate systems. Subjects performed a force-aiming task with the wrist and learned different gains or rotations for different force directions. Generalization was tested without visual feedback for the same extrinsic directions but with the forearm in a different pronation-supination orientation. When the change in forearm orientation caused the adapted visuomotor map to conflict in extrinsic and joint-based coordinates, rotation generalization occurred in extrinsic coordinates but with reduced magnitude. In contrast, gain generalization appeared reduced and phase shifted. When the forearm was rotated further, such that all imposed perturbations aligned in both joint-based and extrinsic coordinates in both postures, rotation generalization was further reduced, whereas there was neither reduction nor phase shift in the pattern of extent generalization. These results show that rotation generalization was expressed in extrinsic coordinates, and that generalization magnitude was modulated by posture. In contrast, gain generalization appeared to depend on target direction defined by an integrated combination of extrinsic and joint-based coordinates and was not reduced substantially by posture changes alone. Although the quality of the model fit underlying our interpretation prevents us from making strong conclusions, the data suggest that adaptations of movement direction and extent are represented according to distinct coordinate systems.NEW & NOTEWORTHY Visuomotor gain and rotation adaptations generalize differently to novel movement directions, which suggests different neural mechanisms. When extrinsic and joint-based coordinates are effectively dissociated in an isometric aiming task, we find that they also generalize in different coordinate systems. Specifically, rotation generalized in extrinsic coordinates and decayed as posture departed from that adopted during adaptation. In contrast, gain generalization was expressed according to mixed extrinsic/joint-based coordinates and was not substantially reduced by postural changes.

The absence or temporal offset of visual feedback does not influence adaptation to novel movement dynamics

Delays in transmitting and processing sensory information require correctly associating delayed feedback to issued motor commands for accurate error compensation. The flexibility of this alignment between motor signals and feedback has been demonstrated for movement recalibration to visual manipulations, but the alignment dependence for adapting movement dynamics is largely unknown. Here we examined the effect of visual feedback manipulations on force-field adaptation. Three subject groups used a manipulandum while experiencing a lag in the corresponding cursor motion (0, 75, or 150 ms). When the offset was applied at the start of the session (continuous condition), adaptation was not significantly different between groups. However, these similarities may be due to acclimation to the offset before motor adaptation. We tested additional subjects who experienced the same delays concurrent with the introduction of the perturbation (abrupt condition). In this case adaptation was statistically indistinguishable from the continuous condition, indicating that acclimation to feedback delay was not a factor. In addition, end-point errors were not significantly different across the delay or onset conditions, but end-point correction (e.g., deceleration duration) was influenced by the temporal offset. As an additional control, we tested a group of subjects who performed without visual feedback and found comparable movement adaptation results. These results suggest that visual feedback manipulation (absence or temporal misalignment) does not affect adaptation to novel dynamics, independent of both acclimation and perceptual awareness. These findings could have implications for modeling how the motor system adjusts to errors despite concurrent delays in sensory feedback information.NEW & NOTEWORTHY A temporal offset between movement and distorted visual feedback (e.g., visuomotor rotation) influences the subsequent motor recalibration, but the effects of this offset for altered movement dynamics are largely unknown. Here we examined the influence of 1) delayed and 2) removed visual feedback on the adaptation to novel movement dynamics. These results contribute to understanding of the control strategies that compensate for movement errors when there is a temporal separation between motion state and sensory information.

The visual encoding of purely proprioceptive intermanual tasks is due to the need of transforming joint signals, not to their interhemispheric transfer

To perform goal-oriented hand movement, humans combine multiple sensory signals (e.g., vision and proprioception) that can be encoded in various reference frames (body centered and/or exo-centered). In a previous study (Tagliabue M, McIntyre J. PLoS One 8: e68438, 2013), we showed that, when aligning a hand to a remembered target orientation, the brain encodes both target and response in visual space when the target is sensed by one hand and the response is performed by the other, even though both are sensed only through proprioception. Here we ask whether such visual encoding is due 1) to the necessity of transferring sensory information across the brain hemispheres, or 2) to the necessity, due to the arms' anatomical mirror symmetry, of transforming the joint signals of one limb into the reference frame of the other. To answer this question, we asked subjects to perform purely proprioceptive tasks in different conditions: Intra, the same arm sensing the target and performing the movement; Inter/Parallel, one arm sensing the target and the other reproducing its orientation; and Inter/Mirror, one arm sensing the target and the other mirroring its orientation. Performance was very similar between Intra and Inter/Mirror (conditions not requiring joint-signal transformations), while both differed from Inter/Parallel. Manipulation of the visual scene in a virtual reality paradigm showed visual encoding of proprioceptive information only in the latter condition. These results suggest that the visual encoding of purely proprioceptive tasks is not due to interhemispheric transfer of the proprioceptive information per se, but to the necessity of transforming joint signals between mirror-symmetric limbs.NEW & NOTEWORTHY Why does the brain encode goal-oriented, intermanual tasks in a visual space, even in the absence of visual feedback about the target and the hand? We show that the visual encoding is not due to the transfer of proprioceptive signals between brain hemispheres per se, but to the need, due to the mirror symmetry of the two limbs, of transforming joint angle signals of one arm in different joint signals of the other.

EXiO-A Brain-Controlled Lower Limb Exoskeleton for Rhesus Macaques

Recent advances in the field of brain-machine interfaces (BMIs) have demonstrated enormous potential to shape the future of rehabilitation and prosthetic devices. Here, a lower-limb exoskeleton controlled by the intracortical activity of an awake behaving rhesus macaque is presented as a proof-of-concept for a locomotorBMI. A detailed description of the mechanical device, including its innovative features and first experimental results, is provided. During operation, BMI-decoded position and velocity are directly mapped onto the bipedal exoskeleton's motions, which then move the monkey's legs as the monkey remains physicallypassive. To meet the unique requirements of such an application, the exoskeleton's features include: high output torque with backdrivable actuation, size adjustability, and safe user-robot interface. In addition, a novel rope transmission is introduced and implemented. To test the performance of the exoskeleton, a mechanical assessment was conducted, which yielded quantifiable results for transparency, efficiency, stiffness, and tracking performance. Usage under both brain control and automated actuation demonstrates the device's capability to fulfill the demanding needs of this application. These results lay the groundwork for further advancement in BMI-controlled devices for primates including humans.

Computational principles and models of multisensory integration

Combining information from multiple senses creates robust percepts, speeds up responses, enhances learning, and improves detection, discrimination, and recognition. In this review, I discuss computational models and principles that provide insight into how this process of multisensory integration occurs at the behavioral and neural level. My initial focus is on drift-diffusion and Bayesian models that can predict behavior in multisensory contexts. I then highlight how recent neurophysiological and perturbation experiments provide evidence for a distributed redundant network for multisensory integration. I also emphasize studies which show that task-relevant variables in multisensory contexts are distributed in heterogeneous neural populations. Finally, I describe dimensionality reduction methods and recurrent neural network models that may help decipher heterogeneous neural populations involved in multisensory integration.

Separating Predicted and Perceived Sensory Consequences of Motor Learning

During motor adaptation the discrepancy between predicted and actually perceived sensory feedback is thought to be minimized, but it can be difficult to measure predictions of the sensory consequences of actions. Studies attempting to do so have found that self-directed, unseen hand position is mislocalized in the direction of altered visual feedback. However, our lab has shown that motor adaptation also leads to changes in perceptual estimates of hand position, even when the target hand is passively displaced. We attribute these changes to a recalibration of hand proprioception, since in the absence of a volitional movement, efferent or predictive signals are likely not involved. The goal here is to quantify the extent to which changes in hand localization reflect a change in the predicted sensory (visual) consequences or a change in the perceived (proprioceptive) consequences. We did this by comparing changes in localization produced when the hand movement was self-generated (‘active localization’) versus robot-generated (‘passive localization’) to the same locations following visuomotor adaptation to a rotated cursor. In this passive version, there should be no predicted consequences of these robot-generated hand movements. We found that although changes in localization were somewhat larger in active localization, the passive localization task also elicited substantial changes. Our results suggest that the change in hand localization following visuomotor adaptation may not be based entirely on updating predicted sensory consequences, but may largely reflect changes in our proprioceptive state estimate.

Intracortical Brain-Machine Interfaces Advance Sensorimotor Neuroscience

Brain-machine interfaces (BMIs) decode brain activity to control external devices. Over the past two decades, the BMI community has grown tremendously and reached some impressive milestones, including the first human clinical trials using chronically implanted intracortical electrodes. It has also contributed experimental paradigms and important findings to basic neuroscience. In this review, we discuss neuroscience achievements stemming from BMI research, specifically that based upon upper limb prosthetic control with intracortical microelectrodes. We will focus on three main areas: first, we discuss progress in neural coding of reaches in motor cortex, describing recent results linking high dimensional representations of cortical activity to muscle activation. Next, we describe recent findings on learning and plasticity in motor cortex on various time scales. Finally, we discuss how bidirectional BMIs have led to better understanding of somatosensation in and related to motor cortex.

Effect of coordinate frame compatibility on the transfer of implicit and explicit learning across limbs

Insights into the neural representation of motor learning can be obtained by investigating how learning transfers to novel task conditions. We recently demonstrated that visuomotor rotation learning transferred strongly between left and right limbs when the task was performed in a sagittal workspace, which afforded a consistent remapping for the two limbs in both extrinsic and joint-based coordinates. In contrast, transfer was absent when performed in horizontal workspace, where the extrinsically defined perturbation required conflicting joint-based remapping for the left and right limbs. Because visuomotor learning is thought to be supported by both implicit and explicit forms of learning, however, it is unclear to what extent these distinct forms of learning contribute to interlimb transfer. In this study, we assessed the degree to which interlimb transfer, following visuomotor rotation training, reflects explicit vs. implicit learning by obtaining verbal reports of participants' aiming direction before each movement. We also determined the extent to which these distinct components of learning are constrained by the compatibility of coordinate systems by comparing transfer between groups of participants who reached to targets arranged in the horizontal and sagittal planes. Both sagittal and horizontal conditions displayed complete transfer of explicit learning to the untrained limb. In contrast, transfer of implicit learning was incomplete, but the sagittal condition showed greater transfer than the horizontal condition. These findings suggest that explicit strategies developed with one limb can be fully implemented in the opposite limb, whereas implicit transfer depends on the degree to which new sensorimotor maps are spatially compatible for the two limbs.

Decentralized Multisensory Information Integration in Neural Systems

How multiple sensory cues are integrated in neural circuitry remains a challenge. The common hypothesis is that information integration might be accomplished in a dedicated multisensory integration area receiving feedforward inputs from the modalities. However, recent experimental evidence suggests that it is not a single multisensory brain area, but rather many multisensory brain areas that are simultaneously involved in the integration of information. Why many mutually connected areas should be needed for information integration is puzzling. Here, we investigated theoretically how information integration could be achieved in a distributed fashion within a network of interconnected multisensory areas. Using biologically realistic neural network models, we developed a decentralized information integration system that comprises multiple interconnected integration areas. Studying an example of combining visual and vestibular cues to infer heading direction, we show that such a decentralized system is in good agreement with anatomical evidence and experimental observations. In particular, we show that this decentralized system can integrate information optimally. The decentralized system predicts that optimally integrated information should emerge locally from the dynamics of the communication between brain areas and sheds new light on the interpretation of the connectivity between multisensory brain areas.

SIGNIFICANCE STATEMENT

To extract information reliably from ambiguous environments, the brain integrates multiple sensory cues, which provide different aspects of information about the same entity of interest. Here, we propose a decentralized architecture for multisensory integration. In such a system, no processor is in the center of the network topology and information integration is achieved in a distributed manner through reciprocally connected local processors. Through studying the inference of heading direction with visual and vestibular cues, we show that the decentralized system can integrate information optimally, with the reciprocal connections between processers determining the extent of cue integration. Our model reproduces known multisensory integration behaviors observed in experiments and sheds new light on our understanding of how information is integrated in the brain.

Brain-computer interfaces for dissecting cognitive processes underlying sensorimotor control

Sensorimotor control engages cognitive processes such as prediction, learning, and multisensory integration. Understanding the neural mechanisms underlying these cognitive processes with arm reaching is challenging because we currently record only a fraction of the relevant neurons, the arm has nonlinear dynamics, and multiple modalities of sensory feedback contribute to control. A brain-computer interface (BCI) is a well-defined sensorimotor loop with key simplifying advantages that address each of these challenges, while engaging similar cognitive processes. As a result, BCI is becoming recognized as a powerful tool for basic scientific studies of sensorimotor control. Here, we describe the benefits of BCI for basic scientific inquiries and review recent BCI studies that have uncovered new insights into the neural mechanisms underlying sensorimotor control.

Enhanced crosslimb transfer of force-field learning for dynamics that are identical in extrinsic and joint-based coordinates for both limbs

Humans are able to adapt their motor commands to make accurate movements in novel sensorimotor environments, such as when wielding tools that alter limb dynamics. However, it is unclear to what extent sensorimotor representations, obtained through experience with one limb, are available to the opposite, untrained limb and in which form they are available. Here, we compared crosslimb transfer of force-field compensation after participants adapted to a velocity-dependent curl field, oriented either in the sagittal or the transverse plane. Due to the mirror symmetry of the limbs, the force field had identical effects for both limbs in joint and extrinsic coordinates in the sagittal plane but conflicting joint-based effects in the transverse plane. The degree of force-field compensation exhibited by the opposite arm in probe trials immediately after initial learning was significantly greater after sagittal (26 ± 5%) than transverse plane adaptation (9 ± 4%; P < 0.001), irrespective of whether participants learned initially with the left or the right arm or via abrupt or gradual exposure to the force field. Thus transfer was impaired when the orientation of imposed dynamics conflicted in intrinsic coordinates for the two limbs. The data reveal that neural representations of novel dynamics are only partially available to the opposite limb, since transfer is incomplete even when force-field perturbation is spatially compatible for the two limbs, according to both intrinsic and extrinsic coordinates.

The Importance of Lateral Connections in the Parietal Cortex for Generating Motor Plans

Substantial evidence has highlighted the significant role of associative brain areas, such as the posterior parietal cortex (PPC) in transforming multimodal sensory information into motor plans. However, little is known about how different sensory information, which can have different delays or be absent, combines to produce a motor plan, such as executing a reaching movement. To address these issues, we constructed four biologically plausible network architectures to simulate PPC: 1) feedforward from sensory input to the PPC to a motor output area, 2) feedforward with the addition of an efference copy from the motor area, 3) feedforward with the addition of lateral or recurrent connectivity across PPC neurons, and 4) feedforward plus efference copy, and lateral connections. Using an evolutionary strategy, the connectivity of these network architectures was evolved to execute visually guided movements, where the target stimulus provided visual input for the entirety of each trial. The models were then tested on a memory guided motor task, where the visual target disappeared after a short duration. Sensory input to the neural networks had sensory delays consistent with results from monkey studies. We found that lateral connections within the PPC resulted in smoother movements and were necessary for accurate movements in the absence of visual input. The addition of lateral connections resulted in velocity profiles consistent with those observed in human and non-human primate visually guided studies of reaching, and allowed for smooth, rapid, and accurate movements under all conditions. In contrast, Feedforward or Feedback architectures were insufficient to overcome these challenges. Our results suggest that intrinsic lateral connections are critical for executing accurate, smooth motor plans.

A learning-based approach to artificial sensory feedback leads to optimal integration

Proprioception—the sense of the body’s position in space—plays an important role in natural movement planning and execution and will likewise be necessary for successful motor prostheses and Brain–Machine Interfaces (BMIs). Here, we demonstrated that monkeys could learn to use an initially unfamiliar multi–channel intracortical microstimulation (ICMS) signal, which provided continuous information about hand position relative to an unseen target, to complete accurate reaches. Furthermore, monkeys combined this artificial signal with vision to form an optimal, minimum–variance estimate of relative hand position. These results demonstrate that a learning–based approach can be used to provide a rich artificial sensory feedback signal, suggesting a new strategy for restoring proprioception to patients using BMIs as well as a powerful new tool for studying the adaptive mechanisms of sensory integration.

Spectral parameters modulation and source localization of blink-related alpha and low-beta oscillations differentiate minimally conscious state from vegetative state/unresponsive wakefulness syndrome

Recently, the cortical source of blink-related delta oscillations (delta BROs) in resting healthy subjects has been localized in the posterior cingulate cortex/precuneus (PCC/PCu), one of the main core-hubs of the default-mode network. This has been interpreted as the electrophysiological signature of the automatic monitoring of the surrounding environment while subjects are immersed in self-reflecting mental activities. Although delta BROs were directly correlated to the degree of consciousness impairment in patients with disorders of consciousness, they failed to differentiate vegetative state/unresponsive wakefulness syndrome (VS/UWS) from minimally conscious state (MCS). In the present study, we have extended the analysis of BROs to frequency bands other than delta in the attempt to find a biological marker that could support the differential diagnosis between VS/UWS and MCS. Four patients with VS/UWS, 5 patients with MCS, and 12 healthy matched controls (CTRL) underwent standard 19-channels EEG recordings during resting conditions. Three-second-lasting EEG epochs centred on each blink instance were submitted to time-frequency analyses in order to extract the normalized Blink-Related Synchronization/Desynchronization (nBRS/BRD) of three bands of interest (low-alpha, high-alpha and low-beta) in the time-window of 50–550 ms after the blink-peak and to estimate the corresponding cortical sources of electrical activity. VS/UWS nBRS/BRD levels of all three bands were lower than those related to both CTRL and MCS, thus enabling the differential diagnosis between MCS and VS/UWS. Furthermore, MCS showed an intermediate signal intensity on PCC/PCu between CTRL and VS/UWS and a higher signal intensity on the left temporo-parieto-occipital junction and inferior occipito-temporal regions when compared to VS/UWS. This peculiar pattern of activation leads us to hypothesize that resting MCS patients have a bottom-up driven activation of the task positive network and thus are tendentially prone to respond to environmental stimuli, even though in an almost unintentional way.

New visuomotor maps are immediately available to the opposite limb

Humans can learn to make accurate movements when the required map between vision and motor commands changes, but can visuomotor maps obtained through experience with one limb benefit the other? Complete transfer would require new maps to be both fully compatible and accessible between limbs. However, when this question is addressed by providing subjects with rotated visual feedback during reaching, transfer is rarely apparent in the first few trials with the unpracticed limb and is sometimes absent altogether. Partial transfer might be explained by limited accessibility to remapped brain circuits, since critical visuomotor transformations mediating unilateral movements appear to be lateralized. Alternatively, if adaptation involves movement representations associated with both extrinsic (i.e., direction of motion in space) and intrinsic (i.e., joint or muscle based) frames of reference, new visuomotor maps might be incompatible with opposite limb use when visual distortions have opposite effects for the two limbs in intrinsic coordinates. Here we addressed this issue when subjects performed an isometric aiming task with the index finger. We manipulated the alignment of visuomotor distortion for the two hands in different reference frames by altering body posture relative to the orientation of the finger and the visual display. There was strong, immediate transfer of adaptation between limbs only when visuomotor distortion had identical effects in eye- and joint-based coordinates bilaterally. This implies that new visuomotor maps are encoded in neural circuits associated with both intrinsic and extrinsic movement representations and are available to both limbs.

Different evolutionary origins for the reach and the grasp: an explanation for dual visuomotor channels in primate parietofrontal cortex

The Dual Visuomotor Channel Theory proposes that manual prehension consists of two temporally integrated movements, each subserved by distinct visuomotor pathways in occipitoparietofrontal cortex. The Reach is mediated by a dorsomedial pathway and transports the hand in relation to the target's extrinsic properties (i.e., location and orientation). The Grasp is mediated by a dorsolateral pathway and opens, preshapes, and closes the hand in relation to the target's intrinsic properties (i.e., size and shape). Here, neuropsychological, developmental, and comparative evidence is reviewed to show that the Reach and the Grasp have different evolutionary origins. First, the removal or degradation of vision causes prehension to decompose into its constituent Reach and Grasp components, which are then executed in sequence or isolation. Similar decomposition occurs in optic ataxic patients following cortical injury to the Reach and the Grasp pathways and after corticospinal tract lesions in non-human primates. Second, early non-visual PreReach and PreGrasp movements develop into mature Reach and Grasp movements but are only integrated under visual control after a prolonged developmental period. Third, comparative studies reveal many similarities between stepping movements and the Reach and between food handling movements and the Grasp, suggesting that the Reach and the Grasp are derived from different evolutionary antecedents. The evidence is discussed in relation to the ideas that dual visuomotor channels in primate parietofrontal cortex emerged as a result of distinct evolutionary origins for the Reach and the Grasp; that foveated vision in primates serves to integrate the Reach and the Grasp into a single prehensile act; and, that flexible recombination of discrete Reach and Grasp movements under various forms of sensory and cognitive control can produce adaptive behavior.

Creating new functional circuits for action via brain-machine interfaces

Brain-machine interfaces (BMIs) are an emerging technology with great promise for developing restorative therapies for those with disabilities. BMIs also create novel, well-defined functional circuits for action that are distinct from the natural sensorimotor apparatus. Closed-loop control of BMI systems can also actively engage learning and adaptation. These properties make BMIs uniquely suited to study learning of motor and non-physical, abstract skills. Recent work used motor BMIs to shed light on the neural representations of skill formation and motor adaptation. Emerging work in sensory BMIs, and other novel interface systems, also highlight the promise of using BMI systems to study fundamental questions in learning and sensorimotor control. This paper outlines the interpretation of BMIs as novel closed-loop systems and the benefits of these systems for studying learning. We review BMI learning studies, their relation to motor control, and propose future directions for this nascent field. Understanding learning in BMIs may both elucidate mechanisms of natural motor and abstract skill learning, and aid in developing the next generation of neuroprostheses.

When kinesthesia becomes visual: a theoretical justification for executing motor tasks in visual space

Several experimental studies in the literature have shown that even when performing purely kinesthetic tasks, such as reaching for a kinesthetically felt target with a hidden hand, the brain reconstructs a visual representation of the movement. In our previous studies, however, we did not observe any role of a visual representation of the movement in a purely kinesthetic task. This apparent contradiction could be related to a fundamental difference between the studied tasks. In our study subjects used the same hand to both feel the target and to perform the movement, whereas in most other studies, pointing to a kinesthetic target consisted of pointing with one hand to the finger of the other, or to some other body part. We hypothesize, therefore, that it is the necessity of performing inter-limb transformations that induces a visual representation of purely kinesthetic tasks. To test this hypothesis we asked subjects to perform the same purely kinesthetic task in two conditions: INTRA and INTER. In the former they used the right hand to both perceive the target and to reproduce its orientation. In the latter, subjects perceived the target with the left hand and responded with the right. To quantify the use of a visual representation of the movement we measured deviations induced by an imperceptible conflict that was generated between visual and kinesthetic reference frames. Our hypothesis was confirmed by the observed deviations of responses due to the conflict in the INTER, but not in the INTRA, condition. To reconcile these observations with recent theories of sensori-motor integration based on maximum likelihood estimation, we propose here a new model formulation that explicitly considers the effects of covariance between sensory signals that are directly available and internal representations that are ‘reconstructed’ from those inputs through sensori-motor transformations.

Keep your head on straight: facilitating sensori-motor transformations for eye-hand coordination

In many day-to-day situations humans manifest a marked tendency to hold the head vertical while performing sensori-motor actions. For instance, when performing coordinated whole-body motor tasks, such as skiing, gymnastics or simply walking, and even when driving a car, human subjects will strive to keep the head aligned with the gravito-inertial vector. Until now, this phenomenon has been thought of as a means to limit variations of sensory signals emanating from the eyes and inner ears. Recent theories suggest that for the task of aligning the hand to a target, the CNS compares target and hand concurrently in both visual and kinesthetic domains, rather than combining sensory data into a single, multimodal reference frame. This implies that when sensory information is lacking in one modality, it must be 'reconstructed' based on information from the other. Here we asked subjects to reach to a visual target with the unseen hand. In this situation, the CNS might reconstruct the orientation of the target in kinesthetic space or reconstruct the orientation of the hand in visual space, or both. By having subjects tilt the head during target acquisition or during movement execution, we show a greater propensity to perform the sensory reconstruction that can be achieved when the head is held upright. These results suggest that the reason humans tend to keep their head upright may also have to do with how the brain manipulates and stores spatial information between reference frames and between sensory modalities, rather than only being tied to the specific problem of stabilizing visual and vestibular inputs.

Learning multisensory integration and coordinate transformation via density estimation

Sensory processing in the brain includes three key operations: multisensory integration-the task of combining cues into a single estimate of a common underlying stimulus; coordinate transformations-the change of reference frame for a stimulus (e.g., retinotopic to body-centered) effected through knowledge about an intervening variable (e.g., gaze position); and the incorporation of prior information. Statistically optimal sensory processing requires that each of these operations maintains the correct posterior distribution over the stimulus. Elements of this optimality have been demonstrated in many behavioral contexts in humans and other animals, suggesting that the neural computations are indeed optimal. That the relationships between sensory modalities are complex and plastic further suggests that these computations are learned-but how? We provide a principled answer, by treating the acquisition of these mappings as a case of density estimation, a well-studied problem in machine learning and statistics, in which the distribution of observed data is modeled in terms of a set of fixed parameters and a set of latent variables. In our case, the observed data are unisensory-population activities, the fixed parameters are synaptic connections, and the latent variables are multisensory-population activities. In particular, we train a restricted Boltzmann machine with the biologically plausible contrastive-divergence rule to learn a range of neural computations not previously demonstrated under a single approach: optimal integration; encoding of priors; hierarchical integration of cues; learning when not to integrate; and coordinate transformation. The model makes testable predictions about the nature of multisensory representations.

Multisensory and unisensory neurons in ferret parietal cortex exhibit distinct functional properties

Despite the fact that unisensory and multisensory neurons are comingled in every neural structure in which they have been identified, no systematic comparison of their response features has been conducted. Towards that goal, the present study was designed to examine and compare measures of response magnitude, latency, duration and spontaneous activity in unisensory and bimodal neurons from the ferret parietal cortex. Using multichannel single-unit recording, bimodal neurons were observed to demonstrate significantly higher response levels and spontaneous discharge rates than did their unisensory counterparts. These results suggest that, rather than merely reflect different connectional arrangements, unisensory and multisensory neurons are likely to differ at the cellular level. Thus, it can no longer be assumed that the different populations of bimodal and unisensory neurons within a neural region respond similarly to a given external stimulus.