Neural coding of intended and executed grasp force in macaque areas AIP, F5, and M1

Accurate neural control of a hand prosthesis by posture-related activity in the primate grasping circuit

Brain-computer interfaces (BCIs) have the potential to restore hand movement for people with paralysis, but current devices still lack the fine control required to interact with objects of daily living. Following our understanding of cortical activity during arm reaches, hand BCI studies have focused primarily on velocity control. However, mounting evidence suggests that posture, and not velocity, dominates in hand-related areas. To explore whether this signal can causally control a prosthesis, we developed a BCI training paradigm centered on the reproduction of posture transitions. Monkeys trained with this protocol were able to control a multidimensional hand prosthesis with high accuracy, including execution of the very intricate precision grip. Analysis revealed that the posture signal in the target grasping areas was the main contributor to control. We present, for the first time, neural posture control of a multidimensional hand prosthesis, opening the door for future interfaces to leverage this additional information channel.

Mixed selectivity in monkey anterior intraparietal area during visual and motor processes

Classical studies suggest that the anterior intraparietal area (AIP) contributes to the encoding of specific information such as objects and actions of self and others, through a variety of neuronal classes, such as canonical, motor and mirror neurons. However, these studies typically focused on a single variable, leaving it unclear whether distinct sets of AIP neurons encode a single or multiple sources of information and how multimodal coding emerges. Here, we chronically recorded monkey AIP neurons in a variety of tasks and conditions classically employed in separate experiments. Most cells exhibited mixed selectivity for observed objects, executed actions, and observed actions, enhanced when this information came from the monkey's peripersonal working space. In contrast with the classical view, our findings indicate that multimodal coding emerges in AIP from partially-mixed selectivity of individual neurons for a variety of information relevant for planning actions directed to both physical objects and other subjects.

Neural underpinnings of the interplay between actual touch and action imagination in social contexts

While there is established evidence supporting the involvement of the sense of touch in various actions, the neural underpinnings of touch and action interplay in a social context remain poorly understood. To prospectively investigate this phenomenon and offer further insights, we employed a combination of motor and sensory components by asking participants to imagine exerting force with the index finger while experiencing their own touch, the touch of one another individual, the touch of a surface, and no touch. Based on the assumption that the patterns of activation in the motor system are similar when action is imagined or actually performed, we proceeded to apply a single-pulse transcranial magnetic stimulation over the primary motor cortex (M1) while participants engaged in the act of imagination. Touch experience was associated with higher M1 excitability in the presence and in the absence of force production imagination, but only during force production imagination M1 excitability differed among the types of touch: both biological sources, the self-touch and the touch of one other individual, elicited a significant increase in motor system activity when compared to touching a non-living surface or in the absence of touch. A strong correlation between individual touch avoidance questionnaire values and facilitation in the motor system was present while touching another person, indicating a social aspect for touch in action. The present study unveils the motor system correlates when the sensory/motor components of touch are considered in social contexts.

Decoding the Spike-Band Subthreshold Motor Cortical Activity

Intracortical Brain-Computer Interfaces (iBCI) use single-unit activity (SUA), multiunit activity (MUA) and local field potentials (LFP) to control neuroprosthetic devices. SUA and MUA are usually extracted from the bandpassed recording through amplitude thresholding, while subthreshold data are ignored. Here, we show that subthreshold data can actually be decoded to determine behavioral variables with test set accuracy of up to 100%. Although the utility of SUA, MUA and LFP for decoding behavioral variables has been explored previously, this study investigates the utility of spike-band subthreshold activity exclusively. We provide evidence suggesting that this activity can be used to keep decoding performance at acceptable levels even when SUA quality is reduced over time. To the best of our knowledge, the signals that we derive from the subthreshold activity may be the weakest neural signals that have ever been extracted from extracellular neural recordings, while still being decodable with test set accuracy of up to 100%. These results are relevant for the development of fully data-driven and automated methods for amplitude thresholding spike-band extracellular neural recordings in iBCIs containing thousands of electrodes.

F = ma. Is the macaque brain Newtonian?

Intuitive Physics, the ability to anticipate how the physical events involving mass objects unfold in time and space, is a central component of intelligent systems. Intuitive physics is a promising tool for gaining insight into mechanisms that generalize across species because both humans and non-human primates are subject to the same physical constraints when engaging with the environment. Physical reasoning abilities are widely present within the animal kingdom, but monkeys, with acute 3D vision and a high level of dexterity, appreciate and manipulate the physical world in much the same way humans do.

The neural mechanisms of manual dexterity

The hand endows us with unparalleled precision and versatility in our interactions with objects, from mundane activities such as grasping to extraordinary ones such as virtuoso pianism. The complex anatomy of the human hand combined with expansive and specialized neuronal control circuits allows a wide range of precise manual behaviours. To support these behaviours, an exquisite sensory apparatus, spanning the modalities of touch and proprioception, conveys detailed and timely information about our interactions with objects and about the objects themselves. The study of manual dexterity provides a unique lens into the sensorimotor mechanisms that endow the nervous system with the ability to flexibly generate complex behaviour.

Generalizable cursor click decoding using grasp-related neural transients

Objective.Intracortical brain-computer interfaces (iBCI) have the potential to restore independence for individuals with significant motor or communication impairments. One of the most realistic avenues for clinical translation of iBCI technology is enabling control of a computer cursor-i.e. movement-related neural activity is interpreted (decoded) and used to drive cursor function. Here we aim to improve cursor click decoding to allow for both point-and-click and click-and-drag control.Approach.Using chronic microelectrode arrays implanted in the motor cortex of two participants with tetraplegia, we identified prominent neural responses related to attempted hand grasp. We then developed a new approach for decoding cursor click (hand grasp) based on the most salient responses.Main results.We found that the population-wide response contained three dominant components related to hand grasp: an onset transient response, a sustained response, and an offset transient response. The transient responses were larger in magnitude-and thus more reliably detected-than the sustained response, and a click decoder based on these transients outperformed the standard approach of binary state classification.Significance.A transient-based approach for identifying hand grasp can provide a high degree of cursor click control for both point-and-click and click-and-drag applications. This generalized click functionality is an important step toward high-performance cursor control and eventual clinical translation of iBCI technology.

A Turntable Setup for Testing Visual and Tactile Grasping Movements in Non-human Primates

Grasping movements are some of the most common movements primates do every day. They are important for social interactions as well as picking up objects or food. Usually, these grasping movements are guided by vision but proprioceptive and haptic inputs contribute greatly. Since grasping behaviors are common and easy to motivate, they represent an ideal task for understanding the role of different brain areas during planning and execution of complex voluntary movements in primates. For experimental purposes, a stable and repeatable presentation of the same object as well as the variation of objects is important in order to understand the neural control of movement generation. This is even more the case when investigating the role of different senses for movement planning, where objects need to be presented in specific sensory modalities. We developed a turntable setup for non-human primates (macaque monkeys) to investigate visually and tactually guided grasping movements with an option to easily exchange objects. The setup consists of a turntable that can fit six different objects and can be exchanged easily during the experiment to increase the number of presented objects. The object turntable is connected to a stepper motor through a belt system to automate rotation and hence object presentation. By increasing the distance between the turntable and the stepper motor, metallic components of the stepper motor are kept at a distance to the actual recording setup, which allows using a magnetic-based data glove to track hand kinematics. During task execution, the animal sits in the dark and is instructed to grasp the object in front of it. Options to turn on a light above the object allow for visual presentation of the objects, while the object can also remain in the dark for exclusive tactile exploration. A red LED is projected onto the object by a one-way mirror that serves as a grasp cue instruction for the animal to start grasping the object. By comparing kinematic data from the magnetic-based data glove with simultaneously recorded neural signals, this setup enables the systematic investigation of neural population activity involved in the neural control of hand grasping movements.

Stereopsis contributes to the predictive control of grip forces during prehension

Binocular viewing is associated with a superior prehensile performance, which is particularly evident in the latter part of the reach as the hand approaches and makes contact with the target object. However, the visuomotor mechanisms through which binocular vision serves prehension are not fully understood. This study assessed the role of stereopsis in the predictive control of grasping by measuring grip force. Twenty participants performed a precision reach-to-grasp task in four viewing conditions: binocular, monocular, and with reduced stereoacuity (200 arc sec, > 400 arc sec). Monocular, compared to binocular viewing, was associated with a fourfold increase in grasp errors, a 56% increase in grasp duration, 22% decrease in grip force at 50 ms following grasp initiation, and the time of peak force occurred 40% later after grasp initiation (all p < 0.05). Grasp performance was also disrupted when viewing with reduced stereoacuity. Notably, grip force at the time of object lift-off was comparable between all viewing conditions. These results demonstrate that binocular stereopsis contributes to the efficient programming of grip forces. Specifically, stereopsis may provide important sensory information that enables the central nervous system to engage in predictive control of grasping.

The Neural Representation of Force across Grasp Types in Motor Cortex of Humans with Tetraplegia

Intracortical brain-computer interfaces (iBCIs) have the potential to restore hand grasping and object interaction to individuals with tetraplegia. Optimal grasping and object interaction require simultaneous production of both force and grasp outputs. However, since overlapping neural populations are modulated by both parameters, grasp type could affect how well forces are decoded from motor cortex in a closed-loop force iBCI. Therefore, this work quantified the neural representation and offline decoding performance of discrete hand grasps and force levels in two human participants with tetraplegia. Participants attempted to produce three discrete forces (light, medium, hard) using up to five hand grasp configurations. A two-way Welch ANOVA was implemented on multiunit neural features to assess their modulation to force and grasp Demixed principal component analysis (dPCA) was used to assess for population-level tuning to force and grasp and to predict these parameters from neural activity. Three major findings emerged from this work: (1) force information was neurally represented and could be decoded across multiple hand grasps (and, in one participant, across attempted elbow extension as well); (2) grasp type affected force representation within multiunit neural features and offline force classification accuracy; and (3) grasp was classified more accurately and had greater population-level representation than force. These findings suggest that force and grasp have both independent and interacting representations within cortex, and that incorporating force control into real-time iBCI systems is feasible across multiple hand grasps if the decoder also accounts for grasp type.

Shared functional connectivity between the dorso-medial and dorso-ventral streams in macaques

Manipulation of an object requires us to transport our hand towards the object (reach) and close our digits around that object (grasp). In current models, reach-related information is propagated in the dorso-medial stream from posterior parietal area V6A to medial intraparietal area, dorsal premotor cortex, and primary motor cortex. Grasp-related information is processed in the dorso-ventral stream from the anterior intraparietal area to ventral premotor cortex and the hand area of primary motor cortex. However, recent studies have cast doubt on the validity of this separation in separate processing streams. We investigated in 10 male rhesus macaques the whole-brain functional connectivity of these areas using resting state fMRI at 7-T. Although we found a clear separation between dorso-medial and dorso-ventral network connectivity in support of the two-stream hypothesis, we also found evidence of shared connectivity between these networks. The dorso-ventral network was distinctly correlated with high-order somatosensory areas and feeding related areas, whereas the dorso-medial network with visual areas and trunk/hindlimb motor areas. Shared connectivity was found in the superior frontal and precentral gyrus, central sulcus, intraparietal sulcus, precuneus, and insular cortex. These results suggest that while sensorimotor processing streams are functionally separated, they can access information through shared areas.

The Representation of Finger Movement and Force in Human Motor and Premotor Cortices

The ability to grasp and manipulate objects requires controlling both finger movement kinematics and isometric force in rapid succession. Previous work suggests that these behavioral modes are controlled separately, but it is unknown whether the cerebral cortex represents them differently. Here, we asked the question of how movement and force were represented cortically, when executed sequentially with the same finger. We recorded high-density electrocorticography (ECoG) from the motor and premotor cortices of seven human subjects performing a movement-force motor task. We decoded finger movement [0.7 ± 0.3 fractional variance accounted for (FVAF)] and force (0.7 ± 0.2 FVAF) with high accuracy, yet found different spatial representations. In addition, we used a state-of-the-art deep learning method to uncover smooth, repeatable trajectories through ECoG state space during the movement-force task. We also summarized ECoG across trials and participants by developing a new metric, the neural vector angle (NVA). Thus, state-space techniques can help to investigate broad cortical networks. Finally, we were able to classify the behavioral mode from neural signals with high accuracy (90 ± 6%). Thus, finger movement and force appear to have distinct representations in motor/premotor cortices. These results inform our understanding of the neural control of movement, as well as the design of grasp brain-machine interfaces (BMIs).

Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis

Speaking is a sensorimotor behavior whose neural basis is difficult to study with single neuron resolution due to the scarcity of human intracortical measurements. We used electrode arrays to record from the motor cortex 'hand knob' in two people with tetraplegia, an area not previously implicated in speech. Neurons modulated during speaking and during non-speaking movements of the tongue, lips, and jaw. This challenges whether the conventional model of a 'motor homunculus' division by major body regions extends to the single-neuron scale. Spoken words and syllables could be decoded from single trials, demonstrating the potential of intracortical recordings for brain-computer interfaces to restore speech. Two neural population dynamics features previously reported for arm movements were also present during speaking: a component that was mostly invariant across initiating different words, followed by rotatory dynamics during speaking. This suggests that common neural dynamical motifs may underlie movement of arm and speech articulators.

The Concurrent Control of Motion and Contact Force in the Presence of Predictable Disturbances

The simultaneous control of force and motion is important in everyday activities when humans interact with objects. While many studies have analyzed the control of movement within a perturbing force field, few have investigated its dual aspects of controlling a contact force in nonisometric conditions. The mechanism by which the central nervous system controls forces during movements is still unclear, and it can be elucidated by estimating the mechanical properties of the arm during tasks with concurrent motion and contact force goals. We investigate how arm mechanics change when a force control task is accomplished during low-frequency positional perturbations of the arm. Contrary to many force regulation algorithms implemented in robotics, where contact impedance is decreased to reduce force fluctuations in response to position disturbances, we observed a steady increase of arm endpoint stiffness as the task progressed. Based on this evidence, we propose a theoretical framework suggesting that an internal model of the perturbing trajectory is formed. We observed that force regulation in the presence of predictable positional disturbances is implemented using a position control strategy together with the modulation of the endpoint stiffness magnitude, where the direction of the endpoint stiffness ellipse's major axis is oriented toward the desired force.

Multisensory Neurons in the Primate Amygdala

Animals identify, interpret, and respond to complex, natural signals that are often multisensory. The ability to integrate signals across sensory modalities depends on the convergence of sensory inputs at the level of single neurons. Neurons in the amygdala are expected to be multisensory because they respond to complex, natural stimuli, and the amygdala receives inputs from multiple sensory areas. We recorded activity from the amygdala of 2 male monkeys (Macaca mulatta) in response to visual, tactile, and auditory stimuli. Although the stimuli were devoid of inherent emotional or social significance and were not paired with rewards or punishments, the majority of neurons that responded to these stimuli were multisensory. Selectivity for sensory modality was stronger and emerged earlier than selectivity for individual items within a sensory modality. Modality and item selectivity were expressed via three main spike-train metrics: (1) response magnitude, (2) response polarity, and (3) response duration. None of these metrics were unique to a particular sensory modality; rather, each neuron responded with distinct combinations of spike-train metrics to discriminate sensory modalities and items within a modality. The relative proportion of multisensory neurons was similar across the nuclei of the amygdala. The convergence of inputs of multiple sensory modalities at the level of single neurons in the amygdala rests at the foundation for multisensory integration. The integration of visual, auditory, and tactile inputs in the amygdala may serve social communication by binding together social signals carried by facial expressions, vocalizations, and social grooming.SIGNIFICANCE STATEMENT Our brain continuously decodes information detected by multiple sensory systems. The emotional and social significance of the incoming signals is likely extracted by the amygdala, which receives input from all sensory domains. Here we show that a large portion of neurons in the amygdala respond to stimuli from two or more sensory modalities. The convergence of visual, tactile, and auditory signals at the level of individual neurons in the amygdala establishes a foundation for multisensory integration within this structure. The ability to integrate signals across sensory modalities is critical for social communication and other high-level cognitive functions.