Area 2 of primary somatosensory cortex encodes kinematics of the whole arm

Dissociative and prioritized modeling of behaviorally relevant neural dynamics using recurrent neural networks

Understanding the dynamical transformation of neural activity to behavior requires new capabilities to nonlinearly model, dissociate and prioritize behaviorally relevant neural dynamics and test hypotheses about the origin of nonlinearity. We present dissociative prioritized analysis of dynamics (DPAD), a nonlinear dynamical modeling approach that enables these capabilities with a multisection neural network architecture and training approach. Analyzing cortical spiking and local field potential activity across four movement tasks, we demonstrate five use-cases. DPAD enabled more accurate neural-behavioral prediction. It identified nonlinear dynamical transformations of local field potentials that were more behavior predictive than traditional power features. Further, DPAD achieved behavior-predictive nonlinear neural dimensionality reduction. It enabled hypothesis testing regarding nonlinearities in neural-behavioral transformation, revealing that, in our datasets, nonlinearities could largely be isolated to the mapping from latent cortical dynamics to behavior. Finally, DPAD extended across continuous, intermittently sampled and categorical behaviors. DPAD provides a powerful tool for nonlinear dynamical modeling and investigation of neural-behavioral data.

Biases in hand perception are driven by somatosensory computations, not a distorted hand model

To sense and interact with objects in the environment, we effortlessly configure our fingertips at desired locations. It is therefore reasonable to assume that the underlying control mechanisms rely on accurate knowledge about the structure and spatial dimensions of our hand and fingers. This intuition, however, is challenged by years of research showing drastic biases in the perception of finger geometry.1,2,3,4,5 This perceptual bias has been taken as evidence that the brain's internal representation of the body's geometry is distorted,6 leading to an apparent paradox regarding the skillfulness of our actions.7 Here, we propose an alternative explanation of the biases in hand perception-they are the result of the Bayesian integration of noisy, but unbiased, somatosensory signals about finger geometry and posture. To address this hypothesis, we combined Bayesian reverse engineering with behavioral experimentation on joint and fingertip localization of the index finger. We modeled the Bayesian integration either in sensory or in space-based coordinates, showing that the latter model variant led to biases in finger perception despite accurate representation of finger length. Behavioral measures of joint and fingertip localization responses showed similar biases, which were well fitted by the space-based, but not the sensory-based, model variant. The space-based model variant also outperformed a distorted hand model with built-in geometric biases. In total, our results suggest that perceptual distortions of finger geometry do not reflect a distorted hand model but originate from near-optimal Bayesian inference on somatosensory signals.

Modeling proprioception with task-driven neural network models

In a recent issue of Cell, Vargas and colleagues1 demonstrate that task-driven neural network models are superior at predicting proprioceptive activity in the primate cuneate nucleus and sensorimotor cortex compared with other models. This provides valuable insights for better understanding the proprioceptive pathway.

Task-driven neural network models predict neural dynamics of proprioception

Proprioception tells the brain the state of the body based on distributed sensory neurons. Yet, the principles that govern proprioceptive processing are poorly understood. Here, we employ a task-driven modeling approach to investigate the neural code of proprioceptive neurons in cuneate nucleus (CN) and somatosensory cortex area 2 (S1). We simulated muscle spindle signals through musculoskeletal modeling and generated a large-scale movement repertoire to train neural networks based on 16 hypotheses, each representing different computational goals. We found that the emerging, task-optimized internal representations generalize from synthetic data to predict neural dynamics in CN and S1 of primates. Computational tasks that aim to predict the limb position and velocity were the best at predicting the neural activity in both areas. Since task optimization develops representations that better predict neural activity during active than passive movements, we postulate that neural activity in the CN and S1 is top-down modulated during goal-directed movements.

μSim: A goal-driven framework for elucidating the neural control of movement through musculoskeletal modeling

How does the motor cortex (MC) produce purposeful and generalizable movements from the complex musculoskeletal system in a dynamic environment? To elucidate the underlying neural dynamics, we use a goal-driven approach to model MC by considering its goal as a controller driving the musculoskeletal system through desired states to achieve movement. Specifically, we formulate the MC as a recurrent neural network (RNN) controller producing muscle commands while receiving sensory feedback from biologically accurate musculoskeletal models. Given this real-time simulated feedback implemented in advanced physics simulation engines, we use deep reinforcement learning to train the RNN to achieve desired movements under specified neural and musculoskeletal constraints. Activity of the trained model can accurately decode experimentally recorded neural population dynamics and single-unit MC activity, while generalizing well to testing conditions significantly different from training. Simultaneous goal- and data- driven modeling in which we use the recorded neural activity as observed states of the MC further enhances direct and generalizable single-unit decoding. Finally, we show that this framework elucidates computational principles of how neural dynamics enable flexible control of movement and make this framework easy-to-use for future experiments.

Dynamical flexible inference of nonlinear latent factors and structures in neural population activity

Modelling the spatiotemporal dynamics in the activity of neural populations while also enabling their flexible inference is hindered by the complexity and noisiness of neural observations. Here we show that the lower-dimensional nonlinear latent factors and latent structures can be computationally modelled in a manner that allows for flexible inference causally, non-causally and in the presence of missing neural observations. To enable flexible inference, we developed a neural network that separates the model into jointly trained manifold and dynamic latent factors such that nonlinearity is captured through the manifold factors and the dynamics can be modelled in tractable linear form on this nonlinear manifold. We show that the model, which we named 'DFINE' (for 'dynamical flexible inference for nonlinear embeddings') achieves flexible inference in simulations of nonlinear dynamics and across neural datasets representing a diversity of brain regions and behaviours. Compared with earlier neural-network models, DFINE enables flexible inference, better predicts neural activity and behaviour, and better captures the latent neural manifold structure. DFINE may advance the development of neurotechnology and investigations in neuroscience.

Decoding hand kinetics and kinematics using somatosensory cortex activity in active and passive movement

Area 2 of the primary somatosensory cortex (S1), encodes proprioceptive information of limbs. Several studies investigated the encoding of movement parameters in this area. However, the single-trial decoding of these parameters, which can provide additional knowledge about the amount of information available in sub-regions of this area about instantaneous limb movement, has not been well investigated. We decoded kinematic and kinetic parameters of active and passive hand movement during center-out task using conventional and state-based decoders. Our results show that this area can be used to accurately decode position, velocity, force, moment, and joint angles of hand. Kinematics had higher accuracies compared to kinetics and active trials were decoded more accurately than passive trials. Although the state-based decoder outperformed the conventional decoder in the active task, it was the opposite in the passive task. These results can be used in intracortical micro-stimulation procedures to provide proprioceptive feedback to BCI subjects.

Contrasting action and posture coding with hierarchical deep neural network models of proprioception

Biological motor control is versatile, efficient, and depends on proprioceptive feedback. Muscles are flexible and undergo continuous changes, requiring distributed adaptive control mechanisms that continuously account for the body's state. The canonical role of proprioception is representing the body state. We hypothesize that the proprioceptive system could also be critical for high-level tasks such as action recognition. To test this theory, we pursued a task-driven modeling approach, which allowed us to isolate the study of proprioception. We generated a large synthetic dataset of human arm trajectories tracing characters of the Latin alphabet in 3D space, together with muscle activities obtained from a musculoskeletal model and model-based muscle spindle activity. Next, we compared two classes of tasks: trajectory decoding and action recognition, which allowed us to train hierarchical models to decode either the position and velocity of the end-effector of one's posture or the character (action) identity from the spindle firing patterns. We found that artificial neural networks could robustly solve both tasks, and the networks' units show tuning properties similar to neurons in the primate somatosensory cortex and the brainstem. Remarkably, we found uniformly distributed directional selective units only with the action-recognition-trained models and not the trajectory-decoding-trained models. This suggests that proprioceptive encoding is additionally associated with higher-level functions such as action recognition and therefore provides new, experimentally testable hypotheses of how proprioception aids in adaptive motor control.

Learnable latent embeddings for joint behavioural and neural analysis

Mapping behavioural actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioural data increases, there is growing interest in modelling neural dynamics during adaptive behaviours to probe neural representations1-3. In particular, although neural latent embeddings can reveal underlying correlates of behaviour, we lack nonlinear techniques that can explicitly and flexibly leverage joint behaviour and neural data to uncover neural dynamics3-5. Here, we fill this gap with a new encoding method, CEBRA, that jointly uses behavioural and neural data in a (supervised) hypothesis- or (self-supervised) discovery-driven manner to produce both consistent and high-performance latent spaces. We show that consistency can be used as a metric for uncovering meaningful differences, and the inferred latents can be used for decoding. We validate its accuracy and demonstrate our tool's utility for both calcium and electrophysiology datasets, across sensory and motor tasks and in simple or complex behaviours across species. It allows leverage of single- and multi-session datasets for hypothesis testing or can be used label free. Lastly, we show that CEBRA can be used for the mapping of space, uncovering complex kinematic features, for the production of consistent latent spaces across two-photon and Neuropixels data, and can provide rapid, high-accuracy decoding of natural videos from visual cortex.

Peripersonal encoding of forelimb proprioception in the mouse somatosensory cortex

Conscious perception of limb movements depends on proprioceptive neural responses in the somatosensory cortex. In contrast to tactile sensations, proprioceptive cortical coding is barely studied in the mammalian brain and practically non-existent in rodent research. To understand the cortical representation of this important sensory modality we developed a passive forelimb displacement paradigm in behaving mice and also trained them to perceptually discriminate where their limb is moved in space. We delineated the rodent proprioceptive cortex with wide-field calcium imaging and optogenetic silencing experiments during behavior. Our results reveal that proprioception is represented in both sensory and motor cortical areas. In addition, behavioral measurements and responses of layer 2/3 neurons imaged with two-photon microscopy reveal that passive limb movements are both perceived and encoded in the mouse cortex as a spatial direction vector that interfaces the limb with the body's peripersonal space.

Dynamical flexible inference of nonlinear latent structures in neural population activity

Inferring complex spatiotemporal dynamics in neural population activity is critical for investigating neural mechanisms and developing neurotechnology. These activity patterns are noisy observations of lower-dimensional latent factors and their nonlinear dynamical structure. A major unaddressed challenge is to model this nonlinear structure, but in a manner that allows for flexible inference, whether causally, non-causally, or in the presence of missing neural observations. We address this challenge by developing DFINE, a new neural network that separates the model into dynamic and manifold latent factors, such that the dynamics can be modeled in tractable form. We show that DFINE achieves flexible nonlinear inference across diverse behaviors and brain regions. Further, despite enabling flexible inference unlike prior neural network models of population activity, DFINE also better predicts the behavior and neural activity, and better captures the latent neural manifold structure. DFINE can both enhance future neurotechnology and facilitate investigations across diverse domains of neuroscience.

A large-scale neural network training framework for generalized estimation of single-trial population dynamics

Achieving state-of-the-art performance with deep neural population dynamics models requires extensive hyperparameter tuning for each dataset. AutoLFADS is a model-tuning framework that automatically produces high-performing autoencoding models on data from a variety of brain areas and tasks, without behavioral or task information. We demonstrate its broad applicability on several rhesus macaque datasets: from motor cortex during free-paced reaching, somatosensory cortex during reaching with perturbations, and dorsomedial frontal cortex during a cognitive timing task.

Corticospinal neurons from motor and somatosensory cortices exhibit different temporal activity dynamics during motor learning

The ability to learn motor skills implicates an improvement in accuracy, speed and consistency of movements. Motor control is related to movement execution and involves corticospinal neurons (CSp), which are broadly distributed in layer 5B of the motor and somatosensory cortices. CSp neurons innervate the spinal cord and are functionally diverse. However, whether CSp activity differs between different cortical areas throughout motor learning has been poorly explored. Given the importance and interaction between primary motor (M1) and somatosensory (S1) cortices related to movement, we examined the functional roles of CSp neurons in both areas. We induced the expression of GCaMP7s calcium indicator to perform photometric calcium recordings from layer 5B CSp neurons simultaneously in M1 and S1 cortices and track their activity while adult mice learned and performed a cued lever-press task. We found that during early learning sessions, the population calcium activity of CSp neurons in both cortices during movement did not change significantly. In late learning sessions the peak amplitude and duration of calcium activity CSp neurons increased in both, M1 and S1 cortices. However, S1 and M1 CSp neurons display a different temporal dynamic during movements that occurred when animals learned the task; both M1 and S1 CSp neurons activate before movement initiation, however, M1 CSp neurons continue active during movement performance, reinforcing the idea of the diversity of the CSp system and suggesting that CSp neuron activity in M1 and S1 cortices throughout motor learning have different functional roles for sensorimotor integration.

Local field potentials reflect cortical population dynamics in a region-specific and frequency-dependent manner

The spiking activity of populations of cortical neurons is well described by the dynamics of a small number of population-wide covariance patterns, the 'latent dynamics'. These latent dynamics are largely driven by the same correlated synaptic currents across the circuit that determine the generation of local field potentials (LFP). Yet, the relationship between latent dynamics and LFPs remains largely unexplored. Here, we characterised this relationship for three different regions of primate sensorimotor cortex during reaching. The correlation between latent dynamics and LFPs was frequency-dependent and varied across regions. However, for any given region, this relationship remained stable throughout the behaviour: in each of primary motor and premotor cortices, the LFP-latent dynamics correlation profile was remarkably similar between movement planning and execution. These robust associations between LFPs and neural population latent dynamics help bridge the wealth of studies reporting neural correlates of behaviour using either type of recordings.

Neural dynamics of illusory tactile pulling sensations

Directional tactile pulling sensations are integral to everyday life, but their neural mechanisms remain unknown. Prior accounts hold that primary somatosensory (SI) activity is sufficient to generate pulling sensations, with alternative proposals suggesting that amodal frontal or parietal regions may be critical. We combined high-density EEG with asymmetric vibration, which creates an illusory pulling sensation, thereby unconfounding pulling sensations from unrelated sensorimotor processes. Oddballs that created opposite direction pulls to common stimuli were compared to the same oddballs after neutral common stimuli (symmetric vibration) and to neutral oddballs. We found evidence against the sensory-frontal N140 and in favor of the midline P200 tracking the emergence of pulling sensations, specifically contralateral parietal lobe activity 264-320ms, centered on the intraparietal sulcus. This suggests that SI is not sufficient to generate pulling sensations, which instead depend on the parietal association cortex, and may reflect the extraction of orientation information and related spatial processing.

•

Tactile pulling sensations are difficult to isolate in the human brain

•

Illusory pulls from asymmetric vibration allow neural activity to be isolated

•

Pulling sensations are driven by parietal lobe activity 264-320ms post-stimulus

•

Spatial processing in the parietal lobe may be essential for pulling sensations

Recurrent neural networks controlling musculoskeletal models predict motor cortex activity during novel limb movements

Goal-driven networks trained to perform a task analogous to that performed by biological neural populations are being increasingly utilized as insightful computational models of motor control. The resulting dynamics of the trained networks are then analyzed to uncover the neural strategies employed by the motor cortex to produce movements. However, these networks do not take into account the role of sensory feedback in producing movement, nor do they consider the complex biophysical underpinnings of the underlying musculoskeletal system. Moreover, these models can not be used in context of predictive neuromechanical simulations for hypothesis generation and prediction of neural strategies during novel movements. In this research, we adapt state-of-the-art deep reinforcement learning (DRL) algorithms to train a controller to drive a developed anatomically accurate monkey arm model to track experimentally recorded kinematics. We validate that the trained controller mimics biologically observed neural strategies to produce movement. The trained controller generalizes well to unobserved conditions as well as to perturbation analyses. The recorded firing rates of motor cortex neurons can be predicted from the controller activity with high accuracy even on unseen conditions. Finally, we validate that the trained controller outperforms existing goal-driven and representational models of motor cortex in single neuron decoding accuracy, thus showing the utility of the complex underpinnings of anatomically accurate models in shaping motor cortex neural activity during limb movements. The learned controller can be used for hypothesis generation and prediction of neural strategies during novel movements and unobserved conditions.

Grasp-squeeze adaptation to changes in object compliance leads to dynamic beta-band communication between primary somatosensory and motor cortices

In asking the question of how the brain adapts to changes in the softness of manipulated objects, we studied dynamic communication between the primary sensory and motor cortical areas when nonhuman primates grasp and squeeze an elastically deformable manipulandum to attain an instructed force level. We focused on local field potentials recorded from S1 and M1 via intracortical microelectrode arrays. We computed nonparametric spectral Granger Causality to assess directed cortico-cortical interactions between these two areas. We demonstrate that the time-causal relationship between M1 and S1 is bidirectional in the beta-band (15-30 Hz) and that this interareal communication develops dynamically as the subjects adjust the force of hand squeeze to reach the target level. In particular, the directed interaction is strongest when subjects are focused on maintaining the instructed force of hand squeeze in a steady state for several seconds. When the manipulandum's compliance is abruptly changed, beta-band interareal communication is interrupted for a short period (~ 1 s) and then is re-established once the subject has reached a new steady state. These results suggest that transient beta oscillations can provide a communication subspace for dynamic cortico-cortical S1-M1 interactions during maintenance of steady sensorimotor states.

Contralesional Sensorimotor Network Participates in Motor Functional Compensation in Glioma Patients

Background

Some gliomas in sensorimotor areas induce motor deficits, while some do not. Cortical destruction and reorganization contribute to this phenomenon, but detailed reasons remain unclear. This study investigated the differences of the functional connectivity and topological properties in the contralesional sensorimotor network (cSMN) between patients with motor deficit and those with normal motor function.

Methods

We retrospectively reviewed 65 patients (32 men) between 2017 and 2020. The patients were divided into four groups based on tumor laterality and preoperative motor status (deficit or non-deficit). Thirty-three healthy controls (18 men) were enrolled after matching for sex, age, and educational status. Graph theoretical measurement was applied to reveal alterations of the topological properties of the cSMN by analyzing resting-state functional MRI.

Results

The results for patients with different hemispheric gliomas were similar. The clustering coefficient, local efficiency, transitivity, and vulnerability of the cSMN significantly increased in the non-deficit group and decreased in the deficit group compared to the healthy group (p < 0.05). Moreover, the nodes of the motor-related thalamus showed a significantly increased nodal efficiency and nodal local efficiency in the non-deficit group and decreased in the deficit group compared with the healthy group (p < 0.05).

Conclusions

We posited the existence of two stages of alterations of the preoperative motor status. In the compensatory stage, the cSMN sacrificed stability to acquire high efficiency and to compensate for impaired motor function. With the glioma growing and the motor function being totally damaged, the cSMN returned to a stable state and maintained healthy hemispheric motor function, but with low efficiency.

The science and engineering behind sensitized brain-controlled bionic hands

Advances in our understanding of brain function, along with the development of neural interfaces that allow for the monitoring and activation of neurons, have paved the way for brain-machine interfaces (BMIs), which harness neural signals to reanimate the limbs via electrical activation of the muscles or to control extracorporeal devices, thereby bypassing the muscles and senses altogether. BMIs consist of reading out motor intent from the neuronal responses monitored in motor regions of the brain and executing intended movements with bionic limbs, reanimated limbs, or exoskeletons. BMIs also allow for the restoration of the sense of touch by electrically activating neurons in somatosensory regions of the brain, thereby evoking vivid tactile sensations and conveying feedback about object interactions. In this review, we discuss the neural mechanisms of motor control and somatosensation in able-bodied individuals and describe approaches to use neuronal responses as control signals for movement restoration and to activate residual sensory pathways to restore touch. Although the focus of the review is on intracortical approaches, we also describe alternative signal sources for control and noninvasive strategies for sensory restoration.

Similar tactile distance anisotropy across segments of the arm

A substantial literature has described anisotropy of tactile distance perception across many body parts. In general, the distance between two touches is felt as larger when the touches are oriented with the mediolateral axis of the limbs than when oriented with the proximodistal axis. In this study, we investigated tactile distance perception across the arm, measuring anisotropy on the upper arm, forearm, and hand dorsum. Participants made forced-choice judgments of which of two pairs of tactile distances felt larger and anisotropy was measured using the method of constant stimuli. Clear anisotropy was found on all three regions of the arm. There was no apparent difference in the magnitude of anisotropy across segments of the arm. We further measured the physical curvature of the arm and show that this cannot account of the perceptual anisotropy observed.

Metastable dynamics of neural circuits and networks

Cortical neurons emit seemingly erratic trains of action potentials or "spikes," and neural network dynamics emerge from the coordinated spiking activity within neural circuits. These rich dynamics manifest themselves in a variety of patterns, which emerge spontaneously or in response to incoming activity produced by sensory inputs. In this Review, we focus on neural dynamics that is best understood as a sequence of repeated activations of a number of discrete hidden states. These transiently occupied states are termed "metastable" and have been linked to important sensory and cognitive functions. In the rodent gustatory cortex, for instance, metastable dynamics have been associated with stimulus coding, with states of expectation, and with decision making. In frontal, parietal, and motor areas of macaques, metastable activity has been related to behavioral performance, choice behavior, task difficulty, and attention. In this article, we review the experimental evidence for neural metastable dynamics together with theoretical approaches to the study of metastable activity in neural circuits. These approaches include (i) a theoretical framework based on non-equilibrium statistical physics for network dynamics; (ii) statistical approaches to extract information about metastable states from a variety of neural signals; and (iii) recent neural network approaches, informed by experimental results, to model the emergence of metastable dynamics. By discussing these topics, we aim to provide a cohesive view of how transitions between different states of activity may provide the neural underpinnings for essential functions such as perception, memory, expectation, or decision making, and more generally, how the study of metastable neural activity may advance our understanding of neural circuit function in health and disease.

Representation Learning for Dynamic Functional Connectivities via Variational Dynamic Graph Latent Variable Models

Latent variable models (LVMs) for neural population spikes have revealed informative low-dimensional dynamics about the neural data and have become powerful tools for analyzing and interpreting neural activity. However, these approaches are unable to determine the neurophysiological meaning of the inferred latent dynamics. On the other hand, emerging evidence suggests that dynamic functional connectivities (DFC) may be responsible for neural activity patterns underlying cognition or behavior. We are interested in studying how DFC are associated with the low-dimensional structure of neural activities. Most existing LVMs are based on a point process and fail to model evolving relationships. In this work, we introduce a dynamic graph as the latent variable and develop a Variational Dynamic Graph Latent Variable Model (VDGLVM), a representation learning model based on the variational information bottleneck framework. VDGLVM utilizes a graph generative model and a graph neural network to capture dynamic communication between nodes that one has no access to from the observed data. The proposed computational model provides guaranteed behavior-decoding performance and improves LVMs by associating the inferred latent dynamics with probable DFC.

Motor impairment evoked by direct electrical stimulation of human parietal cortex during object manipulation

In primates, the parietal cortex plays a crucial role in hand-object manipulation. However, its involvement in object manipulation and related hand-muscle control has never been investigated in humans with a direct and focal electrophysiological approach. To this aim, during awake surgery for brain tumors, we studied the impact of direct electrical stimulation (DES) of parietal lobe on hand-muscles during a hand-manipulation task (HMt). Results showed that DES applied to fingers-representation of postcentral gyrus (PCG) and anterior intraparietal cortex (aIPC) impaired HMt execution. Different types of EMG-interference patterns were observed ranging from a partial (task-clumsy) or complete (task-arrest) impairment of muscles activity. Within PCG both patterns coexisted along a medio (arrest)-lateral (clumsy) distribution, while aIPC hosted preferentially the task-arrest. The interference patterns were mainly associated to muscles suppression, more pronounced in aIPC with respect to PCG. Moreover, within PCG were observed patterns with different level of muscle recruitment, not reported in the aIPC. Overall, EMG-interference patterns and their probabilistic distribution suggested the presence of different functional parietal sectors, possibly playing different roles in hand-muscle control during manipulation. We hypothesized that task-arrest, compared to clumsy patterns, might suggest the existence of parietal sectors more closely implicated in shaping the motor output.

Rotational dynamics in motor cortex are consistent with a feedback controller

Recent studies have identified rotational dynamics in motor cortex (MC), which many assume arise from intrinsic connections in MC. However, behavioral and neurophysiological studies suggest that MC behaves like a feedback controller where continuous sensory feedback and interactions with other brain areas contribute substantially to MC processing. We investigated these apparently conflicting theories by building recurrent neural networks that controlled a model arm and received sensory feedback from the limb. Networks were trained to counteract perturbations to the limb and to reach toward spatial targets. Network activities and sensory feedback signals to the network exhibited rotational structure even when the recurrent connections were removed. Furthermore, neural recordings in monkeys performing similar tasks also exhibited rotational structure not only in MC but also in somatosensory cortex. Our results argue that rotational structure may also reflect dynamics throughout the voluntary motor system involved in online control of motor actions.

Decoding Neural Activity in Sulcal and White Matter Areas of the Brain to Accurately Predict Individual Finger Movement and Tactile Stimuli of the Human Hand

Millions of people worldwide suffer motor or sensory impairment due to stroke, spinal cord injury, multiple sclerosis, traumatic brain injury, diabetes, and motor neuron diseases such as ALS (amyotrophic lateral sclerosis). A brain-computer interface (BCI), which links the brain directly to a computer, offers a new way to study the brain and potentially restore impairments in patients living with these debilitating conditions. One of the challenges currently facing BCI technology, however, is to minimize surgical risk while maintaining efficacy. Minimally invasive techniques, such as stereoelectroencephalography (SEEG) have become more widely used in clinical applications in epilepsy patients since they can lead to fewer complications. SEEG depth electrodes also give access to sulcal and white matter areas of the brain but have not been widely studied in brain-computer interfaces. Here we show the first demonstration of decoding sulcal and subcortical activity related to both movement and tactile sensation in the human hand. Furthermore, we have compared decoding performance in SEEG-based depth recordings versus those obtained with electrocorticography electrodes (ECoG) placed on gyri. Initial poor decoding performance and the observation that most neural modulation patterns varied in amplitude trial-to-trial and were transient (significantly shorter than the sustained finger movements studied), led to the development of a feature selection method based on a repeatability metric using temporal correlation. An algorithm based on temporal correlation was developed to isolate features that consistently repeated (required for accurate decoding) and possessed information content related to movement or touch-related stimuli. We subsequently used these features, along with deep learning methods, to automatically classify various motor and sensory events for individual fingers with high accuracy. Repeating features were found in sulcal, gyral, and white matter areas and were predominantly phasic or phasic-tonic across a wide frequency range for both HD (high density) ECoG and SEEG recordings. These findings motivated the use of long short-term memory (LSTM) recurrent neural networks (RNNs) which are well-suited to handling transient input features. Combining temporal correlation-based feature selection with LSTM yielded decoding accuracies of up to 92.04 ± 1.51% for hand movements, up to 91.69 ± 0.49% for individual finger movements, and up to 83.49 ± 0.72% for focal tactile stimuli to individual finger pads while using a relatively small number of SEEG electrodes. These findings may lead to a new class of minimally invasive brain-computer interface systems in the future, increasing its applicability to a wide variety of conditions.

Encoding of limb state by single neurons in the cuneate nucleus of awake monkeys

The cuneate nucleus (CN) is among the first sites along the neuraxis where proprioceptive signals can be integrated, transformed, and modulated. The objective of the study was to characterize the proprioceptive representations in CN. To this end, we recorded from single CN neurons in three monkeys during active reaching and passive limb perturbation. We found that many neurons exhibited responses that were tuned approximately sinusoidally to limb movement direction, as has been found for other sensorimotor neurons. The distribution of their preferred directions (PDs) was highly nonuniform and resembled that of muscle spindles within individual muscles, suggesting that CN neurons typically receive inputs from only a single muscle. We also found that the responses of proprioceptive CN neurons tended to be modestly amplified during active reaching movements compared to passive limb perturbations, in contrast to cutaneous CN neurons whose responses were not systematically different in the active and passive conditions. Somatosensory signals thus seem to be subject to a "spotlighting" of relevant sensory information rather than uniform suppression as has been suggested previously.NEW & NOTEWORTHY The cuneate nucleus (CN) is the somatosensory gateway into the brain, and only recently has it been possible to record these signals from an awake animal. We recorded single CN neurons in monkeys. Proprioceptive CN neurons appear to receive input from very few muscles, and their sensitivity to movement changes reliably during reaching relative to passive arm perturbations. Sensitivity is generally increased, but not exclusively so, as though CN "spotlights" critical proprioceptive information during reaching.

Visually and Tactually Guided Grasps Lead to Different Neuronal Activity in Non-human Primates

Movements are defining characteristics of all behaviors. Animals walk around, move their eyes to explore the world or touch structures to learn more about them. So far we only have some basic understanding of how the brain generates movements, especially when we want to understand how different areas of the brain interact with each other. In this study we investigated the influence of sensory object information on grasp planning in four different brain areas involved in vision, touch, movement planning, and movement generation in the parietal, somatosensory, premotor and motor cortex. We trained one monkey to grasp objects that he either saw or touched beforehand while continuously recording neural spiking activity with chronically implanted floating multi-electrode arrays. The animal was instructed to sit in the dark and either look at a shortly illuminated object or reach out and explore the object with his hand in the dark before lifting it up. In a first analysis we confirmed that the animal not only memorizes the object in both tasks, but also applies an object-specific grip type, independent of the sensory modality. In the neuronal population, we found a significant difference in the number of tuned units for sensory modalities during grasp planning that persisted into grasp execution. These differences were sufficient to enable a classifier to decode the object and sensory modality in a single trial exclusively from neural population activity. These results give valuable insights in how different brain areas contribute to the preparation of grasp movement and how different sensory streams can lead to distinct neural activity while still resulting in the same action execution.

Human Somatosensory Cortex Is Modulated during Motor Planning

Recent data and motor control theory argues that movement planning involves preparing the neural state of primary motor cortex (M1) for forthcoming action execution. Theories related to internal models, feedback control, and predictive coding also emphasize the importance of sensory prediction (and processing) before (and during) the movement itself, explaining why motor-related deficits can arise from damage to primary somatosensory cortex (S1). Motivated by this work, here we examined whether motor planning, in addition to changing the neural state of M1, changes the neural state of S1, preparing it for the sensory feedback that arises during action. We tested this idea in two human functional MRI studies (N = 31, 16 females) involving delayed object manipulation tasks, focusing our analysis on premovement activity patterns in M1 and S1. We found that the motor effector to be used in the upcoming action could be decoded, well before movement, from neural activity in M1 in both studies. Critically, we found that this effector information was also present, well before movement, in S1. In particular, we found that the encoding of effector information in area 3b (S1 proper) was linked to the contralateral hand, similarly to that found in M1, whereas in areas 1 and 2 this encoding was present in both the contralateral and ipsilateral hemispheres. Together, these findings suggest that motor planning not only prepares the motor system for movement but also changes the neural state of the somatosensory system, presumably allowing it to anticipate the sensory information received during movement.SIGNIFICANCE STATEMENT Whereas recent work on motor cortex has emphasized the critical role of movement planning in preparing neural activity for movement generation, it has not investigated the extent to which planning also modulates the activity in the adjacent primary somatosensory cortex. This reflects a key gap in knowledge, given that recent motor control theories emphasize the importance of sensory feedback processing in effective movement generation. Here, we find through a convergence of experiments and analyses, that the planning of object manipulation tasks, in addition to modulating the activity in the motor cortex, changes the state of neural activity in different subfields of the human S1. We suggest that this modulation prepares the S1 for the sensory information it will receive during action execution.

Assessing kinesthetic proprioceptive function of the upper limb: a novel dynamic movement reproduction task using a robotic arm

Background

Proprioception refers to the perception of motion and position of the body or body segments in space. A wide range of proprioceptive tests exists, although tests dynamically evaluating sensorimotor integration during upper limb movement are scarce. We introduce a novel task to evaluate kinesthetic proprioceptive function during complex upper limb movements using a robotic device. We aimed to evaluate the test–retest reliability of this newly developed Dynamic Movement Reproduction (DMR) task. Furthermore, we assessed reliability of the commonly used Joint Reposition (JR) task of the elbow, evaluated the association between both tasks, and explored the influence of visual information (viewing arm movement or not) on performance during both tasks.

Methods

During the DMR task, participants actively reproduced movement patterns while holding a handle attached to the robotic arm, with the device encoding actual position throughout movement. In the JR task, participants actively reproduced forearm positions; with the final arm position evaluated using an angle measurement tool. The difference between target movement pattern/position and reproduced movement pattern/position served as measures of accuracy. In study 1 (N = 23), pain-free participants performed both tasks at two test sessions, 24-h apart, both with and without visual information available (i.e., vision occluded using a blindfold). In study 2 (N = 64), an independent sample of pain-free participants performed the same tasks in a single session to replicate findings regarding the association between both tasks and the influence of visual information.

Results

The DMR task accuracy showed good-to-excellent test–retest reliability, while JR task reliability was poor: measurements did not remain sufficiently stable over testing days. The DMR and JR tasks were only weakly associated. Adding visual information (i.e., watching arm movement) had different performance effects on the tasks: it increased JR accuracy but decreased DMR accuracy, though only when the DMR task started with visual information available (i.e., an order effect).

Discussion

The DMR task’s highly standardized protocol (i.e., largely automated), precise measurement and involvement of the entire upper limb kinetic chain (i.e., shoulder, elbow and wrist joints) make it a promising tool. Moreover, the poor association between the JR and DMR tasks indicates that they likely capture unique aspects of proprioceptive function. While the former mainly captures position sense, the latter appears to capture sensorimotor integration processes underlying kinesthesia, largely independent of position sense. Finally, our results show that the integration of visual and proprioceptive information is not straightforward: additional visual information of arm movement does not necessarily make active movement reproduction more accurate, on the contrary, when movement is complex, vision appears to make it worse.

The human primary somatosensory cortex encodes imagined movement in the absence of sensory information

Classical systems neuroscience positions primary sensory areas as early feed-forward processing stations for refining incoming sensory information. This view may oversimplify their role given extensive bi-directional connectivity with multimodal cortical and subcortical regions. Here we show that single units in human primary somatosensory cortex encode imagined reaches in a cognitive motor task, but not other sensory–motor variables such as movement plans or imagined arm position. A population reference-frame analysis demonstrates coding relative to the cued starting hand location suggesting that imagined reaching movements are encoded relative to imagined limb position. These results imply a potential role for primary somatosensory cortex in cognitive imagery, engagement during motor production in the absence of sensation or expected sensation, and suggest that somatosensory cortex can provide control signals for future neural prosthetic systems.

Matiar Jafari, Tyson Aflalo et al. show that the human primary somatosensory cortex is activated when subjects imagine reaches in a cognitive motor task, but not when they plan movement or imagine a static limb position. These results highlight a role for this region in cognitive imagery and motor control in the absence of sensory information.