Agents of the mind

The Lower Limb Muscle Co-Activation Map during Human Locomotion: From Slow Walking to Running

The central nervous system (CNS) controls movements and regulates joint stiffness with muscle co-activation, but until now, few studies have examined muscle pairs during running. This study aims to investigate differences in lower limb muscle coactivation during gait at different speeds, from walking to running. Nineteen healthy runners walked and ran at speeds ranging from 0.8 km/h to 9.3 km/h. Twelve lower limb muscles' co-activation was calculated using the time-varying multi-muscle co-activation function (TMCf) with global, flexor-extension, and rostro-caudal approaches. Spatiotemporal and kinematic parameters were also measured. We found that TMCf, spatiotemporal, and kinematic parameters were significantly affected by gait speed for all approaches. Significant differences were observed in the main parameters of each co-activation approach and in the spatiotemporal and kinematic parameters at the transition between walking and running. In particular, significant differences were observed in the global co-activation (CIglob, main effect F(1,17) = 641.04, p < 0.001; at the transition p < 0.001), the stride length (main effect F(1,17) = 253.03, p < 0.001; at the transition p < 0.001), the stride frequency (main effect F(1,17) = 714.22, p < 0.001; at the transition p < 0.001) and the Center of Mass displacement in the vertical (CoMy, main effect F(1,17) = 426.2, p < 0.001; at the transition p < 0.001) and medial-lateral (CoMz, main effect F(1,17) = 120.29 p < 0.001; at the transition p < 0.001) directions. Regarding the correlation analysis, the CoMy was positively correlated with a higher CIglob (r = 0.88, p < 0.001) and negatively correlated with Full Width at Half Maximum (FWHMglob, r = -0.83, p < 0.001), whereas the CoMz was positively correlated with the global Center of Activity (CoAglob, r = 0.97, p < 0.001). Positive and negative strong correlations were found between global co-activation parameters and center of mass displacements, as well as some spatiotemporal parameters, regardless of gait speed. Our findings suggest that walking and running have different co-activation patterns and kinematic characteristics, with the whole-limb stiffness exerted more synchronously and stably during running. The co-activation indexes and kinematic parameters could be the result of global co-activation, which is a sensory-control integration process used by the CNS to deal with more demanding and potentially unstable tasks like running.

Higher visual gain contributions to bilateral motor synergies and force control

This study investigated the effects of altered visual gain levels on bilateral motor synergies determined by the uncontrolled manifold (UCM) hypothesis and force control. Twelve healthy participants performed bimanual index finger abduction force control tasks at 20% of their maximal voluntary contraction across four different visual gain conditions: 8, 80, 256, and 512 pixels/N. Quantifying force accuracy and variability within a trial provided a bimanual force control outcome. The UCM analysis measured bilateral motor synergies, a proportion of good variance to bad variance across multiple trials. Correlation analyses determined whether changes in the UCM variables were related to changes in force control variables from the lowest to highest visual gain conditions, respectively. Multiple analyses indicated that the three highest visual gain conditions in comparison to the lowest visual gain increased values of bilateral motor synergies and target force accuracy. The correlation findings showed that a reduction of bad variance from the lowest to three highest visual gain conditions was related to increased force accuracy. These findings reveal that visual gain greater than 8 pixels/N facilitates bimanual force control.

Intramuscle Synergies: Their Place in the Neural Control Hierarchy

We accept a definition of synergy introduced by Nikolai Bernstein and develop it for various actions, from those involving the whole body to those involving a single muscle. Furthermore, we use two major theoretical developments in the field of motor control—the idea of hierarchical control with spatial referent coordinates and the uncontrolled manifold hypothesis—to discuss recent studies of synergies within spaces of individual motor units (MUs) recorded within a single muscle. During the accurate finger force production tasks, MUs within hand extrinsic muscles form robust groups, with parallel scaling of the firing frequencies. The loading factors at individual MUs within each of the two main groups link them to the reciprocal and coactivation commands. Furthermore, groups are recruited in a task-specific way with gains that covary to stabilize muscle force. Such force-stabilizing synergies are seen in MUs recorded in the agonist and antagonist muscles but not in the spaces of MUs combined over the two muscles. These observations reflect inherent trade-offs between synergies at different levels of a control hierarchy. MU-based synergies do not show effects of hand dominance, whereas such effects are seen in multifinger synergies. Involuntary, reflex-based, force changes are stabilized by intramuscle synergies but not by multifinger synergies. These observations suggest that multifinger (multimuscle synergies) are based primarily on supraspinal circuitry, whereas intramuscle synergies reflect spinal circuitry. Studies of intra- and multimuscle synergies promise a powerful tool for exploring changes in spinal and supraspinal circuitry across patient populations.

One more time about motor (and non-motor) synergies

We revisit the concept of synergy based on the recently translated classical book by Nikolai Bernstein (On the construction of movements, Medgiz, Moscow 1947; Latash, Bernstein's Construction of Movements, Routledge, Abingdon 2020b) and progress in understanding the physics and neurophysiology of biological action. Two aspects of synergies are described: organizing elements into stable groups (modes) and ensuring dynamical stability of salient performance variables. The ability of the central nervous system to attenuate synergies in preparation for a quick action-anticipatory synergy adjustments-is emphasized. Recent studies have demonstrated synergies at the level of hypothetical control variables associated with spatial referent coordinates for effectors. Overall, the concept of synergies fits naturally the hierarchical scheme of control with referent coordinates with an important role played by back-coupling loops within the central nervous system and from peripheral sensory endings. Further, we review studies showing non-trivial changes in synergies with development, aging, fatigue, practice, and a variety of neurological disorders. Two aspects of impaired synergic control-impaired stability and impaired agility-are introduced. The recent generalization of the concept of synergies for non-motor domains, including perception, is discussed. We end the review with a list of unresolved and troubling issues.

Stability of Action and Kinesthetic Perception in Parkinson's Disease

We present a review of action and perception stability within the theoretical framework based on the idea of control with spatial referent coordinates for the effectors at a number of hierarchical levels. Stability of salient variables is ensured by synergies, neurophysiological structures that act in multi-dimensional spaces of elemental variables and limit variance to the uncontrolled manifold during action and iso-perceptual manifold during perception. Patients with Parkinson’s disease show impaired synergic control reflected in poor stability (low synergy indices) and poor agility (low indices of anticipatory synergy adjustments prior to planned quick actions). They also show impaired perception across modalities, including kinesthetic perception. We suggest that poor stability at the level of referent coordinates can be the dominant factor leading to poor stability of percepts.

A Revised Computational Neuroanatomy for Motor Control

We discuss a new framework for understanding the structure of motor control. Our approach integrates existing models of motor control with the reality of hierarchical cortical processing and the parallel segregated loops that characterize cortical-subcortical connections. We also incorporate the recent claim that cortex functions via predictive representation and optimal information utilization. Our framework assumes that each cortical area engaged in motor control generates a predictive model of a different aspect of motor behavior. In maintaining these predictive models, each area interacts with a different part of the cerebellum and BG. These subcortical areas are thus engaged in domain-appropriate system identification and optimization. This refocuses the question of division of function among different cortical areas. What are the different aspects of motor behavior that are predictively modeled? We suggest that one fundamental division is between modeling of task and body whereas another is the model of state and action. Thus, we propose that the posterior parietal cortex, somatosensory cortex, premotor cortex, and motor cortex represent task state, body state, task action, and body action, respectively. In the second part of this review, we demonstrate how this division of labor can better account for many recent findings of movement encoding, especially in the premotor and posterior parietal cortices.

On Primitives in Motor Control

The concept of primitives has been used in motor control both as a theoretical construct and as a means of describing the results of experimental studies involving multiple moving elements. This concept is close to Bernstein’s notion of engrams and level of synergies. Performance primitives have been explored in spaces of peripheral variables but interpreted in terms of neural control primitives. Performance primitives reflect a variety of mechanisms ranging from body mechanics to spinal mechanisms and to supraspinal circuitry. This review suggests that primitives originate at the task level as preferred time functions of spatial referent coordinates or at mappings from higher level referent coordinates to lower level, frequently abundant, referent coordinate sets. Different patterns of performance primitives can emerge depending, in particular, on the external force field.

The basal ganglia and the cerebellum: nodes in an integrated network

The basal ganglia and the cerebellum are considered to be distinct subcortical systems that perform unique functional operations. The outputs of the basal ganglia and the cerebellum influence many of the same cortical areas but do so by projecting to distinct thalamic nuclei. As a consequence, the two subcortical systems were thought to be independent and to communicate only at the level of the cerebral cortex. Here, we review recent data showing that the basal ganglia and the cerebellum are interconnected at the subcortical level. The subthalamic nucleus in the basal ganglia is the source of a dense disynaptic projection to the cerebellar cortex. Similarly, the dentate nucleus in the cerebellum is the source of a dense disynaptic projection to the striatum. These observations lead to a new functional perspective that the basal ganglia, the cerebellum and the cerebral cortex form an integrated network. This network is topographically organized so that the motor, cognitive and affective territories of each node in the network are interconnected. This perspective explains how synaptic modifications or abnormal activity at one node can have network-wide effects. A future challenge is to define how the unique learning mechanisms at each network node interact to improve performance.

Contributions of the Cerebellum for Predictive and Instructional Control of Movement

The cerebellum with its layered structure and stereotyped and conserved connectivity has long puzzled neurobiologists. While it is well established that the cerebellum functions in regulating balance, motor coordination and motor learning, how it achieves these end results has not been very clear. Recent technical advances have made it possible to tease apart the contributions of cerebellar cell types to movement in behaving animals. We review these studies focusing on the three major cerebellar cell types, namely: granule cells, Purkinje neurons and the cells of the deep cerebellar nuclei. Further, we also review our current understanding of cortico-cerebellar and basal ganglia-cerebellar interactions that play vital roles in motor planning and motor learning.

Structural and functional abnormality of the putamen in children with developmental dyslexia

There is currently debate with regards to the role of phonological deficit in Chinese reading difficulty, even though some researchers have suggested that the deficit of phonological processing is also a signature of developmental dyslexia in Chinese, as has been found in alphabetic languages. In this study, we examined the brain mechanisms of phonological deficit in Chinese children with developmental dyslexia (DD) during an auditory rhyming judgment task. First, we examined structural differences in Chinese dyslexia by comparing gray and white matter volume in Chinese children with DD, age-matched controls (AC), and reading-matched controls (RC). Next, we examined whether the regions with an abnormal volume in DD showed deficient functional connectivity with the rest of the brain during a phonological task (i.e. auditory rhyming judgment). We found that both AC and RC had greater gray matter volume (GMV) at the left putamen and right dorsal lateral frontal cortex than DD, suggesting possible neural signatures of developmental dyslexia. Functional connectivity analysis revealed that the left putamen was more connected with the right inferior occipital gyrus (IOG) in AC and RC than in DD, suggesting that automatic orthographic involvement during spoken language processing is more salient in controls, while the left putamen was more connected with the left transverse temporal gyrus (TTG) and left insula in DD than in AC and RC, suggesting the phonological articulation -auditory feedback loop is more involved in DD. These findings suggest that the reduced left putamen might contribute to phonological deficits experienced in DD, since it showed deficient connectivity with the rest of the brain during phonological processing.

Systemic effects of deep brain stimulation on synergic control in Parkinson's disease

OBJECTIVE

We explored effects of deep brain stimulation (DBS) in patients with Parkinson's disease (PD) on the synergic control of fingers in a multi-finger force production task and of muscles in a task involving vertical posture.

METHODS

The finger task involved the four fingers of a hand producing accurate total force followed by a targeted quick force pulse. The postural task involved releasing a load from extended arms. The analysis of synergies was performed within the framework of the uncontrolled manifold hypothesis.

RESULTS

DBS led to no significant changes in indices of stability during steady-state phases. In contrast, DBS improved indices of agility, quantified as anticipatory synergy adjustments that reduced stability of salient performance variables in preparation to their quick change. There were moderate-to-strong correlations between indices of both stability and agility measured in the multi-finger force production and multi-muscle whole-body action.

CONCLUSIONS

Our results point at systemic changes in synergic control in PD. They show that DBS is effective in improving only one components of synergic control related to agility in performance being relatively ineffective for the stability component.

SIGNIFICANCE

The results show systemic brain mechanisms of synergies and suggest differential effects of DBS on indices of stability and agility.

Muscle coactivation: definitions, mechanisms, and functions

The phenomenon of agonist-antagonist muscle coactivation is discussed with respect to its consequences for movement mechanics (such as increasing joint apparent stiffness, facilitating faster movements, and effects on action stability), implication for movement optimization, and involvement of different neurophysiological structures. Effects of coactivation on movement stability are ambiguous and depend on the effector representing a kinematic chain with a fixed origin or free origin. Furthermore, coactivation is discussed within the framework of the equilibrium-point hypothesis and the idea of hierarchical control with spatial referent coordinates. Relations of muscle coactivation to changes in one of the basic commands, the c-command, are discussed and illustrated. A hypothesis is suggested that agonist-antagonist coactivation reflects a deliberate neural control strategy to preserve effector-level control and avoid making it degenerate and facing the necessity to control at the level of signals to individual muscles. This strategy, in particular, allows stabilizing motor actions by covaried adjustments in spaces of control variables. This hypothesis is able to account for higher levels of coactivation in young healthy persons performing challenging tasks and across various populations with movement impairments.

The neural exploitation hypothesis and its implications for an embodied approach to language and cognition: Insights from the study of action verbs processing and motor disorders in Parkinson's disease

As it is widely known, Parkinson's disease is clinically characterized by motor disorders such as the loss of voluntary movement control, including resting tremor, postural instability, and bradykinesia (Bocanegra et al., 2015; Helmich, Hallett, Deuschl, Toni, & Bloem, 2012; Liu et al., 2006; Rosin, Topka, & Dichgans, 1997). In the last years, many empirical studies (e.g., Bocanegra et al., 2015; Spadacenta et al., 2012) have also shown that the processing of action verbs is selectively impaired in patients affected by this neurodegenerative disorder. In the light of these findings, it has been suggested that Parkinson disorder can be interpreted within an embodied cognition framework (e.g., Bocanegra et al., 2015). The central tenet of any embodied approach to language and cognition is that high order cognitive functions are grounded in the sensory-motor system. With regard to this point, Gallese (2008) proposed the neural exploitation hypothesis to account for, at the phylogenetic level, how key aspects of human language are underpinned by brain mechanisms originally evolved for sensory-motor integration. Glenberg and Gallese (2012) also applied the neural exploitation hypothesis to the ontogenetic level. On the basis of these premises, they developed a theory of language acquisition according to which, sensory-motor mechanisms provide a neurofunctional architecture for the acquisition of language, while retaining their original functions as well. The neural exploitation hypothesis is here applied to interpret the profile of patients affected by Parkinson's disease. It is suggested that action semantic impairments directly tap onto motor disorders. Finally, a discussion of what theory of language is needed to account for the interactions between language and movement disorders is presented.

Consensus Paper: Towards a Systems-Level View of Cerebellar Function: the Interplay Between Cerebellum, Basal Ganglia, and Cortex

Despite increasing evidence suggesting the cerebellum works in concert with the cortex and basal ganglia, the nature of the reciprocal interactions between these three brain regions remains unclear. This consensus paper gathers diverse recent views on a variety of important roles played by the cerebellum within the cerebello-basal ganglia-thalamo-cortical system across a range of motor and cognitive functions. The paper includes theoretical and empirical contributions, which cover the following topics: recent evidence supporting the dynamical interplay between cerebellum, basal ganglia, and cortical areas in humans and other animals; theoretical neuroscience perspectives and empirical evidence on the reciprocal influences between cerebellum, basal ganglia, and cortex in learning and control processes; and data suggesting possible roles of the cerebellum in basal ganglia movement disorders. Although starting from different backgrounds and dealing with different topics, all the contributors agree that viewing the cerebellum, basal ganglia, and cortex as an integrated system enables us to understand the function of these areas in radically different ways. In addition, there is unanimous consensus between the authors that future experimental and computational work is needed to understand the function of cerebellar-basal ganglia circuitry in both motor and non-motor functions. The paper reports the most advanced perspectives on the role of the cerebellum within the cerebello-basal ganglia-thalamo-cortical system and illustrates other elements of consensus as well as disagreements and open questions in the field.

Mechanisms of automaticity and anticipatory control in fluid intelligence

The constructs of fluid (Gf) and crystalized (Gc) intelligence represent an early attempt to describe the mechanisms of problem solving in the vertebrate brain. Modern neuroscience demonstrates that problem solving involves interplay between the mechanisms of automaticity and anticipatory control, enabling nature's elegant solution to the challenges animals face in their environment. Studies of neural functioning are making clear the primary role of cortical-subcortical interactions in the manifestation of intelligent behavior in humans and other vertebrates. A tridimensional model of intelligent problem solving is explored, wherein the basal ganglia system (BGS) and cerebrocerebellar system (CCS) interact within large scale brain networks. The BGS and CCS work together to enable automaticity to occur. The BGS enables the organism to learn what to do through a powerful instrumental learning system. The BGS also regulates when behavior is released through an inhibitory system which is incredibly sensitive to context. The CCS enables the organism to learn how to perform adaptive behaviors. Internal cerebellar models enable gradual improvements in the quality of behavioral output. The BGS and CCS interact within large scale brain networks, including the dorsal attention network (DAN), ventral attention network (VAN), default mode network (DMN) and frontoparietal network (FPN). The interactions of these systems enable vertebrate organisms to develop a vast array of complex adaptive behaviors. The benefits and importance of developing clinical tests to measure the integrity of these systems is considered.

Motor equivalence and structure of variance: multi-muscle postural synergies in Parkinson's disease

We explored posture-stabilizing multi-muscle synergies with two methods of analysis of multi-element, abundant systems: (1) Analysis of inter-cycle variance; and (2) Analysis of motor equivalence, both quantified within the framework of the uncontrolled manifold (UCM) hypothesis. Data collected in two earlier studies of patients with Parkinson's disease (PD) were re-analyzed. One study compared synergies in the space of muscle modes (muscle groups with parallel scaling of activation) during tasks performed by early-stage PD patients and controls. The other study explored the effects of dopaminergic medication on multi-muscle-mode synergies. Inter-cycle variance and absolute magnitude of the center of pressure displacement across consecutive cycles were quantified during voluntary whole-body sway within the UCM and orthogonal to the UCM space. The patients showed smaller indices of variance within the UCM and motor equivalence compared to controls. The indices were also smaller in the off-drug compared to on-drug condition. There were strong across-subject correlations between the inter-cycle variance within/orthogonal to the UCM and motor equivalent/non-motor equivalent displacements. This study has shown that, at least for cyclical tasks, analysis of variance and analysis of motor equivalence lead to metrics of stability that correlate with each other and show similar effects of disease and medication. These results show, for the first time, intimate links between indices of variance and motor equivalence. They suggest that analysis of motor equivalence, which requires only a handful of trials, could be used broadly in the field of motor disorders to analyze problems with action stability.

The role of the motor system in action naming in patients with neurodegenerative extrapyramidal syndromes

Previous studies of patients with brain damage have suggested a close relationship between aphasia and movement disorders. Neurodegenerative extrapyramidal syndromes associated with cognitive impairment provide an interesting model for studying the neural substrates of cognitive and motor symptoms. In this review, we focused on studies investigating language production abilities in patients with Parkinson's disease (PD), Corticobasal Syndrome (CBS) and Progressive Supranuclear Palsy (PSP). According to some reports, these patients exhibit a reduction in performance in both action and object naming or verb production compared to healthy individuals. Furthermore, a disproportional impairment of action naming compared to object naming was systematically observed in patients with these disorders. The study of these clinical conditions offers the unique opportunity to examine the close link between linguistic features and motor characteristics of action. This particular pattern of language impairment may contribute to the debate on embodiment theory and on the involvement of the basal ganglia in language and in integrating language and movement. From a translational perspective, we suggest that language ability assessments are useful in the clinical work-up, along with neuropsychological and motor evaluations. Specific protocols should be developed in the near future to better characterize language deficits and to permit an early cognitive diagnosis. Moreover, the link between language deficits and motor impairment opens a new issue for treatment approaches. Treatment of one of these two symptoms may ameliorate the other, and treating both may produce a greater improvement in patients' global clinical conditions.

Mapping Critical Language Sites in Children Performing Verb Generation: Whole-Brain Connectivity and Graph Theoretical Analysis in MEG

A classic left frontal-temporal brain network is known to support language processes. However, the level of participation of constituent regions, and the contribution of extra-canonical areas, is not fully understood; this is particularly true in children, and in individuals who have experienced early neurological insult. In the present work, we propose whole-brain connectivity and graph-theoretical analysis of magnetoencephalography (MEG) source estimates to provide robust maps of the pediatric expressive language network. We examined neuromagnetic data from a group of typically-developing young children (n = 15, ages 4–6 years) and adolescents (n = 14, 16–18 years) completing an auditory verb generation task in MEG. All source analyses were carried out using a linearly-constrained minimum-variance (LCMV) beamformer. Conventional differential analyses revealed significant (p < 0.05, corrected) low-beta (13–23 Hz) event related desynchrony (ERD) focused in the left inferior frontal region (Broca’s area) in both groups, consistent with previous studies. Connectivity analyses were carried out in broadband (3–30 Hz) on time-course estimates obtained at the voxel level. Patterns of connectivity were characterized by phase locking value (PLV), and network hubs identified through eigenvector centrality (EVC). Hub analysis revealed the importance of left perisylvian sites, i.e., Broca’s and Wernicke’s areas, across groups. The hemispheric distribution of frontal and temporal lobe EVC values was asymmetrical in most subjects; left dominant EVC was observed in 20% of young children, and 71% of adolescents. Interestingly, the adolescent group demonstrated increased critical sites in the right cerebellum, left inferior frontal gyrus (IFG) and left putamen. Here, we show that whole brain connectivity and network analysis can be used to map critical language sites in typical development; these methods may be useful for defining the margins of eloquent tissue in neurosurgical candidates.

The Pedunculopontine Tegmental Nucleus as a Motor and Cognitive Interface between the Cerebellum and Basal Ganglia

As an important component of ascending activating systems, brainstem cholinergic neurons in the pedunculopontine tegmental nucleus (PPTg) are involved in the regulation of motor control (locomotion, posture and gaze) and cognitive processes (attention, learning and memory). The PPTg is highly interconnected with several regions of the basal ganglia, and one of its key functions is to regulate and relay activity from the basal ganglia. Together, they have been implicated in the motor control system (such as voluntary movement initiation or inhibition), and modulate aspects of executive function (such as motivation). In addition to its intimate connection with the basal ganglia, projections from the PPTg to the cerebellum have been recently reported to synaptically activate the deep cerebellar nuclei. Classically, the cerebellum and basal ganglia were regarded as forming separated anatomical loops that play a distinct functional role in motor and cognitive behavioral control. Here, we suggest that the PPTg may also act as an interface device between the basal ganglia and cerebellum. As such, part of the therapeutic effect of PPTg deep brain stimulation (DBS) to relieve gait freezing and postural instability in advanced Parkinson’s disease (PD) patients might also involve modulation of the cerebellum. We review the anatomical position and role of the PPTg in the pathway of basal ganglia and cerebellum in relation to motor control, cognitive function and PD.

Towards physics of neural processes and behavior

Behavior of biological systems is based on basic physical laws, common across inanimate and living systems, and currently unknown physical laws that are specific for living systems. Living systems are able to unite basic laws of physics into chains and clusters leading to new stable and pervasive relations among variables (new physical laws) involving new parameters and to modify these parameters in a purposeful way. Examples of such laws are presented starting from the tonic stretch reflex. Further, the idea of control with referent coordinates is formulated and merged with the idea of hierarchical control and the principle of abundance. The notion of controlled stability of behaviors is linked to the idea of structured variability, which is a common feature across living systems and actions. The explanatory and predictive power of this approach is illustrated with respect to the control of both intentional and unintentional movements, the phenomena of equifinality and its violations, preparation to quick actions, development of motor skills, changes with aging and neurological disorders, and perception.

Toward a systems-oriented approach to the role of the extended amygdala in adaptive responding

Research into the structure and function of the basal forebrain macrostructure called the extended amygdala (EA) has recently seen considerable growth. This paper reviews that work, with the objectives of identifying underlying themes and developing a common goal towards which investigators of EA function might work. The paper begins with a brief review of the structure and the ontological and phylogenetic origins of the EA. It continues with a review of research into the role of the EA in both aversive and appetitive states, noting that these two seemingly disparate avenues of research converge on the concept of reinforcement - either negative or positive - of adaptive responding. These reviews lead to a proposal as to where the EA may fit in the organization of the basal forebrain, and an invitation to investigators to place their findings in a unifying conceptual framework of the EA as a collection of neural ensembles that mediate adaptive responding.

Effects of unilateral stroke on multi-finger synergies and their feed-forward adjustments

We explored the changes in multi-finger synergies in patients after a single cortical stroke with mild motor impairments. We hypothesized that both synergy indices and anticipatory synergy adjustments prior to the initiation of a self-paced quick action would be diminished in the patients compared to age-matched controls. The patients with history of cortical stroke, and age-matched controls (n=12 in each group) performed one-finger and multi-finger accurate force production tasks involving both steady-state and quick force pulse production. Finger interdependence (enslaving) and multi-finger synergies stabilizing total force were quantified. The stroke patients showed lower maximal finger forces, in particular in the contralesional hand, which also showed increased enslaving indices. Multi-finger synergies during steady-state force production were, however, unchanged after stroke. In contrast, a drop in the synergy index prior to the force pulse generation was significantly delayed in the stroke patients. Our results show that mild cortical stroke leads to no significant changes in multifinger synergies, but there is impairment in feed-forward adjustments of the synergies prior to a quick action, a drop in the maximal force production, and an increase in enslaving. We conclude that studies of synergies reveal two aspects of synergic control differentially affected by cortical stroke.

Fifty Years of Physics of Living Systems

The equilibrium-point hypothesis and its more recent version, the referent configuration hypothesis, represent the physical approach to the neural control of action. This hypothesis can be naturally combined with the idea of hierarchical control of movements and of synergic organization of the abundant systems involved in all actions. Any action starts with defining trajectories of a few referent coordinates for a handful of salient task-specific variables. Further, referent coordinates at hierarchically lower levels emerge down to thresholds of the tonic stretch reflex for the participating muscles. Stability of performance with respect to salient variables is reflected in the structure of inter-trial variance and phenomena of motor equivalence. Three lines of recent research within this framework are reviewed. First, synergic adjustments of the referent coordinate and apparent stiffness have been demonstrated during finger force production supporting the main idea of control with referent coordinates. Second, the notion of unintentional voluntary movements has been introduced reflecting unintentional drifts in referent coordinates. Two types of unintentional movements have been observed with different characteristic times. Third, this framework has been applied to studies of impaired movements in neurological patients. Overall, the physical approach searching for laws of nature underlying biological movement has been highly stimulating and productive.

Presynaptic Inhibition in the Striatum of the Basal Ganglia Improves Pattern Classification and Thus Promotes Superior Goal Selection

This review article takes a multidisciplinary approach to understand how presynaptic inhibition in the striatum of the basal ganglia (BG) contributes to pattern classification and the selection of goals that control behavior. It is a difficult problem both because it is multidimensional and because it is has complex system dynamics. We focus on the striatum because, as the main site for input to the BG, it gets to decide what goals are important to consider.

Neural control of movement stability: Lessons from studies of neurological patients

The concept of synergy provides a theoretical framework for movement stability resulting from the neural organization of multiple elements (digits, muscles, etc.) that all contribute to salient performance variables. Although stability of performance is obviously important for steady-state tasks leading to high synergy indices, a feed-forward drop in synergy indices is seen in preparation to a quick action (i.e., anticipatory synergy adjustments, ASAs). We review recent studies of multi-finger and multi-muscle synergies that show decreased indices of synergies and ASAs in patients with Parkinson's disease (PD) or multisystem atrophy. In PD, the impairments in synergies and ASAs are partially reversed by dopaminergic drugs, and changes in synergy indices are present even in PD patients at earliest diagnosis. Taken together, these results point at subcortical structures that are crucial for proper control of movement stability. It is timely to introduce the concept of impaired control of stability as an objective, quantifiable, and theory-based clinical descriptor of movement disorders that can increase our understanding of the neural control of movement with all of its implications for clinical practice.

Functional connectivity of the striatum in experts of stenography

INTRODUCTION

Stenography, or shorthand, is a unique set of skills that involves intensive training which is nearly life-long and orchestrating various brain functional modules, including auditory, linguistic, cognitive, mnemonic, and motor. Stenography provides cognitive neuroscientists with a unique opportunity to investigate the neural mechanisms underlying the neural plasticity that enables such a high degree of expertise. However, shorthand is quickly being replaced with voice recognition technology. We took this nearly final opportunity to scan the brains of the last alive shorthand experts of the Japanese language.

METHODS

Thirteen right-handed stenographers and fourteen right-handed controls participated in the functional magnetic resonance imaging (fMRI) study.

RESULTS

The fMRI data revealed plastic reorganization of the neural circuits around the putamen. The acquisition of expert skills was accompanied by structural and functional changes in the area. The posterior putamen is known as the execution center of acquired sensorimotor skills. Compared to nonexperts, the posterior putamen in stenographers had high covariation with the cerebellum and midbrain.The stenographers' brain developed different neural circuits from those of the nonexpert brain.

CONCLUSIONS

The current data illustrate the vigorous plasticity in the putamen and in its connectivity to other relevant areas in the expert brain. This is a case of vigorous neural plastic reorganization in response to massive overtraining, which is rare especially considering that it occurred in adulthood.

Stability of multifinger action in different state spaces

We investigated stability of action by a multifinger system with three methods: analysis of intertrial variance, application of transient perturbations, and analysis of the system's motion in different state spaces. The "inverse piano" device was used to apply transient (lifting-and-lowering) perturbations to individual fingers during single- and two-finger accurate force production tasks. In each trial, the perturbation was applied either to a finger explicitly involved in the task or one that was not. We hypothesized that, in one-finger tasks, task-specific stability would be observed in the redundant space of finger forces but not in the nonredundant space of finger modes (commands to explicitly involved fingers). In two-finger tasks, we expected that perturbations applied to a nontask finger would not contribute to task-specific stability in mode space. In contrast to our expectations, analyses in both force and mode spaces showed lower stability in directions that did not change total force output compared with directions that did cause changes in total force. In addition, the transient perturbations led to a significant increase in the enslaving index. We consider these results within a theoretical scheme of control with referent body configurations organized hierarchically, using multiple few-to-many mappings organized in a synergic way. The observed volatility of enslaving, greater equifinality of total force compared with elemental variables, and large magnitude of motor equivalent motion in both force and mode spaces provide support for the concept of task-specific stability of performance and the existence of multiple neural loops, which ensure this stability.

Cognitive control over learning: creating, clustering, and generalizing task-set structure

Learning and executive functions such as task-switching share common neural substrates, notably prefrontal cortex and basal ganglia. Understanding how they interact requires studying how cognitive control facilitates learning but also how learning provides the (potentially hidden) structure, such as abstract rules or task-sets, needed for cognitive control. We investigate this question from 3 complementary angles. First, we develop a new context-task-set (C-TS) model, inspired by nonparametric Bayesian methods, specifying how the learner might infer hidden structure (hierarchical rules) and decide to reuse or create new structure in novel situations. Second, we develop a neurobiologically explicit network model to assess mechanisms of such structured learning in hierarchical frontal cortex and basal ganglia circuits. We systematically explore the link between these modeling levels across task demands. We find that the network provides an approximate implementation of high-level C-TS computations, with specific neural mechanisms modulating distinct C-TS parameters. Third, this synergism yields predictions about the nature of human optimal and suboptimal choices and response times during learning and task-switching. In particular, the models suggest that participants spontaneously build task-set structure into a learning problem when not cued to do so, which predicts positive and negative transfer in subsequent generalization tests. We provide experimental evidence for these predictions and show that C-TS provides a good quantitative fit to human sequences of choices. These findings implicate a strong tendency to interactively engage cognitive control and learning, resulting in structured abstract representations that afford generalization opportunities and, thus, potentially long-term rather than short-term optimality.

Dopaminergic modulation of motor coordinaton in Parkinson's disease

BACKGROUND

We applied the idea of synergies and the framework of the uncontrolled manifold hypothesis to explore the effects of dopamine replacement therapy on finger interaction and coordination in patients with early-stage Parkinson's disease (PD).

METHODS

Eight patients performed single-finger and multi-finger force production tasks with both the dominant and non-dominant hand before (off-drug) and after (on-drug) taking their dopaminergic medications. Synergy indices were defined as co-varied adjustments of commands to fingers that stabilized the total force produced by the hand.

RESULTS

PD patients showed significantly lower maximal finger forces off-drug compared to the on-drug condition, while indices of finger individuation (enslaving) were unchanged. The synergy indices were weaker during steady-state force production off-drug compared to on-drug. Anticipatory adjustments of synergies prior to the quick force pulse initiation were delayed and reduced off-drug as compared to the on-drug condition. These drug effects were observed in both the symptomatic and asymptomatic hands of the patients whose symptoms were limited to one side of the body.

CONCLUSIONS

The study demonstrates dopaminergic modulation of motor coordination in PD and supports that the analysis of different components of multi-finger synergies offers a set of indices sensitive to the effects of dopamine replacement therapy in early-stage PD. The results suggest an important role of the basal ganglia in synergy formation and in feed-forward synergy adjustments. Future studies using these methods may yield more objective, quantitative biomarker(s) of motor coordination impairments in PD, and better understanding of subcortical involvement in the neural control of natural actions.

Effects of Parkinson's disease on optimization and structure of variance in multi-finger tasks

We explored the role of the basal ganglia in two components of multi-finger synergies by testing a group of patients with early-stage Parkinson's disease and a group of healthy controls. Synergies were defined as co-varied adjustments of commands to individual fingers that reduced variance of the total force and moment of force. The framework of the uncontrolled manifold hypothesis was used to quantify such co-variation patterns, while average performance across repetitive trials (sharing patterns) was analyzed using the analytical inverse optimization (ANIO) approach. The subjects performed four-finger pressing tasks that involved the accurate production of combinations of the total force and total moment of force and also repetitive trials at two selected combinations of the total force and moment. The ANIO approach revealed significantly larger deviations of the experimental data planes from an optimal plane for the patients compared to the control subjects. The synergy indices computed for total force stabilization were significantly higher in the control subjects compared to the patients; this was not true for synergy indices computed for moment of force stabilization. The differences in the synergy indices were due to the larger amount of variance that affected total force in the patients, while the amount of variance that did not affect total force was comparable between the groups. We conclude that the basal ganglia play an important role in both components of synergies reflecting optimization of the sharing patterns and stability of performance with respect to functionally important variables.

Effects of muscle vibration on multi-finger interaction and coordination

The purpose of this study was to investigate the effects of changes in the proprioceptive signals induced by muscle vibration on multi-finger interaction and coordination. We hypothesized that unintended force production by non-instructed fingers (enslaving) would increase with muscle vibration while synergy indices during steady-state force production would drop. The framework of the uncontrolled manifold hypothesis was used to quantify indices of multi-finger synergies stabilizing total force during steady-state force production and anticipatory changes in these indices (anticipatory synergy adjustments, ASAs) in preparation to a quick force pulse production with and without hand-muscle vibration at 80 Hz. The dominant hands of twelve healthy right-handed subjects were tested under three conditions: no vibration, vibration of the palmar surface of the hand, and vibration of the forearm applied over the flexor muscles. There were no significant effects of vibration on maximal voluntary force. The magnitude of enslaving was larger during vibration of the hand compared to the other two conditions. During steady-state force production, strong synergies stabilizing total force were seen in all three conditions; however, indices of force-stabilizing synergies were lower during vibration of the hand. Prior to the force pulse initiation, the synergy index started to drop earlier and over a larger magnitude without vibration compared to either vibration condition. Effects of vibration on enslaving and synergy index may be due to diffuse reflex effects of the induced afferent activity on alpha-motoneuronal pools innervating the extrinsic flexor compartments. We conclude that multi-finger synergies are not based on signals from muscle receptors. The smaller synergy indices and ASAs may reflect supraspinal effects of the vibration-induced afferent activity, in particular its interactions with trans-thalamic loops.

Cerebellar networks with the cerebral cortex and basal ganglia

The dominant view of cerebellar function has been that it is exclusively concerned with motor control and coordination. Recent findings from neuroanatomical, behavioral, and imaging studies have profoundly changed this view. Neuroanatomical studies using virus transneuronal tracers have demonstrated that cerebellar output reaches vast areas of the neocortex, including regions of prefrontal and posterior parietal cortex. Furthermore, it has recently become clear that the cerebellum is reciprocally connected with the basal ganglia, which suggests that the two subcortical structures are part of a densely interconnected network. Taken together, these findings elucidate the neuroanatomical substrate for cerebellar involvement in non-motor functions mediated by the prefrontal and posterior parietal cortex, as well as in processes traditionally associated with the basal ganglia.

Striatal morphology as a biomarker in neurodegenerative disease

The striatum, comprising the caudate nucleus, putamen and nucleus accumbens, occupies a strategic location within cortico-striato-pallido-thalamic-cortical (corticostriatal) re-entrant neural circuits. Striatal neurodevelopment is precisely determined by phylogenetically conserved homeobox genes. Consisting primarily of medium spiny neurons, the striatum is strictly topographically organized based on cortical afferents and efferents. Particular corticostriatal neural circuits are considered to subserve certain domains of cognition, emotion and behaviour. Thus, the striatum may serve as a map of structural change in the cortical afferent pathways owing to deafferentation or neuroplasticity, and conversely, structural change in the striatum per se may structurally disrupt corticostriatal pathways. The morphology of the striatum may be quantified in vivo using advanced magnetic resonance imaging, as may cognitive functioning pertaining to corticostriatal circuits. It is proposed that striatal morphology may be a biomarker in neurodegenerative disease and potentially the basis of an endophenotype.

Effects of olivo-ponto-cerebellar atrophy (OPCA) on finger interaction and coordination

OBJECTIVES

We investigated changes in finger interaction and coordination in patients with olivo-ponto-cerebellar atrophy (OPCA) using the recently developed approach to motor synergies based on the principle of motor abundance.

METHODS

OPCA patients and control subjects performed sets of maximal and submaximal force production tasks by the fingers of each of the hands. Indices of multi-finger synergies were quantified within the framework of the uncontrolled manifold hypothesis.

RESULTS

The patients showed lower maximal forces, higher indices of finger interdependence (enslaving), and lower indices of multi-finger synergies stabilizing total force in four-finger tasks. In addition, the patients showed an impaired ability to adjust synergies in preparation to a quick action (small and delayed anticipatory synergy adjustments). The synergy indices showed significant correlations with the clinical scores (both UPDRS total motor scores and ataxia related sub-scores). The observed changes in the indices of finger interaction and coordination were qualitatively similar to those reported earlier for patients with Parkinson's disease; however, the magnitude of the changes was much higher in the OPCA group.

CONCLUSIONS

These findings fit the hypotheses on the role of the cerebellum in assembling motor synergies and in the feed-forward control of action. They suggest that the synergy index measured in artificial, constrained laboratory tasks may be predictive of more general changes in motor behavior.

SIGNIFICANCE

The results suggest that studies of multi-digit synergies may be particularly sensitive to subcortical disorders and may provide a much-needed tool for quantitative assessment of impaired coordination in such patients.

Changes in multifinger interaction and coordination in Parkinson's disease

In this study, we tested several hypotheses related to changes in finger interaction and multifinger synergies during multifinger force production tasks in Parkinson's disease. Ten patients with Parkinson's disease, mostly early stage, and 11 healthy control subjects participated in the study. Synergies were defined as covaried adjustment of commands to fingers that stabilized the total force produced by the hand. Both Parkinson's disease patients and control subjects performed accurate isometric force production tasks with the fingers of both the dominant and nondominant hands. The Parkinson's disease patients showed significantly lower maximal finger forces and higher unintended force production (enslaving). These observations suggest that changes in supraspinal control have a major effect on finger individuation. The synergy indexes in the patients were weaker in both steady-state and cyclic force production tasks compared with the controls. These indexes also were stronger in the left (nondominant) hand in support of the dynamic-dominance hypothesis. Half of the patients could not perform the cyclic task at the highest frequency (2 Hz). Anticipatory adjustments of synergies prior to a quick force pulse production were delayed and reduced in the patients compared with the controls. Similar differences were observed between the asymptomatic hands of the patients with symptoms limited to one side of the body and matched hands of control subjects. Our study demonstrates that the elusive changes in motor coordination in Parkinson's disease can be quantified objectively, even in patients at a relatively early stage of the disease. The results suggest an important role of the basal ganglia in synergy formation and demonstrate a previously unknown component of impaired feedforward control in Parkinson's disease reflected in the reduced and delayed anticipatory synergy adjustments.

Motor-language coupling: direct evidence from early Parkinson's disease and intracranial cortical recordings

Language and action systems are functionally coupled in the brain as demonstrated by converging evidence using Functional magnetic resonance imaging (fMRI), electroencephalography (EEG), transcranial magnetic stimulation (TMS), and lesion studies. In particular, this coupling has been demonstrated using the action-sentence compatibility effect (ACE) in which motor activity and language interact. The ACE task requires participants to listen to sentences that described actions typically performed with an open hand (e.g., clapping), a closed hand (e.g., hammering), or without any hand action (neutral); and to press a large button with either an open hand position or closed hand position immediately upon comprehending each sentence. The ACE is defined as a longer reaction time (RT) in the action-sentence incompatible conditions than in the compatible conditions. Here we investigated direct motor-language coupling in two novel and uniquely informative ways. First, we measured the behavioural ACE in patients with motor impairment (early Parkinson's disease - EPD), and second, in epileptic patients with direct electrocorticography (ECoG) recordings. In experiment 1, EPD participants with preserved general cognitive repertoire, showed a much diminished ACE relative to non-EPD volunteers. Moreover, a correlation between ACE performance and action-verb processing (kissing and dancing test - KDT) was observed. Direct cortical recordings (ECoG) in motor and language areas (experiment 2) demonstrated simultaneous bidirectional effects: motor preparation affected language processing (N400 at left inferior frontal gyrus and middle/superior temporal gyrus), and language processing affected activity in movement-related areas (motor potential at premotor and M1). Our findings show that the ACE paradigm requires ongoing integration of preserved motor and language coupling (abolished in EPD) and engages motor-temporal cortices in a bidirectional way. In addition, both experiments suggest the presence of a motor-language network which is not restricted to somatotopically defined brain areas. These results open new pathways in the fields of motor diseases, theoretical approaches to language understanding, and models of action-perception coupling.

Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis

Growing evidence suggests that the prefrontal cortex (PFC) is organized hierarchically, with more anterior regions having increasingly abstract representations. How does this organization support hierarchical cognitive control and the rapid discovery of abstract action rules? We present computational models at different levels of description. A neural circuit model simulates interacting corticostriatal circuits organized hierarchically. In each circuit, the basal ganglia gate frontal actions, with some striatal units gating the inputs to PFC and others gating the outputs to influence response selection. Learning at all of these levels is accomplished via dopaminergic reward prediction error signals in each corticostriatal circuit. This functionality allows the system to exhibit conditional if-then hypothesis testing and to learn rapidly in environments with hierarchical structure. We also develop a hybrid Bayesian-reinforcement learning mixture of experts (MoE) model, which can estimate the most likely hypothesis state of individual participants based on their observed sequence of choices and rewards. This model yields accurate probabilistic estimates about which hypotheses are attended by manipulating attentional states in the generative neural model and recovering them with the MoE model. This 2-pronged modeling approach leads to multiple quantitative predictions that are tested with functional magnetic resonance imaging in the companion paper.

A modular neural model of motor synergies

Animals such as reptiles, amphibians and mammals (including humans) are mechanically extremely complex. It has been estimated that the human body has between 500 and 1400 degrees of freedom! And yet, these animals can generate an infinite variety of very precise, complicated and goal-directed movements in continuously changing and uncertain environments. Understanding how this is achieved is of great interest to both biologists and engineers. There are essentially two questions that must be addressed: (1) What type of control strategy is used to handle the large number of degrees of freedom involved? and (2) How is this strategy instantiated in the substrate of neural and musculoskeletal elements comprising the animal bodies? The first question has been studied intensively for several decades, providing strong indications that, rather than using standard feedback control based on continuous tracking of desired trajectories, animals' movements emerge from the controlled combination of pre-configured movement primitives or synergies. These synergies represent coordinated activity patterns over groups of muscles, and can be triggered as a whole with controlled amplitude and temporal offset. Complex movements can thus be constructed from the appropriate combination of a relatively small number of synergies, greatly simplifying the control problem. Although experimental studies on animal movements have confirmed the existence of motor synergies, and their utility has been demonstrated in the control of fairly complex robots, their neural basis remains poorly understood. In this paper, we introduce a simple but plausible and general neural model for motor synergies based on the principle that these functional modules reflect the structural modularity of the underlying physical system. Using this model, we show that a small set of synergies selected through a redundancy-reduction principle can generate a rich motor repertoire in a model two-jointed arm system. We investigate the synergies generated by this model systematically with respect to various parameters, and compare them to those observed in experiments.

Predicting comorbidities with neuroimaging in children with cerebral palsy

A population-based registry was used to ascertain whether neuroimaging findings of children with cerebral palsy could predict the occurrence of certain comorbidities. Neuroimaging findings and comorbidities data were extracted from the Quebec Cerebral Palsy Registry for children born in a 4-year birth interval (1999-2002) covering half of the province's population. Neuroimaging studies were classified into 10 mutually exclusive categories (periventricular white matter injury/leukomalacia, cerebral malformation, cerebral vascular accident, deep gray matter injury, superficial gray matter injury, diffuse gray matter injury, intracranial hemorrhage, infection, nonspecific findings, and normal). Comorbidities studied included cortical blindness, severe auditory impairment, inability to communicate verbally, assisted feeding, and the presence of afebrile seizures in the prior 12 months. Neuroimaging results were available for a total of 213 children. Only deep gray matter injury (defined as signal abnormality or volume loss in subcortical gray matter, n = 9) was significantly (P < 0.05) linked with the occurrence of both the inability to communicate verbally (n = 5, 55.6% vs n = 46, 22.5%, P = 0.04) and with a higher mean number of comorbidities (1.67 vs 0.70, P < 0.01), and therefore with increased burden of comorbidities. These findings may improve our ability to prognosticate the outcome of children with cerebral palsy, enabling targeted early direct interventions.

Modeling signal transduction in classical conditioning with network motifs

Biological networks are constructed of repeated simplified patterns, or modules, called network motifs. Network motifs can be found in a variety of organisms including bacteria, plants, and animals, as well as intracellular transcription networks for gene expression and signal transduction processes in neuronal circuits. Standard models of signal transduction events for synaptic plasticity and learning often fail to capture the complexity and cooperativity of the molecular interactions underlying these processes. Here, we apply network motifs to a model for signal transduction during an in vitro form of eyeblink classical conditioning that reveals an underlying organization of these molecular pathways. Experimental evidence suggests there are two stages of synaptic AMPA receptor (AMPAR) trafficking during conditioning. Synaptic incorporation of GluR1-containing AMPARs occurs early to activate silent synapses conveying the auditory conditioned stimulus and this initial step is followed by delivery of GluR4 subunits that supports acquisition of learned conditioned responses (CRs). Overall, the network design of the two stages of synaptic AMPAR delivery during conditioning describes a coherent feed-forward loop (C1-FFL) with AND logic. The combined inputs of GluR1 synaptic delivery AND the sustained activation of 3-phosphoinositide-dependent protein-kinase-1 (PDK-1) results in synaptic incorporation of GluR4-containing AMPARs and the gradual acquisition of CRs. The network architecture described here for conditioning is postulated to act generally as a sign-sensitive delay element that is consistent with the non-linearity of the conditioning process. Interestingly, this FFL structure also performs coincidence detection. A motif-based approach to modeling signal transduction can be used as a new tool for understanding molecular mechanisms underlying synaptic plasticity and learning and for comparing findings across forms of learning and model systems.

An imperfect dopaminergic error signal can drive temporal-difference learning

An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD) learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards.

What are the physiological changes that take place in the brain when we solve a problem or learn a new skill? It is commonly assumed that behavior adaptations are realized on the microscopic level by changes in synaptic efficacies. However, this is hard to verify experimentally due to the difficulties of identifying the relevant synapses and monitoring them over long periods during a behavioral task. To address this question computationally, we develop a spiking neuronal network model of actor-critic temporal-difference learning, a variant of reinforcement learning for which neural correlates have already been partially established. The network learns a complex task by means of an internally generated reward signal constrained by recent findings on the dopaminergic system. Our model combines top-down and bottom-up modelling approaches to bridge the gap between synaptic plasticity and system-level learning. It paves the way for further investigations of the dopaminergic system in reward learning in the healthy brain and in pathological conditions such as Parkinson's disease, and can be used as a module in functional models based on brain-scale circuitry.