On the Role of Cortex-Basal Ganglia Interactions for Category Learning: A Neurocomputational Approach

The gradual coevolution of syntactic combinatorics and categorization under the effects of human self-domestication: a proposal

The gradual emergence of syntax has been claimed to be engaged in a feedback loop with Human Self-Domestication (HSD), both processes resulting from, and contributing to, enhanced connectivity in selected cortico-striatal networks, which is the mechanism for attenuating reactive aggression, the hallmark of HSD, but also the mechanism of cross-modality, relevant for syntax. Here, we aim to bridge the gap between these brain changes and further changes facilitated by the gradual complexification of grammars. We propose that increased cross-modality would have enabled and supported, more specifically, a feedback loop between categorization abilities relevant for vocabulary building and the gradual emergence of syntactic structure, including Merge. In brief, an enhanced categorization ability not only brings about more distinct categories, but also a critical number of tokens in each category necessary for Merge to take off in a systematic and productive fashion; in turn, the benefits of expressive capabilities brought about by productive Merge encourage more items to be categorized, and more categories to be formed, thus further potentiating categorization abilities, and with it, syntax again. We support our hypothesis with evidence from the domains of language development and animal communication, but also from biology, neuroscience, paleoanthropology, and clinical linguistics.

Hierarchical categorization learning is associated with representational changes in the dorsal striatum and posterior frontal and parietal cortex

Learning and recognition can be improved by sorting novel items into categories and subcategories. Such hierarchical categorization is easy when it can be performed according to learned rules (e.g., "if car, then automatic or stick shift" or "if boat, then motor or sail"). Here, we present results showing that human participants acquire categorization rules for new visual hierarchies rapidly, and that, as they do, corresponding hierarchical representations of the categorized stimuli emerge in patterns of neural activation in the dorsal striatum and in posterior frontal and parietal cortex. Participants learned to categorize novel visual objects into a hierarchy with superordinate and subordinate levels based on the objects' shape features, without having been told the categorization rules for doing so. On each trial, participants were asked to report the category and subcategory of the object, after which they received feedback about the correctness of their categorization responses. Participants trained over the course of a one-hour-long session while their brain activation was measured using functional magnetic resonance imaging. Over the course of training, significant hierarchy learning took place as participants discovered the nested categorization rules, as evidenced by the occurrence of a learning trial, after which performance suddenly increased. This learning was associated with increased representational strength of the newly acquired hierarchical rules in a corticostriatal network including the posterior frontal and parietal cortex and the dorsal striatum. We also found evidence suggesting that reinforcement learning in the dorsal striatum contributed to hierarchical rule learning.

The contribution of the basal ganglia and cerebellum to motor learning: A neuro-computational approach

Motor learning involves a widespread brain network including the basal ganglia, cerebellum, motor cortex, and brainstem. Despite its importance, little is known about how this network learns motor tasks and which role different parts of this network take. We designed a systems-level computational model of motor learning, including a cortex-basal ganglia motor loop and the cerebellum that both determine the response of central pattern generators in the brainstem. First, we demonstrate its ability to learn arm movements toward different motor goals. Second, we test the model in a motor adaptation task with cognitive control, where the model replicates human data. We conclude that the cortex-basal ganglia loop learns via a novelty-based motor prediction error to determine concrete actions given a desired outcome, and that the cerebellum minimizes the remaining aiming error.

Evolution of neural activity in circuits bridging sensory and abstract knowledge

The ability to associate sensory stimuli with abstract classes is critical for survival. How are these associations implemented in brain circuits? And what governs how neural activity evolves during abstract knowledge acquisition? To investigate these questions, we consider a circuit model that learns to map sensory input to abstract classes via gradient-descent synaptic plasticity. We focus on typical neuroscience tasks (simple, and context-dependent, categorization), and study how both synaptic connectivity and neural activity evolve during learning. To make contact with the current generation of experiments, we analyze activity via standard measures such as selectivity, correlations, and tuning symmetry. We find that the model is able to recapitulate experimental observations, including seemingly disparate ones. We determine how, in the model, the behaviour of these measures depends on details of the circuit and the task. These dependencies make experimentally testable predictions about the circuitry supporting abstract knowledge acquisition in the brain.

Dorsomedial striatum, but not dorsolateral striatum, is necessary for rat category learning

Categorization is an adaptive cognitive function that allows us to generalize knowledge to novel situations. Converging evidence from neuropsychological, neuroimaging, and neurophysiological studies suggest that categorization is mediated by the basal ganglia; however, there is debate regarding the necessity of each subregion of the basal ganglia and their respective functions. The current experiment examined the roles of the dorsomedial striatum (DMS; homologous to the head of the caudate nucleus) and dorsolateral striatum (DLS; homologous to the body and tail of the caudate nucleus) in category learning by combining selective lesions with computational modeling. Using a touchscreen apparatus, rats were trained to categorize distributions of visual stimuli that varied along two continuous dimensions (i.e., spatial frequency and orientation). The tasks either required attention to one stimulus dimension (spatial frequency or orientation; 1D tasks) or both stimulus dimensions (spatial frequency and orientation; 2D tasks). Rats with NMDA lesions of the DMS were impaired on both the 1D tasks and 2D tasks, whereas rats with DLS lesions showed no impairments. The lesions did not affect performance on a discrimination task that had the same trial structure as the categorization tasks, suggesting that the category impairments effected processes relevant to categorization. Model simulations were conducted using a neural network to assess the effect of the DMS lesions on category learning. Together, the results suggest that the DMS is critical to map category representations to appropriate behavioral responses, whereas the DLS is not necessary for categorization.

A brain-inspired intention prediction model and its applications to humanoid robot

With the development of artificial intelligence and robotic technology in recent years, robots are gradually integrated into human daily life. Most of the human-robot interaction technologies currently applied to home service robots are programmed by the manufacturer first, and then instruct the user to trigger the implementation through voice commands or gesture commands. Although these methods are simple and effective, they lack some flexibility, especially when the programming program is contrary to user habits, which will lead to a significant decline in user experience satisfaction. To make that robots can better serve human beings, adaptable, simple, and flexible human-robot interaction technology is essential. Based on the neural mechanism of reinforcement learning, we propose a brain-inspired intention prediction model to enable the robot to perform actions according to the user's intention. With the spike-timing-dependent plasticity (STDP) mechanisms and the simple feedback of right or wrong, the humanoid robot NAO could successfully predict the user's intentions in Human Intention Prediction Experiment and Trajectory Tracking Experiment. Compared with the traditional Q-learning method, the training times required by the proposed model are reduced by (N² − N)/4, where N is the number of intentions.

Multilevel development of cognitive abilities in an artificial neural network

Several neuronal mechanisms have been proposed to account for the formation of cognitive abilities through postnatal interactions with the physical and sociocultural environment. Here, we introduce a three-level computational model of information processing and acquisition of cognitive abilities. We propose minimal architectural requirements to build these levels, and how the parameters affect their performance and relationships. The first sensorimotor level handles local nonconscious processing, here during a visual classification task. The second level or cognitive level globally integrates the information from multiple local processors via long-ranged connections and synthesizes it in a global, but still nonconscious, manner. The third and cognitively highest level handles the information globally and consciously. It is based on the global neuronal workspace (GNW) theory and is referred to as the conscious level. We use the trace and delay conditioning tasks to, respectively, challenge the second and third levels. Results first highlight the necessity of epigenesis through the selection and stabilization of synapses at both local and global scales to allow the network to solve the first two tasks. At the global scale, dopamine appears necessary to properly provide credit assignment despite the temporal delay between perception and reward. At the third level, the presence of interneurons becomes necessary to maintain a self-sustained representation within the GNW in the absence of sensory input. Finally, while balanced spontaneous intrinsic activity facilitates epigenesis at both local and global scales, the balanced excitatory/inhibitory ratio increases performance. We discuss the plausibility of the model in both neurodevelopmental and artificial intelligence terms.

Cumulative Genetic Score of DRD2 Polymorphisms Is Associated with Impulsivity and Masked Semantic Priming

Individual differences in the magnitude of semantic priming effects are associated with executive functions (EF). Striatal dopamine has been shown to be associated with EF as well as impulsivity and could therefore be associated with differences in the magnitude of semantic priming. We investigated n = 155 individuals in an unmasked as well as in a masked semantic priming paradigm. We additionally assessed self-reported impulsivity and a cumulative genetic score (CGS) comprising six polymorphisms that have been found to be functionally relevant for the expression of the DRD2 gene. We found a significantly negative association between the DRD2 CGS and reaction time priming in the masked semantic priming paradigm. In addition, the DRD2 CGS was positively associated with self-reported impulsivity. Our findings complement previous research by showing a role of the DRD2 gene for masked semantic priming. Therefore, the investigation of genes within the dopamine system might improve our understanding of the genetic basis of impulsivity and semantic processing. Thus, the DRD2 CGS is of interest for clinical as well as experimental psychological research.

Enhanced habit formation in Tourette patients explained by shortcut modulation in a hierarchical cortico-basal ganglia model

Devaluation protocols reveal that Tourette patients show an increased propensity to habitual behaviors as they continue to respond to devalued outcomes in a cognitive stimulus-response-outcome association task. We use a neuro-computational model of hierarchically organized cortico-basal ganglia-thalamo-cortical loops to shed more light on habit formation and its alteration in Tourette patients. In our model, habitual behavior emerges from cortico-thalamic shortcut connections, where enhanced habit formation can be linked to faster plasticity in the shortcut or to a stronger feedback from the shortcut to the basal ganglia. We explore two major hypotheses of Tourette pathophysiology-local striatal disinhibition and increased dopaminergic modulation of striatal medium spiny neurons-as causes for altered shortcut activation. Both model changes altered shortcut functioning and resulted in higher rates of responses towards devalued outcomes, similar to what is observed in Tourette patients. We recommend future experimental neuroscientific studies to locate shortcuts between cortico-basal ganglia-thalamo-cortical loops in the human brain and study their potential role in health and disease.

Performance of biologically grounded models of the early visual system on standard object recognition tasks

Computational neuroscience models of vision and neural network models for object recognition are often framed by different research agendas. Computational neuroscience mainly aims at replicating experimental data, while (artificial) neural networks target high performance on classification tasks. However, we propose that models of vision should be validated on object recognition tasks. At some point, mechanisms of realistic neuro-computational models of the visual cortex have to convince in object recognition as well. In order to foster this idea, we report the recognition accuracy for two different neuro-computational models of the visual cortex on several object recognition datasets. The models were trained using unsupervised Hebbian learning rules on natural scene inputs for the emergence of receptive fields comparable to their biological counterpart. We assume that the emerged receptive fields result in a general codebook of features, which should be applicable to a variety of visual scenes. We report the performances on datasets with different levels of difficulty, ranging from the simple MNIST to the more complex CIFAR-10 or ETH-80. We found that both networks show good results on simple digit recognition, comparable with previously published biologically plausible models. We also observed that our deeper layer neurons provide for naturalistic datasets a better recognition codebook. As for most datasets, recognition results of biologically grounded models are not available yet, our results provide a broad basis of performance values to compare methodologically similar models.

Optimal attention tuning in a neuro-computational model of the visual cortex-basal ganglia-prefrontal cortex loop

Visual attention is widely considered a vital factor in the perception and analysis of a visual scene. Several studies explored the effects and mechanisms of top-down attention, but the mechanisms that determine the attentional signal are less explored. By developing a neuro-computational model of visual attention including the visual cortex-basal ganglia loop, we demonstrate how attentional alignment can evolve based on dopaminergic reward during a visual search task. Unlike most previous modeling studies of feature-based attention, we do not implement a manually predefined attention template. Dopamine-modulated covariance learning enable the basal ganglia to learn rewarded associations between the visual input and the attentional gain represented in the PFC of the model. Hence, the model shows human-like performance on a visual search task by optimally tuning the attention signal. In particular, similar as in humans, this reward-based tuning in the model leads to an attentional template that is not centered on the target feature, but a relevant feature deviating away from the target due to the presence of highly similar distractors. Further analyses of the model shows, attention is mainly guided by the signal-to-noise ratio between target and distractors.

Morphological, Structural, and Functional Networks Highlight the Role of the Cortical-Subcortical Circuit in Individuals With Subjective Cognitive Decline

Subjective cognitive decline (SCD) is considered the earliest stage of the clinical manifestations of the continuous progression of Alzheimer’s Disease (AD). Previous studies have suggested that multimodal brain networks play an important role in the early diagnosis and mechanisms underlying SCD. However, most of the previous studies focused on a single modality, and lacked correlation analysis between different modal biomarkers and brain regions. In order to further explore the specific characteristic of the multimodal brain networks in the stage of SCD, 22 individuals with SCD and 20 matched healthy controls (HCs) were recruited in the present study. We constructed the individual morphological, structural and functional brain networks based on 3D-T1 structural magnetic resonance imaging (sMRI), diffusion tensor imaging (DTI) and resting-state functional magnetic resonance imaging (rs-fMRI), respectively. A t-test was used to select the connections with significant difference, and a multi-kernel support vector machine (MK-SVM) was applied to combine the selected multimodal connections to distinguish SCD from HCs. Moreover, we further identified the consensus connections of brain networks as the most discriminative features to explore the pathological mechanisms and potential biomarkers associated with SCD. Our results shown that the combination of three modal connections using MK-SVM achieved the best classification performance, with an accuracy of 92.68%, sensitivity of 95.00%, and specificity of 90.48%. Furthermore, the consensus connections and hub nodes based on the morphological, structural, and functional networks identified in our study exhibited abnormal cortical-subcortical connections in individuals with SCD. In addition, the functional networks presented more discriminative connections and hubs in the cortical-subcortical regions, and were found to perform better in distinguishing SCD from HCs. Therefore, our findings highlight the role of the cortical-subcortical circuit in individuals with SCD from the perspective of a multimodal brain network, providing potential biomarkers for the diagnosis and prediction of the preclinical stage of AD.

Mouse prefrontal cortex represents learned rules for categorization

The ability to categorize sensory stimuli is crucial for an animal’s survival in a complex environment. Memorizing categories instead of individual exemplars enables greater behavioural flexibility and is computationally advantageous. Neurons that show category selectivity have been found in several areas of the mammalian neocortex^1–4, but the prefrontal cortex seems to have a prominent role^4,5 in this context. Specifically, in primates that are extensively trained on a categorization task, neurons in the prefrontal cortex rapidly and flexibly represent learned categories^6,7. However, how these representations first emerge in naive animals remains unexplored, leaving it unclear whether flexible representations are gradually built up as part of semantic memory or assigned more or less instantly during task execution^8,9. Here we investigate the formation of a neuronal category representation throughout the entire learning process by repeatedly imaging individual cells in the mouse medial prefrontal cortex. We show that mice readily learn rule-based categorization and generalize to novel stimuli. Over the course of learning, neurons in the prefrontal cortex display distinct dynamics in acquiring category selectivity and are differentially engaged during a later switch in rules. A subset of neurons selectively and uniquely respond to categories and reflect generalization behaviour. Thus, a category representation in the mouse prefrontal cortex is gradually acquired during learning rather than recruited ad hoc. This gradual process suggests that neurons in the medial prefrontal cortex are part of a specific semantic memory for visual categories.

Neurons in the mouse medial prefrontal cortex acquire category-selective responses with learning.

Mouse visual cortex areas represent perceptual and semantic features of learned visual categories

Associative memories are stored in distributed networks extending across multiple brain regions. However, it is unclear to what extent sensory cortical areas are part of these networks. Using a paradigm for visual category learning in mice, we investigated whether perceptual and semantic features of learned category associations are already represented at the first stages of visual information processing in the neocortex. Mice learned categorizing visual stimuli, discriminating between categories and generalizing within categories. Inactivation experiments showed that categorization performance was contingent on neuronal activity in the visual cortex. Long-term calcium imaging in nine areas of the visual cortex identified changes in feature tuning and category tuning that occurred during this learning process, most prominently in the postrhinal area (POR). These results provide evidence for the view that associative memories form a brain-wide distributed network, with learning in early stages shaping perceptual representations and supporting semantic content downstream.

Habit learning in hierarchical cortex-basal ganglia loops

How do the multiple cortico-basal ganglia-thalamo-cortical loops interact? Are they parallel and fully independent or controlled by an arbitrator, or are they hierarchically organized? We introduce here a set of four key concepts, integrated and evaluated by means of a neuro-computational model, that bring together current ideas regarding cortex-basal ganglia interactions in the context of habit learning. According to key concept 1, each loop learns to select an intermediate objective at a different abstraction level, moving from goals in the ventral striatum to motor in the putamen. Key concept 2 proposes that the cortex integrates the basal ganglia selection with environmental information regarding the achieved objective. Key concept 3 claims shortcuts between loops, and key concept 4 predicts that loops compute their own prediction error signal for learning. Computational benefits of the key concepts are demonstrated. Contrasting with former concepts of habit learning, the loops collaborate to select goal-directed actions while training slower shortcuts develops habitual responses.

Beyond the "Conceptual Nervous System": Can computational cognitive neuroscience transform learning theory?

In the last century, learning theory has been dominated by an approach assuming that associations between hypothetical representational nodes can support the acquisition of knowledge about the environment. The similarities between this approach and connectionism did not go unnoticed to learning theorists, with many of them explicitly adopting a neural network approach in the modeling of learning phenomena. Skinner famously criticized such use of hypothetical neural structures for the explanation of behavior (the "Conceptual Nervous System"), and one aspect of his criticism has proven to be correct: theory underdetermination is a pervasive problem in cognitive modeling in general, and in associationist and connectionist models in particular. That is, models implementing two very different cognitive processes often make the exact same behavioral predictions, meaning that important theoretical questions posed by contrasting the two models remain unanswered. We show through several examples that theory underdetermination is common in the learning theory literature, affecting the solvability of some of the most important theoretical problems that have been posed in the last decades. Computational cognitive neuroscience (CCN) offers a solution to this problem, by including neurobiological constraints in computational models of behavior and cognition. Rather than simply being inspired by neural computation, CCN models are built to reflect as much as possible about the actual neural structures thought to underlie a particular behavior. They go beyond the "Conceptual Nervous System" and offer a true integration of behavioral and neural levels of analysis.

Holistic Reinforcement Learning: The Role of Structure and Attention

Compact representations of the environment allow humans to behave efficiently in a complex world. Reinforcement learning models capture many behavioral and neural effects but do not explain recent findings showing that structure in the environment influences learning. In parallel, Bayesian cognitive models predict how humans learn structured knowledge but do not have a clear neurobiological implementation. We propose an integration of these two model classes in which structured knowledge learned via approximate Bayesian inference acts as a source of selective attention. In turn, selective attention biases reinforcement learning towards relevant dimensions of the environment. An understanding of structure learning will help to resolve the fundamental challenge in decision science: explaining why people make the decisions they do.

A Computational Model of Dual Competition between the Basal Ganglia and the Cortex

We propose a model that includes interactions between the cortex, the basal ganglia (BG), and the thalamus based on a dual competition. We hypothesize that the striatum, the subthalamic nucleus (STN), the internal globus pallidus (GPi), the thalamus, and the cortex are involved in closed feedback loops through the hyperdirect and direct pathways. These loops support a competition process that results in the ability of BG to make a cognitive decision followed by a motor one. Considering lateral cortical interactions, another competition takes place inside the cortex allowing the latter to make a cognitive and a motor decision. We show how this dual competition endows the model with two regimes. One is driven by reinforcement learning and the other by Hebbian learning. The final decision is made according to a combination of these two mechanisms with a gradual transfer from the former to the latter. We confirmed these theoretical results on primates (Macaca mulatta) using a novel paradigm predicted by the model.