The credit assignment problem in cortico‐basal ganglia‐thalamic networks: A review, a problem and a possible solution

CBGTPy: An extensible cortico-basal ganglia-thalamic framework for modeling biological decision making

Here we introduce CBGTPy, a virtual environment for designing and testing goal-directed agents with internal dynamics that are modeled on the cortico-basal-ganglia-thalamic (CBGT) pathways in the mammalian brain. CBGTPy enables researchers to investigate the internal dynamics of the CBGT system during a variety of tasks, allowing for the formation of testable predictions about animal behavior and neural activity. The framework has been designed around the principle of flexibility, such that many experimental parameters in a decision making paradigm can be easily defined and modified. Here we demonstrate the capabilities of CBGTPy across a range of single and multi-choice tasks, highlighting the ease of set up and the biologically realistic behavior that it produces. We show that CBGTPy is extensible enough to apply to a range of experimental protocols and to allow for the implementation of model extensions with minimal developmental effort.

Author summary

We introduce a toolbox for producing biologically realistic simulations of the cortico-basal ganglia-thalamic dynamics during a variety of experimental tasks. The purpose is to foster the theory-experiment cycle, offering a tool for generating testable predictions of behavioral and neural responses that can be validated experimentally, in a framework that allows for simple updating as new experimental evidence emerges. We outline how our toolbox works and demonstrate its performance on a set of normative cognitive tasks.

Competing neural representations of choice shape evidence accumulation in humans

Making adaptive choices in dynamic environments requires flexible decision policies. Previously, we showed how shifts in outcome contingency change the evidence accumulation process that determines decision policies. Using in silico experiments to generate predictions, here we show how the cortico-basal ganglia-thalamic (CBGT) circuits can feasibly implement shifts in decision policies. When action contingencies change, dopaminergic plasticity redirects the balance of power, both within and between action representations, to divert the flow of evidence from one option to another. When competition between action representations is highest, the rate of evidence accumulation is the lowest. This prediction was validated in in vivo experiments on human participants, using fMRI, which showed that (1) evoked hemodynamic responses can reliably predict trial-wise choices and (2) competition between action representations, measured using a classifier model, tracked with changes in the rate of evidence accumulation. These results paint a holistic picture of how CBGT circuits manage and adapt the evidence accumulation process in mammals.

Dissecting the cross-trait effects of the FOXP2 GWAS hit on clinical and brain phenotypes in adults with ADHD

The Forkhead box P2 (FOXP2) encodes for a transcription factor with a broad role in embryonic development. It is especially represented among GWAS hits for neurodevelopmental disorders and related traits, including attention-deficit/hyperactivity disorder (ADHD), autism spectrum disorder, neuroticism, and risk-taking behaviors. While several functional studies are underway to understand the consequences of FOXP2 variation, this study aims to expand previous findings to clinically and genetically related phenotypes and neuroanatomical features among subjects with ADHD. The sample included 407 adults with ADHD and 463 controls. Genotyping was performed on the Infinium PsychArray-24 BeadChip, and the FOXP2 gene region was extracted. A gene-wide approach was adopted to evaluate the combined effects of FOXP2 variants (n = 311) on ADHD status, severity, comorbidities, and personality traits. Independent risk variants presenting potential functional effects were further tested for association with cortical surface areas in a subsample of cases (n = 87). The gene-wide analyses within the ADHD sample showed a significant association of the FOXP2 gene with harm avoidance (P = 0.001; P_FDR = 0.015) and nominal associations with hyperactivity symptoms (P = 0.026; P_FDR = 0.130) and antisocial personality disorder (P = 0.026; P_FDR = 0.130). An insertion/deletion variant (rs79622555) located downstream of FOXP2 was associated with the three outcomes and nominally with the surface area of superior parietal and anterior cingulate cortices. Our results extend and refine previous GWAS findings pointing to a role of FOXP2 in several neurodevelopment-related phenotypes, mainly those involving underlying symptomatic domains of self-regulation and inhibitory control. Taken together, the available evidence may constitute promising insights into the puzzle of the FOXP2-related pathophysiology.

Extended training improves the accuracy and efficiency of goal-directed reaching guided by supplemental kinesthetic vibrotactile feedback

Prior studies have shown that the accuracy and efficiency of reaching can be improved using novel sensory interfaces to apply task-specific vibrotactile feedback (VTF) during movement. However, those studies have typically evaluated performance after less than 1 h of training using VTF. Here, we tested the effects of extended training using a specific form of vibrotactile cues-supplemental kinesthetic VTF-on the accuracy and temporal efficiency of goal-directed reaching. Healthy young adults performed planar reaching with VTF encoding of the moving hand's instantaneous position, applied to the non-moving arm. We compared target capture errors and movement times before, during, and after approximately 10 h (20 sessions) of training on the VTF-guided reaching task. Initial performance of VTF-guided reaching showed that people were able to use supplemental VTF to improve reaching accuracy. Performance improvements were retained from one training session to the next. After 20 sessions of training, the accuracy and temporal efficiency of VTF-guided reaching were equivalent to or better than reaches performed with only proprioception. However, hand paths during VTF-guided reaching exhibited a persistent strategy where movements were decomposed into discrete sub-movements along the cardinal axes of the VTF display. We also used a dual-task condition to assess the extent to which performance gains in VTF-guided reaching resist dual-task interference. Dual-tasking capability improved over the 20 sessions, such that the primary VTF-guided reaching and a secondary choice reaction time task were performed with increasing concurrency. Thus, VTF-guided reaching is a learnable skill in young adults, who can achieve levels of accuracy and temporal efficiency equaling or exceeding those observed during movements guided only by proprioception. Future studies are warranted to explore learnability in older adults and patients with proprioceptive deficits, who might benefit from using wearable sensory augmentation technologies to enhance control of arm movements.

Identifying control ensembles for information processing within the cortico-basal ganglia-thalamic circuit

In situations featuring uncertainty about action-reward contingencies, mammals can flexibly adopt strategies for decision-making that are tuned in response to environmental changes. Although the cortico-basal ganglia thalamic (CBGT) network has been identified as contributing to the decision-making process, it features a complex synaptic architecture, comprised of multiple feed-forward, reciprocal, and feedback pathways, that complicate efforts to elucidate the roles of specific CBGT populations in the process by which evidence is accumulated and influences behavior. In this paper we apply a strategic sampling approach, based on Latin hypercube sampling, to explore how variations in CBGT network properties, including subpopulation firing rates and synaptic weights, map to variability of parameters in a normative drift diffusion model (DDM), representing algorithmic aspects of information processing during decision-making. Through the application of canonical correlation analysis, we find that this relationship can be characterized in terms of three low-dimensional control ensembles within the CBGT network that impact specific qualities of the emergent decision policy: responsiveness (a measure of how quickly evidence evaluation gets underway, associated with overall activity in corticothalamic and direct pathways), pliancy (a measure of the standard of evidence needed to commit to a decision, associated largely with overall activity in components of the indirect pathway of the basal ganglia), and choice (a measure of commitment toward one available option, associated with differences in direct and indirect pathways across action channels). These analyses provide mechanistic predictions about the roles of specific CBGT network elements in tuning the way that information is accumulated and translated into decision-related behavior.

Dynamic decision policy reconfiguration under outcome uncertainty

In uncertain or unstable environments, sometimes the best decision is to change your mind. To shed light on this flexibility, we evaluated how the underlying decision policy adapts when the most rewarding action changes. Human participants performed a dynamic two-armed bandit task that manipulated the certainty in relative reward (conflict) and the reliability of action-outcomes (volatility). Continuous estimates of conflict and volatility contributed to shifts in exploratory states by changing both the rate of evidence accumulation (drift rate) and the amount of evidence needed to make a decision (boundary height), respectively. At the trialwise level, following a switch in the optimal choice, the drift rate plummets and the boundary height weakly spikes, leading to a slow exploratory state. We find that the drift rate drives most of this response, with an unreliable contribution of boundary height across experiments. Surprisingly, we find no evidence that pupillary responses associated with decision policy changes. We conclude that humans show a stereotypical shift in their decision policies in response to environmental changes.

Cell-type-specific neuromodulation guides synaptic credit assignment in a spiking neural network

Synaptic connectivity provides the foundation for our present understanding of neuronal network function, but static connectivity cannot explain learning and memory. We propose a computational role for the diversity of cortical neuronal types and their associated cell-type–specific neuromodulators in improving the efficiency of synaptic weight adjustments for task learning in neuronal networks.

Brains learn tasks via experience-driven differential adjustment of their myriad individual synaptic connections, but the mechanisms that target appropriate adjustment to particular connections remain deeply enigmatic. While Hebbian synaptic plasticity, synaptic eligibility traces, and top-down feedback signals surely contribute to solving this synaptic credit-assignment problem, alone, they appear to be insufficient. Inspired by new genetic perspectives on neuronal signaling architectures, here, we present a normative theory for synaptic learning, where we predict that neurons communicate their contribution to the learning outcome to nearby neurons via cell-type–specific local neuromodulation. Computational tests suggest that neuron-type diversity and neuron-type–specific local neuromodulation may be critical pieces of the biological credit-assignment puzzle. They also suggest algorithms for improved artificial neural network learning efficiency.