Bistability of prefrontal states gates access to consciousness

Access of sensory information to consciousness has been linked to the ignition of content-specific representations in association cortices. How does ignition interact with intrinsic cortical state fluctuations to give rise to conscious perception? We addressed this question in the prefrontal cortex (PFC) by combining multi-electrode recordings with a binocular rivalry (BR) paradigm inducing spontaneously driven changes in the content of consciousness, inferred from the reflexive optokinetic nystagmus (OKN) pattern. We find that fluctuations between low-frequency (LF, 1-9 Hz) and beta (∼20-40 Hz) local field potentials (LFPs) reflect competition between spontaneous updates and stability of conscious contents, respectively. Both LF and beta events were locally modulated. The phase of the former locked differentially to the competing populations just before a spontaneous transition while the latter synchronized the neuronal ensemble coding the consciously perceived content. These results suggest that prefrontal state fluctuations gate conscious perception by mediating internal states that facilitate perceptual update and stability.

Executive Functions in Aging: An Experimental and Computational Study of the Wisconsin Card Sorting and Brixton Spatial Anticipation Tests

In order to explore the effect of normal aging on executive function, we tested 25 younger adults and 25 neurologically healthy older adults on the Wisconsin Card Sorting Test (WCST) and the Brixton Spatial Anticipation Test (BRXT), two classic tests of executive function. We found that older participants were more likely than younger participants to err on both tasks, but the additional errors of older participants tended to be related to task set maintenance and rule inference rather than perseveration. We further found that the tendency to perseverate (across all participants) on the WCST was related to the tendency to produce stimulus or response perseverations on the BRXT, rather than any tendency to perseverate on BRXT rule application. Finally, on both tasks, older participants were also slower, particularly on trials following an error, than younger participants. To explore the neurocomputational basis for the observed behaviours we then extended an existing model of schema-modulated action selection on the WCST to the BRXT. We argue on the basis of the model that the performance of older participants on both tasks reflects a slower update of schema thresholds within the basal ganglia, coupled with a decrease in sensitivity to feedback.

Examining the Trainability and Transferability of Working-Memory Gating Policies

Internal working memory (WM) gating control policies have been suggested to constitute a critical component of task-sets that can be learned and transferred to very similar task contexts (Bhandari and Badre (Cognition, 172, 33–43, 2018). Here, we attempt to expand these findings, examining whether such control policies can be also trained and transferred to other untrained cognitive control tasks, namely to task switching and AX-CPT. To this end, a context-processing WM task was used for training, allowing to manipulate either input (i.e., top-down selective entry of information into WM) or output (i.e., bottom-up selective retrieval of WM) gating control policies by employing either a context-first (CF) or context-last (CL) task structure, respectively. In this task, two contextual cues were each associated with two different stimuli. In CF condition, each trial began with a contextual cue, determining which of the two subsequent stimuli is target relevant. In contrast, in the CL condition the contextual cue appeared last, preceded by a target and non-target stimulus successively. Participants completed a task switching baseline assessment, followed by one practice and six training blocks with the WM context-processing training task. After completing training, task-switching and AX-CPT transfer blocks were administrated, respectively. As hypothesized, compared to CL training condition, CF training led to improved task-switching performance. However, contrary to our predictions, training type did not influence AX-CPT performance. Taken together, the current results provide further evidence that internal control policies are (1) inherent element of task-sets, also in task switching and (2) independent of S-R mappings. However, these results need to be cautiously interpreted due to baseline differences in task-switching performance between the conditions (overall slower RTs in the CF condition). Importantly though, our results open a new venue for the realm of cognitive enhancement, pointing here for the first time to the potential of control policies training in promoting wider transfer effects.

Tonic exploration governs both flexibility and lapses

In many cognitive tasks, lapses (spontaneous errors) are tacitly dismissed as the result of nuisance processes like sensorimotor noise, fatigue, or disengagement. However, some lapses could also be caused by exploratory noise: randomness in behavior that facilitates learning in changing environments. If so, then strategic processes would need only up-regulate (rather than generate) exploration to adapt to a changing environment. This view predicts that more frequent lapses should be associated with greater flexibility because these behaviors share a common cause. Here, we report that when rhesus macaques performed a set-shifting task, lapse rates were negatively correlated with perseverative error frequency across sessions, consistent with a common basis in exploration. The results could not be explained by local failures to learn. Furthermore, chronic exposure to cocaine, which is known to impair cognitive flexibility, did increase perseverative errors, but, surprisingly, also improved overall set-shifting task performance by reducing lapse rates. We reconcile these results with a state-switching model in which cocaine decreases exploration by deepening attractor basins corresponding to rule states. These results support the idea that exploratory noise contributes to lapses, affecting rule-based decision-making even when it has no strategic value, and suggest that one key mechanism for regulating exploration may be the depth of rule states.

Gated spiking neural network using Iterative Free-Energy Optimization and rank-order coding for structure learning in memory sequences (INFERNO GATE)

We present a framework based on iterative free-energy optimization with spiking neural networks for modeling the fronto-striatal system (PFC-BG) for the generation and recall of audio memory sequences. In line with neuroimaging studies carried out in the PFC, we propose a genuine coding strategy using the gain-modulation mechanism to represent abstract sequences based solely on the rank and location of items within them. Based on this mechanism, we show that we can construct a repertoire of neurons sensitive to the temporal structure in sequences from which we can represent any novel sequences. Free-energy optimization is then used to explore and to retrieve the missing indices of the items in the correct order for executive control and compositionality. We show that the gain-modulation mechanism permits the network to be robust to variabilities and to have long-term dependencies as it implements a gated recurrent neural network. This model, called Inferno Gate, is an extension of the neural architecture Inferno standing for Iterative Free-Energy Optimization of Recurrent Neural Networks with Gating or Gain-modulation. In experiments performed with an audio database of ten thousand MFCC vectors, Inferno Gate is capable of encoding efficiently and retrieving chunks of fifty items length. We then discuss the potential of our network to model the features of working memory in the PFC-BG loop for structural learning, goal-direction and hierarchical reinforcement learning.

A neural model of hierarchical reinforcement learning

We develop a novel, biologically detailed neural model of reinforcement learning (RL) processes in the brain. This model incorporates a broad range of biological features that pose challenges to neural RL, such as temporally extended action sequences, continuous environments involving unknown time delays, and noisy/imprecise computations. Most significantly, we expand the model into the realm of hierarchical reinforcement learning (HRL), which divides the RL process into a hierarchy of actions at different levels of abstraction. Here we implement all the major components of HRL in a neural model that captures a variety of known anatomical and physiological properties of the brain. We demonstrate the performance of the model in a range of different environments, in order to emphasize the aim of understanding the brain’s general reinforcement learning ability. These results show that the model compares well to previous modelling work and demonstrates improved performance as a result of its hierarchical ability. We also show that the model’s behaviour is consistent with available data on human hierarchical RL, and generate several novel predictions.

Investigation of Biases and Compensatory Strategies Using a Probabilistic Variant of the Wisconsin Card Sorting Test

The Wisconsin Card Sorting Test (WCST) evaluates a subject’s ability to shift to a new pattern of behavior in response to the presentation of unexpected negative feedback. The present study introduces a novel version of the traditional WCST by integrating a probabilistic component into its traditional rule shifting to add uncertainty to the task, as well as the option to forage for information during any particular trial. These changes transformed a task that is trivial for neurotypical individuals into a challenging environment useful for evaluating biases and compensatory strategizing. Sixty subjects performed the probabilistic WCST at four uncertainty levels to determine the effect of uncertainty on subject performance and strategy. Results revealed that increasing the level of uncertainty during a run of trials correlated with a reduction in rational strategizing in favor of both random choice and information foraging, evoking biases and suboptimal strategies such as satisfaction of search, negativity bias, and probability matching.

Development of compositional and contextual communicable congruence in robots by using dynamic neural network models

The current study presents neurorobotics experiments on acquisition of skills for "communicable congruence" with human via learning. A dynamic neural network model which is characterized by its multiple timescale dynamics property was utilized as a neuromorphic model for controlling a humanoid robot. In the experimental task, the humanoid robot was trained to generate specific sequential movement patterns as responding to various sequences of imperative gesture patterns demonstrated by the human subjects by following predefined compositional semantic rules. The experimental results showed that (1) the adopted MTRNN can achieve generalization by learning in the lower feature perception level by using a limited set of tutoring patterns, (2) the MTRNN can learn to extract compositional semantic rules with generalization in its higher level characterized by slow timescale dynamics, (3) the MTRNN can develop another type of cognitive capability for controlling the internal contextual processes as situated to on-going task sequences without being provided with cues for explicitly indicating task segmentation points. The analysis on the dynamic property developed in the MTRNN via learning indicated that the aforementioned cognitive mechanisms were achieved by self-organization of adequate functional hierarchy by utilizing the constraint of the multiple timescale property and the topological connectivity imposed on the network configuration. These results of the current research could contribute to developments of socially intelligent robots endowed with cognitive communicative competency similar to that of human.

Modeling the Evolution of Beliefs Using an Attentional Focus Mechanism

For making decisions in everyday life we often have first to infer the set of environmental features that are relevant for the current task. Here we investigated the computational mechanisms underlying the evolution of beliefs about the relevance of environmental features in a dynamical and noisy environment. For this purpose we designed a probabilistic Wisconsin card sorting task (WCST) with belief solicitation, in which subjects were presented with stimuli composed of multiple visual features. At each moment in time a particular feature was relevant for obtaining reward, and participants had to infer which feature was relevant and report their beliefs accordingly. To test the hypothesis that attentional focus modulates the belief update process, we derived and fitted several probabilistic and non-probabilistic behavioral models, which either incorporate a dynamical model of attentional focus, in the form of a hierarchical winner-take-all neuronal network, or a diffusive model, without attention-like features. We used Bayesian model selection to identify the most likely generative model of subjects' behavior and found that attention-like features in the behavioral model are essential for explaining subjects' responses. Furthermore, we demonstrate a method for integrating both connectionist and Bayesian models of decision making within a single framework that allowed us to infer hidden belief processes of human subjects.

A matched filter hypothesis for cognitive control

The prefrontal cortex exerts top-down influences on several aspects of higher-order cognition by functioning as a filtering mechanism that biases bottom-up sensory information toward a response that is optimal in context. However, research also indicates that not all aspects of complex cognition benefit from prefrontal regulation. Here we review and synthesize this research with an emphasis on the domains of learning and creative cognition, and outline how the appropriate level of cognitive control in a given situation can vary depending on the organism's goals and the characteristics of the given task. We offer a matched filter hypothesis for cognitive control, which proposes that the optimal level of cognitive control is task-dependent, with high levels of cognitive control best suited to tasks that are explicit, rule-based, verbal or abstract, and can be accomplished given the capacity limits of working memory and with low levels of cognitive control best suited to tasks that are implicit, reward-based, non-verbal or intuitive, and which can be accomplished irrespective of working memory limitations. Our approach promotes a view of cognitive control as a tool adapted to a subset of common challenges, rather than an all-purpose optimization system suited to every problem the organism might encounter.

Self-organizing high-order cognitive functions in artificial agents: implications for possible prefrontal cortex mechanisms

In our daily life, we often adapt plans and behaviors according to dynamically changing world circumstances, selecting activities that make us feel more confident about the future. In this adaptation, the prefrontal cortex (PFC) is believed to have an important role, applying executive control on other cognitive processes to achieve context switching and confidence monitoring; however, many questions remain open regarding the nature of neural processes supporting executive control. The current work explores possible mechanisms of this high-order cognitive function, transferring executing control in the domain of artificial cognitive systems. In particular, we study the self-organization of artificial neural networks accomplishing a robotic rule-switching task analogous to the Wisconsin Card Sorting Test. The obtained results show that behavioral rules may be encoded in neuro-dynamic attractors, with their geometric arrangements in phase space affecting the shaping of confidence. Analysis of the emergent dynamical structures suggests possible explanations of the interactions of high-level and low-level processes in the real brain.

Including cognitive biases and distance-based rewards in a connectionist model of complex problem solving

We present a cognitive, connectionist-based model of complex problem solving that integrates cognitive biases and distance-based and environmental rewards under a temporal-difference learning mechanism. The model is tested against experimental data obtained in a well-defined and planning-intensive problem. We show that incorporating cognitive biases (symmetry and simplicity) in a temporal-difference learning rule (SARSA) increases model adequacy-the solution space explored by biased models better fits observed human solutions. While learning from explicit rewards alone is intrinsically slow, adding distance-based rewards, a measure of closeness to goal, to the learning rule significantly accelerates learning. Finally, the model correctly predicts that explicit rewards have little impact on problem solvers' ability to discover optimal solutions.

Sequential Learning Models for the Wisconsin Card Sort Task: Assessing Processes in Substance Dependent Individuals

The Wisconsin Card Sort Task (WCST) is a commonly used neuropsychological test of executive or frontal lobe functioning. Traditional behavioral measures from the task (e.g., perseverative errors) distinguish healthy controls from clinical populations, but such measures can be difficult to interpret. In an attempt to supplement traditional measures, we developed and tested a family of sequential learning models that allowed for estimation of processes at the individual subject level in the WCST. Testing the model with substance dependent individuals and healthy controls, the model parameters significantly predicted group membership even when controlling for traditional behavioral measures from the task. Substance dependence was associated with a) slower attention shifting following punished trials and b) reduced decision consistency. Results suggest that model parameters may offer both incremental content validity and incremental predictive validity.

Implicit and explicit representations

During the past decades, the symbol grounding problem, as has been identified by Harnard [Harnard, S. (1990). The symbol grounding problem. Physica D: Nonlinear Phenomena, 42, 335-346], became a prominent problem in the cognitive science society. The idea that a symbol is much more than a mere meaningless token that can be processed through some algorithm, sheds new light on higher brain functions such as language and cognition. We present in this article a computational framework that may help in our understanding of the nature of grounded representations. Two models are briefly introduced that aim at emphasizing the difference we make between implicit and explicit representations.

Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system

The prefrontal cortex (PFC) has long been thought to serve as an 'executive' that controls the selection of actions and cognitive functions more generally. However, the mechanistic basis of this executive function has not been clearly specified often amounting to a homunculus. This paper reviews recent attempts to deconstruct this homunculus by elucidating the precise computational and neural mechanisms underlying the executive functions of the PFC. The overall approach builds upon existing mechanistic models of the basal ganglia (BG) and frontal systems known to play a critical role in motor control and action selection, where the BG provide a 'Go' versus 'NoGo' modulation of frontal action representations. In our model, the BG modulate working memory representations in prefrontal areas to support more abstract executive functions. We have developed a computational model of this system that is capable of developing human-like performance on working memory and executive control tasks through trial-and-error learning. This learning is based on reinforcement learning mechanisms associated with the midbrain dopaminergic system and its activation via the BG and amygdala. Finally, we briefly describe various empirical tests of this framework.

A neural network model of hippocampal-striatal-prefrontal interactions in contextual conditioning

The hippocampus is thought to be critical for encoding contextually bound memories and setting the context for ongoing behavior. However, the mechanisms by which the hippocampal-cortical system controls behavior are poorly understood. We propose a computational model in which the hippocampus exerts contextual control over motivated behavior by gating prefrontal cortex inputs to the nucleus accumbens. The model integrates the episodic memory functions of the hippocampus, the prefrontal role in representing the motivational stimuli and cognitive control, and the role of striatal regions in conditioned learning within a single theoretical framework. Simulation results are consistent with the hypothesis that hippocampal-prefrontal interactions may act as the neural substrate that allows contextual cues to override conditioned responses at the level of the nucleus accumbens. Prefrontal and hippocampal input overrides the predominant CS-US association if the context is inconsistent, and promotes flexible selection of previously learned associations and behaviors. Simulated transection of the fornix, effectively eliminating hippocampal and prefrontal influence over the nucleus accumbens, abolishes normal contextual modulation of behavior. The model is consistent with a wide range of empirical data.

Effects of synaptic depression and recovery on synchronous network activity

SUMMARY

The output of an artificial neural network of spiking neurons linked by glutamatergic synapses subject to use-dependent depression was compared with physiologic data obtained from rat hippocampal area CA3 in vitro. The authors evaluated how network burst initiation and termination was affected by activity-dependent depression and recovery under a variety of experimental conditions including neuronal membrane depolarization, altered glutamate release probability, the strength of synaptic inhibition, and long-term potentiation and long-term depression of recurrent glutamatergic synapses. The results of computational experiments agreed with the in vitro data and support the idea that synaptic properties, including activity-dependent depression and recovery, play important roles in the timing and duration of spontaneous bursts of network activity. This validated network model is useful for experiments that are not feasible in vitro, and makes possible the investigation of two-dimensional aspects of burst propagation and termination.

The neurodynamics underlying attentional control in set shifting tasks

In this work we address key phenomena observed with classical set shifting tasks as the "Wisconsin Card Sorting Test" or the "Stroop" task: Different types of errors and increased response times reflecting decreased attention. A component of major importance in these tasks is referred to as the "attentional control" thought to be implemented by the prefrontal cortex which acts primarily by an amplification of task relevant information. This mode of operation is illustrated by a neurodynamical model developed for a new kind of set shifting experiment: The Wisconsin-Delayed-Match-to-Sample task combines uninstructed shifts as investigated in Wisconsin-like tasks with a Delayed-Match-to-Sample paradigm. These newly developed WDMS experiments in conjunction with the neurodynamical simulations are able to explain the reason for decreased attention in set shifting experiments as well the different consequences of decreased attention in tasks requiring bivalent yes/no responses compared to tasks requiring multivalent responses.

The neuronal dynamics underlying cognitive flexibility in set shifting tasks

The ability to switch attention from one aspect of an object to another or in other words to switch the "attentional set" as investigated in tasks like the "Wisconsin Card Sorting Test" is commonly referred to as cognitive flexibility. In this work we present a biophysically detailed neurodynamical model which illustrates the neuronal base of the processes related to this cognitive flexibility. For this purpose we conducted behavioral experiments which allow the combined evaluation of different aspects of set shifting tasks: uninstructed set shifts as investigated in Wisconsin-like tasks, effects of stimulus congruency as investigated in Stroop-like tasks and the contribution of working memory as investigated in "Delayed-Match-to-Sample" tasks. The work describes how general experimental findings are usable to design the architecture of a biophysical detailed though minimalistic model with a high orientation on neurobiological findings and how, in turn, the simulations support experimental investigations. The resulting model is able to account for experimental and individual response times and error rates and enables the switch of attention as a system inherent model feature: The switching process suggested by the model is based on the memorization of the visual stimuli and does not require any synaptic learning. The operation of the model thus demonstrates with at least a high probability the neuronal dynamics underlying a key component of human behavior: the ability to adapt behavior according to context requirements--cognitive flexibility.

Testing the semantic differential as a model of task processes with the implicit association test

In this study, we examined the hypothesis that semantic judgment tasks share overlapping processes if they require processing on common dimensions but not if they require processing on orthogonal dimensions in semantic space (Osgood, Suci, & Tannenbaum, 1957). We tested the hypothesis with the implicit association test (IATl Greenwald, McGhee, & Schwartz, 1998) in three experiments. Consistent with the hypothesis, IAT effects (costs in reaction time because of incompatible response mapping between associated judgment tasks) occurred consistently when judgment tasks tapped into common semantic dimensions, whereas no IAT effect appeared when judgment tasks entailed processing on orthogonal semantic dimensions.

Banishing the homunculus: making working memory work

The prefrontal cortex has long been thought to subserve both working memory and "executive" function, but the mechanistic basis of their integrated function has remained poorly understood, often amounting to a homunculus. This paper reviews the progress in our laboratory and others pursuing a long-term research agenda to deconstruct this homunculus by elucidating the precise computational and neural mechanisms underlying these phenomena. We outline six key functional demands underlying working memory, and then describe the current state of our computational model of the prefrontal cortex and associated systems in the basal ganglia (BG). The model, called PBWM (prefrontal cortex, basal ganglia working memory model), relies on actively maintained representations in the prefrontal cortex, which are dynamically updated/gated by the basal ganglia. It is capable of developing human-like performance largely on its own by taking advantage of powerful reinforcement learning mechanisms, based on the midbrain dopaminergic system and its activation via the basal ganglia and amygdala. These learning mechanisms enable the model to learn to control both itself and other brain areas in a strategic, task-appropriate manner. The model can learn challenging working memory tasks, and has been corroborated by several important empirical studies.

Prefrontal cortex and flexible cognitive control: rules without symbols

Human cognitive control is uniquely flexible and has been shown to depend on prefrontal cortex (PFC). But exactly how the biological mechanisms of the PFC support flexible cognitive control remains a profound mystery. Existing theoretical models have posited powerful task-specific PFC representations, but not how these develop. We show how this can occur when a set of PFC-specific neural mechanisms interact with breadth of experience to self organize abstract rule-like PFC representations that support flexible generalization in novel tasks. The same model is shown to apply to benchmark PFC tasks (Stroop and Wisconsin card sorting), accurately simulating the behavior of neurologically intact and frontally damaged people.

Hippocampus, cortex, and basal ganglia: insights from computational models of complementary learning systems

We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work together to support cognitive and behavioral function in the mammalian brain. This framework is based on computational tradeoffs that arise in neural network models, where achieving one type of learning function requires very different parameters from those necessary to achieve another form of learning. For example, we dissociate the hippocampus from cortex with respect to general levels of activity, learning rate, and level of overlap between activation patterns. Similarly, the frontal cortex and associated basal ganglia system have important neural specializations not required of the posterior cortex system. Taken together, this overall cognitive architecture, which has been implemented in functioning computational models, provides a rich and often subtle means of explaining a wide range of behavioral and cognitive neuroscience data. Here, we summarize recent results in the domains of recognition memory, contextual fear conditioning, effects of basal ganglia lesions on stimulus-response and place learning, and flexible responding.

Neuroimaging studies of shifting attention: a meta-analysis

This paper reports a meta-analysis of neuroimaging studies of attention shifting and executive processes in working memory. We analyzed peak activation coordinates from 31 fMRI and PET studies of five types of shifting using kernel-based methods [NeuroImage 19 (2003) 513]. Analyses collapsing across different types of shifting gave more consistent results overall than analysis within individual types, suggesting a commonality across types of shifting. These areas shared substantial, significant overlap with regions derived from kernel-based analyses of reported peaks for executive processes in working memory (WM). The results suggest that there is a common set of brain regions active in diverse executive control operations, including medial prefrontal, superior and inferior parietal, medial parietal, and premotor cortices. However, within several of these regions, different types of switching produced spatially discriminable activation foci. Precise locations of meta analysis-derived regions from both attention shifting and working memory are defined electronically and may be used as regions of interest in future studies.

Neural correlates of switching set as measured in fast, event-related functional magnetic resonance imaging

Attentional switching has shown to involve several prefrontal and parietal brain regions. Most cognitive paradigms used to measure cognitive switching such as the Wisconsin Card Sorting Task (WCST) involve additional cognitive processes besides switching, in particular working memory (WM). It is, therefore, questionable whether prefrontal brain regions activated in these conditions, especially dorsolateral prefrontal cortex (DLPFC), are involved in cognitive switching per se, or are related to WM components involved in switching tasks. Functional magnetic resonance imaging (fMRI) was used to examine neural correlates of pure switching using a paradigm purposely designed to minimize WM functions. The switching paradigm required subjects to switch unpredictably between two spatial dimensions, clearly indicated throughout the task before each trial. Fast, event-related fMRI was used to compare neural activation associated with switch trials to that related to repeat trials in 20 healthy, right-handed, adult males. A large cluster of activation was observed in the right hemisphere, extending from inferior prefrontal and pre- and postcentral gyri to superior temporal and inferior parietal cortices. A smaller and more caudal cluster of homologous activation in the left hemisphere was accompanied by activation of left dorsolateral prefrontal cortex (DLPFC). We conclude that left DLPFC activation is involved directly in cognitive switching, in conjunction with parietal and temporal brain regions. Pre- and postcentral gyrus activation may be related to motor components of switching set.