Cognitive flexibility and decision-making in a model of conditional visuomotor associations

Neural Mechanisms That Make Perceptual Decisions Flexible

Neural mechanisms of perceptual decision making have been extensively studied in experimental settings that mimic stable environments with repeating stimuli, fixed rules, and payoffs. In contrast, we live in an ever-changing environment and have varying goals and behavioral demands. To accommodate variability, our brain flexibly adjusts decision-making processes depending on context. Here, we review a growing body of research that explores the neural mechanisms underlying this flexibility. We highlight diverse forms of context dependency in decision making implemented through a variety of neural computations. Context-dependent neural activity is observed in a distributed network of brain structures, including posterior parietal, sensory, motor, and subcortical regions, as well as the prefrontal areas classically implicated in cognitive control. We propose that investigating the distributed network underlying flexible decisions is key to advancing our understanding and discuss a path forward for experimental and theoretical investigations.

Conceptual Knowledge Influences Decision Making Differently in Individuals with High or Low Cognitive Flexibility: An ERP Study

OBJECTIVE

Studies using the Iowa Gambling Task (IGT) have distinguished between good and bad decision makers and have provided an explanation for deficits in decision making. Previous studies have demonstrated a link between Wisconsin Card Sorting Test (WCST) performance and IGT performance, but the results were not consistent and failed to explain why WCST performance can predict IGT performance. The present study aimed to demonstrate that WCST performance can predict IGT performance and to identify the cognitive component of the WCST that affects IGT performance using event-related potentials (ERPs).

METHODS

In this study, 39 healthy subjects (5 subjects were excluded) were divided into a high group and a low group based on their global score on the WCST. A single-choice version of the IGT was used to eliminate the impact of retrieval strategies on the choice evaluation process and interference due to uncorrelated decks. Differences in the underlying neural mechanisms and explicit knowledge between the two groups during the three stages of the decision-making process were described.

RESULTS

Based on the information processing perspective, we divided the decision-making process into three stages: choice evaluation, response selection, and feedback processing. The behavioral results showed that the highly cognitively flexible participants performed better on the IGT and acquired more knowledge of the task. The ERP results showed that during the choice evaluation stage, the P300 recorded from central and parietal regions when a bad deck appeared was larger in the high group participants than in the low group participants. During the response selection stage, the effect of choice type was significant only in the frontal region in the high group, with a larger effect for passing. During the feedback evaluation stage, a larger FRN was evoked for a loss than for a win in the high group, whereas the FRN effect was absent in the low group.

CONCLUSION

Compared with the participants with low cognitive flexibility, the participants with high cognitive flexibility performed better on the IGT, acquired more knowledge of the task, and displayed more obvious somatic markers. The low group participants showed reduced working memory abilities during the choice evaluation stage. The appropriate somatic markers reflected by the DPN is formed only when conceptual knowledge is gained in the response selection stage. The absence of an FRN effect in the subjects who performed poorly on the WCST suggests a significant deficit in feedback learning and reward prediction.

A mechanism for decision rule discrimination by supplementary eye field neurons

A decision to select an action from alternatives is often guided by rules that flexibly map sensory inputs to motor outputs when certain conditions are satisfied. However, the neural mechanisms underlying rule-based decision making remain poorly understood. Two complementary types of neurons in the supplementary eye field (SEF) of macaques have been identified that modulate activity differentially to interpret rules in an ocular go-nogo task, which stipulates that the animal either visually pursue a moving object if it intersects a visible zone ('go'), or maintain fixation if it does not ('nogo'). These neurons discriminate between go and nogo rule-states by increasing activity to signal their preferred (agonist) rule-state and decreasing activity to signal their non-preferred (antagonist) rule-state. In the current study, we found that SEF neurons decrease activity in anticipation of the antagonist rule-state, and do so more rapidly when the rule-state is easier to predict. This rapid decrease in activity could underlie a process of elimination in which trajectories that do not invoke the preferred rule-state receive no further computational resources. Furthermore, discrimination between difficult and easy trials in the antagonist rule-state occurs prior to when discrimination within the agonist rule-state occurs. A winner-take-all like model that incorporates a pair of mutually inhibited integrators to accumulate evidence in favor of either the decision to pursue or the decision to continue fixation accounts for the observed neural phenomena.

Inter-synaptic learning of combination rules in a cortical network model

Selecting responses in working memory while processing combinations of stimuli depends strongly on their relations stored in long-term memory. However, the learning of XOR-like combinations of stimuli and responses according to complex rules raises the issue of the non-linear separability of the responses within the space of stimuli. One proposed solution is to add neurons that perform a stage of non-linear processing between the stimuli and responses, at the cost of increasing the network size. Based on the non-linear integration of synaptic inputs within dendritic compartments, we propose here an inter-synaptic (IS) learning algorithm that determines the probability of potentiating/depressing each synapse as a function of the co-activity of the other synapses within the same dendrite. The IS learning is effective with random connectivity and without either a priori wiring or additional neurons. Our results show that IS learning generates efficacy values that are sufficient for the processing of XOR-like combinations, on the basis of the sole correlational structure of the stimuli and responses. We analyze the types of dendrites involved in terms of the number of synapses from pre-synaptic neurons coding for the stimuli and responses. The synaptic efficacy values obtained show that different dendrites specialize in the detection of different combinations of stimuli. The resulting behavior of the cortical network model is analyzed as a function of inter-synaptic vs. Hebbian learning. Combinatorial priming effects show that the retrospective activity of neurons coding for the stimuli trigger XOR-like combination-selective prospective activity of neurons coding for the expected response. The synergistic effects of inter-synaptic learning and of mixed-coding neurons are simulated. The results show that, although each mechanism is sufficient by itself, their combined effects improve the performance of the network.

Why we may not find intentions in the brain

Intentions are commonly conceived of as discrete mental states that are the direct cause of actions. In the last several decades, neuroscientists have taken up the project of finding the neural implementation of intentions, and a number of areas have been posited as implementing these states. We argue, however, that the processes underlying action initiation and control are considerably more dynamic and context sensitive than the concept of intention can allow for. Therefore, adopting the notion of 'intention' in neuroscientific explanations can easily lead to misinterpretation of the data, and can negatively influence investigation into the neural correlates of intentional action. We suggest reinterpreting the mechanisms underlying intentional action, and we will discuss the elements that such a reinterpretation needs to account for.

Internal representation of task rules by recurrent dynamics: the importance of the diversity of neural responses

Neural activity of behaving animals, especially in the prefrontal cortex, is highly heterogeneous, with selective responses to diverse aspects of the executed task. We propose a general model of recurrent neural networks that perform complex rule-based tasks, and we show that the diversity of neuronal responses plays a fundamental role when the behavioral responses are context-dependent. Specifically, we found that when the inner mental states encoding the task rules are represented by stable patterns of neural activity (attractors of the neural dynamics), the neurons must be selective for combinations of sensory stimuli and inner mental states. Such mixed selectivity is easily obtained by neurons that connect with random synaptic strengths both to the recurrent network and to neurons encoding sensory inputs. The number of randomly connected neurons needed to solve a task is on average only three times as large as the number of neurons needed in a network designed ad hoc. Moreover, the number of needed neurons grows only linearly with the number of task-relevant events and mental states, provided that each neuron responds to a large proportion of events (dense/distributed coding). A biologically realistic implementation of the model captures several aspects of the activity recorded from monkeys performing context-dependent tasks. Our findings explain the importance of the diversity of neural responses and provide us with simple and general principles for designing attractor neural networks that perform complex computation.

Temporal context and conditional associative learning

BACKGROUND

We investigated how temporal context affects the learning of arbitrary visuo-motor associations. Human observers viewed highly distinguishable, fractal objects and learned to choose for each object the one motor response (of four) that was rewarded. Some objects were consistently preceded by specific other objects, while other objects lacked this task-irrelevant but predictive context.

RESULTS

The results of five experiments showed that predictive context consistently and significantly accelerated associative learning. A simple model of reinforcement learning, in which three successive objects informed response selection, reproduced our behavioral results.

CONCLUSIONS

Our results imply that not just the representation of a current event, but also the representations of past events, are reinforced during conditional associative learning. In addition, these findings are broadly consistent with the prediction of attractor network models of associative learning and their prophecy of a persistent representation of past objects.

Attractor concretion as a mechanism for the formation of context representations

Complex tasks often require the memory of recent events, the knowledge about the context in which they occur, and the goals we intend to reach. All this information is stored in our mental states. Given a set of mental states, reinforcement learning (RL) algorithms predict the optimal policy that maximizes future reward. RL algorithms assign a value to each already-known state so that discovering the optimal policy reduces to selecting the action leading to the state with the highest value. But how does the brain create representations of these mental states in the first place? We propose a mechanism for the creation of mental states that contain information about the temporal statistics of the events in a particular context. We suggest that the mental states are represented by stable patterns of reverberating activity, which are attractors of the neural dynamics. These representations are built from neurons that are selective to specific combinations of external events (e.g. sensory stimuli) and pre-existent mental states. Consistent with this notion, we find that neurons in the amygdala and in orbitofrontal cortex (OFC) often exhibit this form of mixed selectivity. We propose that activating different mixed selectivity neurons in a fixed temporal order modifies synaptic connections so that conjunctions of events and mental states merge into a single pattern of reverberating activity. This process corresponds to the birth of a new, different mental state that encodes a different temporal context. The concretion process depends on temporal contiguity, i.e. on the probability that a combination of an event and mental states follows or precedes the events and states that define a certain context. The information contained in the context thereby allows an animal to assign unambiguously a value to the events that initially appeared in different situations with different meanings.

Audiovisual matching in speech and nonspeech sounds: a neurodynamical model

Audiovisual speech perception provides an opportunity to investigate the mechanisms underlying multimodal processing. By using nonspeech stimuli, it is possible to investigate the degree to which audiovisual processing is specific to the speech domain. It has been shown in a match-to-sample design that matching across modalities is more difficult in the nonspeech domain as compared to the speech domain. We constructed a biophysically realistic neural network model simulating this experimental evidence. We propose that a stronger connection between modalities in speech underlies the behavioral difference between the speech and the nonspeech domain. This could be the result of more extensive experience with speech stimuli. Because the match-to-sample paradigm does not allow us to draw conclusions concerning the integration of auditory and visual information, we also simulated two further conditions based on the same paradigm, which tested the integration of auditory and visual information within a single stimulus. New experimental data for these two conditions support the simulation results and suggest that audiovisual integration of discordant stimuli is stronger in speech than in nonspeech stimuli. According to the simulations, the connection strength between auditory and visual information, on the one hand, determines how well auditory information can be assigned to visual information, and on the other hand, it influences the magnitude of multimodal integration.

Neurodynamics of the prefrontal cortex during conditional visuomotor associations

The prefrontal cortex is believed to be important for cognitive control, working memory, and learning. It is known to play an important role in the learning and execution of conditional visuomotor associations, a cognitive task in which stimuli have to be associated with actions by trial-and-error learning. In our modeling study, we sought to integrate several hypotheses on the function of the prefrontal cortex using a computational model, and compare the results to experimental data. We constructed a module of prefrontal cortex neurons exposed to two different inputs, which we envision to originate from the inferotemporal cortex and the basal ganglia. We found that working memory properties do not describe the dominant dynamics in the prefrontal cortex, but the activation seems to be transient, probably progressing along a pathway from sensory to motor areas. During the presentation of the cue, the dynamics of the prefrontal cortex is bistable, yielding a distinct activation for correct and error trails. We find that a linear change in network parameters relates to the changes in neural activity in consecutive correct trials during learning, which is important evidence for the underlying learning mechanisms.

Transient cognitive dynamics, metastability, and decision making

The idea that cognitive activity can be understood using nonlinear dynamics has been intensively discussed at length for the last 15 years. One of the popular points of view is that metastable states play a key role in the execution of cognitive functions. Experimental and modeling studies suggest that most of these functions are the result of transient activity of large-scale brain networks in the presence of noise. Such transients may consist of a sequential switching between different metastable cognitive states. The main problem faced when using dynamical theory to describe transient cognitive processes is the fundamental contradiction between reproducibility and flexibility of transient behavior. In this paper, we propose a theoretical description of transient cognitive dynamics based on the interaction of functionally dependent metastable cognitive states. The mathematical image of such transient activity is a stable heteroclinic channel, i.e., a set of trajectories in the vicinity of a heteroclinic skeleton that consists of saddles and unstable separatrices that connect their surroundings. We suggest a basic mathematical model, a strongly dissipative dynamical system, and formulate the conditions for the robustness and reproducibility of cognitive transients that satisfy the competing requirements for stability and flexibility. Based on this approach, we describe here an effective solution for the problem of sequential decision making, represented as a fixed time game: a player takes sequential actions in a changing noisy environment so as to maximize a cumulative reward. As we predict and verify in computer simulations, noise plays an important role in optimizing the gain.

Dynamics of sequential decision making

We suggest a new paradigm for intelligent decision-making suitable for dynamical sequential activity of animals or artificial autonomous devices that depends on the characteristics of the internal and external world. To do it we introduce a new class of dynamical models that are described by ordinary differential equations with a finite number of possibilities at the decision points, and also include rules solving this uncertainty. Our approach is based on the competition between possible cognitive states using their stable transient dynamics. The model controls the order of choosing successive steps of a sequential activity according to the environment and decision-making criteria. Two strategies (high-risk and risk-aversion conditions) that move the system out of an erratic environment are analyzed.