Rule activation and ventromedial prefrontal engagement support accurate stopping in self-paced learning

Exploration versus exploitation decisions in the human brain: A systematic review of functional neuroimaging and neuropsychological studies

Thoughts and actions are often driven by a decision to either explore new avenues with unknown outcomes, or to exploit known options with predictable outcomes. Yet, the neural mechanisms underlying this exploration-exploitation trade-off in humans remain poorly understood. This is attributable to variability in the operationalization of exploration and exploitation as psychological constructs, as well as the heterogeneity of experimental protocols and paradigms used to study these choice behaviours. To address this gap, here we present a comprehensive review of the literature to investigate the neural basis of explore-exploit decision-making in humans. We first conducted a systematic review of functional magnetic resonance imaging (fMRI) studies of exploration-versus exploitation-based decision-making in healthy adult humans during foraging, reinforcement learning, and information search. Eleven fMRI studies met inclusion criterion for this review. Adopting a network neuroscience framework, synthesis of the findings across these studies revealed that exploration-based choice was associated with the engagement of attentional, control, and salience networks. In contrast, exploitation-based choice was associated with engagement of default network brain regions. We interpret these results in the context of a network architecture that supports the flexible switching between externally and internally directed cognitive processes, necessary for adaptive, goal-directed behaviour. To further investigate potential neural mechanisms underlying the exploration-exploitation trade-off we next surveyed studies involving neurodevelopmental, neuropsychological, and neuropsychiatric disorders, as well as lifespan development, and neurodegenerative diseases. We observed striking differences in patterns of explore-exploit decision-making across these populations, again suggesting that these two decision-making modes are supported by independent neural circuits. Taken together, our review highlights the need for precision-mapping of the neural circuitry and behavioural correlates associated with exploration and exploitation in humans. Characterizing exploration versus exploitation decision-making biases may offer a novel, trans-diagnostic approach to assessment, surveillance, and intervention for cognitive decline and dysfunction in normal development and clinical populations.

Identifying the neural dynamics of category decisions with computational model-based functional magnetic resonance imaging

Successful categorization requires a careful coordination of attention, representation, and decision making. Comprehensive theories that span levels of analysis are key to understanding the computational and neural dynamics of categorization. Here, we build on recent work linking neural representations of category learning to computational models to investigate how category decision making is driven by neural signals across the brain. We uniquely combine functional magnetic resonance imaging with drift diffusion and exemplar-based categorization models to show that trial-by-trial fluctuations in neural activation from regions of occipital, cingulate, and lateral prefrontal cortices are linked to category decisions. Notably, only lateral prefrontal cortex activation was associated with exemplar-based model predictions of trial-by-trial category evidence. We propose that these brain regions underlie distinct functions that contribute to successful category learning.

Resolving the Electroencephalographic Correlates of Rapid Goal-Directed Chunking in the Frontal-Parietal Network

Previous studies have revealed a specific role of the prefrontal-parietal network in rapid goal-directed chunking (RGDC), which dissociates prefrontal activity related to chunking from parietal working memory demands. However, it remains unknown how the prefrontal and parietal cortices collaborate to accomplish RGDC. To this end, a novel experimental design was used that presented Chinese characters in a chunking task, testing eighteen undergraduate students (9 females, mean age = 22.4 years) while recoding the electroencephalogram (EEG). In the experiment, radical-level chunking was accomplished in a timely stringent way (RT = 1485 ms, SD = 371 ms), whereas the stroke-level chunking was accomplished less coherently (RT = 3278 ms, SD = 1083 ms). By comparing the differences between radical-level chunking vs. stroke-level chunking, we were able to dissociate the chunking processes in the radical-level chunking condition within the analyzed time window (-200 to 1300 ms). The chunking processes resulted in an early increase of gamma band synchronization over parietal and occipital cortices, followed by enhanced power in the beta-gamma band (25-38 Hz) over frontal areas. We suggest that the posterior rhythmic activities in the gamma band may underlie the processes that are directly associated with perceptual manipulations of chunking, while the subsequent beta-gamma activation over frontal areas appears to reflect a post-evaluation process such as reinforcement of the selected rules over alternative solutions, which may be an important characteristic of goal-directed chunking.

Occipitotemporal representations reflect individual differences in conceptual knowledge

Through selective attention, decision-makers can learn to ignore behaviorally irrelevant stimulus dimensions. This can improve learning and increase the perceptual discriminability of relevant stimulus information. Across cognitive models of categorization, this is typically accomplished through the inclusion of attentional parameters, which provide information about the importance assigned to each stimulus dimension by each participant. The effect of these parameters on psychological representation is often described geometrically, such that perceptual differences over relevant psychological dimensions are accentuated (or stretched), and differences over irrelevant dimensions are down-weighted (or compressed). In sensory and association cortex, representations of stimulus features are known to covary with their behavioral relevance. Although this implies that neural representational space might closely resemble that hypothesized by formal categorization theory, to date, attentional effects in the brain have been demonstrated through powerful experimental manipulations (e.g., contrasts between relevant and irrelevant features). This approach sidesteps the role of idiosyncratic conceptual knowledge in guiding attention to useful information sources. To bridge this divide, we used formal categorization models, which were fit to behavioral data, to make inferences about the concepts and strategies used by individual participants during decision-making. We found that when greater attentional weight was devoted to a particular visual feature (e.g., "color"), its value (e.g., "red") was more accurately decoded from occipitotemporal cortex. We also found that this effect was sufficiently sensitive to reflect individual differences in conceptual knowledge, indicating that occipitotemporal stimulus representations are embedded within a space closely resembling that formalized by classic categorization theory. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Model-based fMRI reveals dissimilarity processes underlying base rate neglect

Extensive evidence suggests that people use base rate information inconsistently in decision making. A classic example is the inverse base rate effect (IBRE), whereby participants classify ambiguous stimuli sharing features of both common and rare categories as members of the rare category. Computational models of the IBRE have posited that it arises either from associative similarity-based mechanisms or from dissimilarity-based processes that may depend on higher-level inference. Here we develop a hybrid model, which posits that similarity- and dissimilarity-based evidence both contribute to the IBRE, and test it using functional magnetic resonance imaging data collected from human subjects completing an IBRE task. Consistent with our model, multivoxel pattern analysis reveals that activation patterns on ambiguous test trials contain information consistent with dissimilarity-based processing. Further, trial-by-trial activation in left rostrolateral prefrontal cortex tracks model-based predictions for dissimilarity-based processing, consistent with theories positing a role for higher-level symbolic processing in the IBRE.