Search and the Aging Mind: The Promise and Limits of the Cognitive Control Hypothesis of Age Differences in Search

Exploration versus exploitation decisions in the human brain: A systematic review of functional neuroimaging and neuropsychological studies

Thoughts and actions are often driven by a decision to either explore new avenues with unknown outcomes, or to exploit known options with predictable outcomes. Yet, the neural mechanisms underlying this exploration-exploitation trade-off in humans remain poorly understood. This is attributable to variability in the operationalization of exploration and exploitation as psychological constructs, as well as the heterogeneity of experimental protocols and paradigms used to study these choice behaviours. To address this gap, here we present a comprehensive review of the literature to investigate the neural basis of explore-exploit decision-making in humans. We first conducted a systematic review of functional magnetic resonance imaging (fMRI) studies of exploration-versus exploitation-based decision-making in healthy adult humans during foraging, reinforcement learning, and information search. Eleven fMRI studies met inclusion criterion for this review. Adopting a network neuroscience framework, synthesis of the findings across these studies revealed that exploration-based choice was associated with the engagement of attentional, control, and salience networks. In contrast, exploitation-based choice was associated with engagement of default network brain regions. We interpret these results in the context of a network architecture that supports the flexible switching between externally and internally directed cognitive processes, necessary for adaptive, goal-directed behaviour. To further investigate potential neural mechanisms underlying the exploration-exploitation trade-off we next surveyed studies involving neurodevelopmental, neuropsychological, and neuropsychiatric disorders, as well as lifespan development, and neurodegenerative diseases. We observed striking differences in patterns of explore-exploit decision-making across these populations, again suggesting that these two decision-making modes are supported by independent neural circuits. Taken together, our review highlights the need for precision-mapping of the neural circuitry and behavioural correlates associated with exploration and exploitation in humans. Characterizing exploration versus exploitation decision-making biases may offer a novel, trans-diagnostic approach to assessment, surveillance, and intervention for cognitive decline and dysfunction in normal development and clinical populations.

From exploration to exploitation: a shifting mental mode in late life development

Changes in cognition, affect, and brain function combine to promote a shift in the nature of mentation in older adulthood, favoring exploitation of prior knowledge over exploratory search as the starting point for thought and action. Age-related exploitation biases result from the accumulation of prior knowledge, reduced cognitive control, and a shift toward affective goals. These are accompanied by changes in cortical networks, as well as attention and reward circuits. By incorporating these factors into a unified account, the exploration-to-exploitation shift offers an integrative model of cognitive, affective, and brain aging. Here, we review evidence for this model, identify determinants and consequences, and survey the challenges and opportunities posed by an exploitation-biased mental mode in later life.

Hybrid foraging search in younger and older age

In hybrid foraging tasks, observers search visual displays, so called patches, for multiple instances of any of several types of targets with the goal of collecting targets as quickly as possible. Here, targets were photorealistic objects. Younger and older adults collected targets by mouse clicks. They could move to the next patch whenever they decided to do so. The number of targets held in memory varied between 8 and 64 objects, and the number of items (targets and distractors) in the patches varied between 60 and 105 objects. Older adults foraged somewhat less efficiently than younger adults due to a more exploitative search strategy. When target items became depleted in a patch and search slowed down, younger adults acted according to the optimal foraging theory and moved on to the next patch when the instantaneous rate of collection was close to their average rate of collection. Older adults, by contrast, were more likely to stay longer and spend time searching for the last few targets. Within a patch, both younger and older adults tended to collect the same type of target in "runs." This behavior is more efficient than continual switching between target types. Furthermore, after correction for general age-related slowing, RT × set size functions revealed largely preserved attention and memory functions in older age. Hybrid foraging tasks share features with important real-world search tasks. Differences between younger and older observers on this task may therefore help to explain age differences in many complex search tasks of daily life. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Adult age differences in information foraging in an interactive reading environment

When learning about a single topic in natural reading environments, readers are confronted with multiple sources varying in the type and amount of information. In this situation, readers are free to adaptively respond to the constraints of the environment (e.g., through selection of resources and time allocation for study), but there may be costs of exploring and switching between sources (e.g., disruption of attention, opportunity costs for study). From an ecological perspective, such properties of the environment are expected to influence learning strategies. In the current study, we used a novel reading paradigm to investigate age differences in the effects of information richness (i.e., sentence elaboration) and costs of switching between texts (i.e., time delay) on selection of sources and study time allocation. Consistent with the ecological view, participants progressed from less informative to more informative texts. Furthermore, increased switch cost led to a tendency to allocate more effort to easier materials and to greater persistence in reading, which in turn, led to better memory in both immediate and delayed recall. Older adults showed larger effects of switch cost, such that the age difference in delayed recall was eliminated in the high switch cost condition. Based on an ecological paradigm of reading that affords choice and self-regulation, our study provided evidence for preservation with age in the ability to adapt to changing learning environments so as to improve performance. (PsycINFO Database Record