Sequential Control Underlies Robust Ramping Dynamics in the Rostrolateral Prefrontal Cortex

Abstract task sequence initiation deficit dissociates anxiety disorders from obsessive-compulsive disorder and healthy controls

In everyday life, humans perform sequences of tasks. These tasks may be disrupted in people with obsessive-compulsive disorder (OCD). Symptoms, such as compulsions, can be considered sequential and often cause repetitions of tasks that disrupt daily living (e.g., checking the stove while cooking). Motor sequences have been used to study behavioral deficits in OCD. However, not all sequences are motor sequences. Some are more "abstract" in that they are composed of a series of tasks (e.g., chopping and stirring) rather than being dependent on individual actions or stimuli. These abstract task sequences require cognitive control mechanisms for their execution. Although theory has proposed deficits in these sequences in OCD as well, they have not been directly investigated. We tested the hypotheses that OCD participants exhibit deficits in the control mechanisms specific to abstract task sequences and more general flexible behavior (measured with task switching within the sequences), relative to health controls (HCs) and clinical controls (participants with anxiety disorders [ANX]). A total of 112 participants completed abstract task sequences consisting of simple categorization tasks. Surprisingly, participants with OCD did not perform worse than HCs or ANX. However, ANX participants showed impairments specific to sequential control that did not extend to more general flexible control. Thus, we showed a novel behavioral dissociation between OCD and ANX specific to abstract task sequential control. These results also implicate deficits in specific frontal sequential control neural circuitry in ANX and not in OCD, where implicit sequential deficits may more closely align with striatal circuits.

Different subregions of monkey lateral prefrontal cortex respond to abstract sequences and their components

Sequential information permeates daily activities, such as when watching for the correct series of buildings to determine when to get off the bus or train. These sequences include periodicity (the spacing of the buildings), the identity of the stimuli (the kind of house), and higher-order more abstract rules that may not depend on the exact stimulus (e.g. house, house, house, business). Previously, we found that the posterior fundus of area 46 in the monkey lateral prefrontal cortex (LPFC) responds to rule changes in such abstract visual sequences. However, it is unknown if this region responds to other components of the sequence, i.e., image periodicity and identity, in isolation. Further, it is unknown if this region dissociates from other, more ventral LPFC subregions that have been associated with sequences and their components. To address these questions, we used awake functional magnetic resonance imaging in three male macaque monkeys during two no-report visual tasks. One task contained abstract visual sequences, and the other contained no visual sequences but maintained the same image periodicity and identities. We found the fundus of area 46 responded only to abstract sequence rule violations. In contrast, the ventral bank of area 46 responded to changes in image periodicity and identity, but not changes in the abstract sequence. These results suggest a functional specialization within anatomical substructures of LPFC to signal different kinds of stimulus regularities. This specialization may provide key scaffolding to identify abstract patterns and construct complex models of the world for daily living.

Significance Statement

Daily tasks, such as a bus commute, require tracking or monitoring your place (same, same, same, different building) until your stop. Sequence components such as rule, periodicity (timing), and item identity are involved in this process. While prior work located responses to sequence rule changes to area 46 of monkey lateral prefrontal cortex (LPFC) using awake monkey fMRI, less was known about other components. We found that LPFC subregions differentiated between sequence components. Area 46 posterior fundus responded to abstract visual sequence rule changes, but not to changes in image periodicity or identity. The converse was true for the more ventral, adjacent shoulder region. These results suggest that interactions between adjacent LPFC subregions provide key scaffolding for complex daily behaviors.

Exploring the dynamics of prefrontal cortex in the interaction between orienteering experience and cognitive performance by fNIRS

Sporting experience plays a pivotal role in shaping exercise habits, with a mutually reinforcing relationship that enhances cognitive performance. The acknowledged plasticity of cognition driven by sports necessitates a comprehensive examination. Hence, this study delves into the dynamic intricacies of the prefrontal cortex, exploring the impact of orienteering experience on cognitive performance. Our findings contribute empirical evidence regarding the functional activation of specific brain regions bridging the nexus between experiential factors and cognitive capabilities. In this cross-sectional study, a cohort of forty-nine athletes was enrolled to meticulously examine behavioral variances and prefrontal cortex dynamics among orienteering athletes of varying experience levels across diverse non-specialized scenarios. These investigations involved the utilization of functional near-infrared spectroscopy (fNIRS) to detect alterations in oxygenated hemoglobin (HbO2). The high-experience expert group exhibited neurological efficiency, demonstrating significantly diminished brain activation in the dorsolateral prefrontal, left ventral lateral prefrontal, and right orbitofrontal regions compared to the low-experience group. Within the low-experience novice group, superior performance in the spatial memory task was observed compared to the mental rotation task, with consistently lower reaction times across all conditions compared to the high-experience group. Notably, cerebral blood oxygenation activation exhibited a significant reduction in the high-experience expert group compared to the low-experience novice group, irrespective of task type. The dorsolateral prefrontal lobe exhibited activation upon task onset, irrespective of experience level. Correct rates in the spatial memory task were consistently higher than those in the mental rotation task, while brain region activation was significantly greater during the mental rotation task than the spatial memory task." This study elucidates disparities in prefrontal cortex dynamics between highly seasoned experts and neophyte novices, showcasing a cognitive edge within the highly experienced cohort and a spatial memory advantage in the inexperienced group. Our findings contribute to the comprehension of the neural mechanisms that underlie the observed cognitive advantage and provide insights into the forebrain resources mobilized by orienteering experience during spatial cognitive tasks."

Monkey Dorsolateral Prefrontal Cortex Represents Abstract Visual Sequences during a No-Report Task

Monitoring sequential information is an essential component of our daily lives. Many of these sequences are abstract, in that they do not depend on the individual stimuli, but do depend on an ordered set of rules (e.g., chop then stir when cooking). Despite the ubiquity and utility of abstract sequential monitoring, little is known about its neural mechanisms. Human rostrolateral prefrontal cortex (RLPFC) exhibits specific increases in neural activity (i.e., "ramping") during abstract sequences. Monkey dorsolateral prefrontal cortex (DLPFC) has been shown to represent sequential information in motor (not abstract) sequence tasks, and contains a subregion, area 46, with homologous functional connectivity to human RLPFC. To test the prediction that area 46 may represent abstract sequence information, and do so with parallel dynamics to those found in humans, we conducted functional magnetic resonance imaging (fMRI) in three male monkeys. When monkeys performed no-report abstract sequence viewing, we found that left and right area 46 responded to abstract sequential changes. Interestingly, responses to rule and number changes overlapped in right area 46 and left area 46 exhibited responses to abstract sequence rules with changes in ramping activation, similar to that observed in humans. Together, these results indicate that monkey DLPFC monitors abstract visual sequential information, potentially with a preference for different dynamics in the two hemispheres. More generally, these results show that abstract sequences are represented in functionally homologous regions across monkeys and humans.SIGNIFICANCE STATEMENT Daily, we complete sequences that are "abstract" because they depend on an ordered set of rules (e.g., chop then stir when cooking) rather than the identity of individual items. Little is known about how the brain tracks, or monitors, this abstract sequential information. Based on previous human work showing abstract sequence related dynamics in an analogous area, we tested whether monkey dorsolateral prefrontal cortex (DLPFC), specifically area 46, represents abstract sequential information using awake monkey functional magnetic resonance imaging (fMRI). We found that area 46 responded to abstract sequence changes, with a preference for more general responses on the right and dynamics similar to humans on the left. These results suggest that abstract sequences are represented in functionally homologous regions across monkeys and humans.

The tie that binds: temporal coding and adaptive emotion

Emotions are temporally dynamic, but the persistence of emotions outside of their appropriate temporal context is detrimental to health and well-being. Yet, precisely how temporal coding and emotional processing interact remains unclear. Recently unveiled temporal context representations in the hippocampus, entorhinal cortex (EC), and prefrontal cortex (PFC) support memory for what happened when. Here, we discuss how these neural temporal representations may interact with densely interconnected amygdala circuitry to shape emotional functioning. We propose a neuroanatomically informed framework suggesting that high-fidelity temporal representations linked to dynamic experiences promote emotion regulation and adaptive emotional memories. Then, we discuss how newly-identified synaptic and molecular features of amygdala-hippocampal projections suggest that intense, amygdala-dependent emotional responses may distort temporal-coding mechanisms. We conclude by identifying key avenues for future research.

What Does the Frontopolar Cortex Contribute to Goal-Directed Cognition and Action?

Understanding the unique functions of different subregions of primate prefrontal cortex has been a longstanding goal in cognitive neuroscience. Yet, the anatomy and function of one of its largest subregions (the frontopolar cortex) remain enigmatic and underspecified. Our Society for Neuroscience minisymposium Primate Frontopolar Cortex: From Circuits to Complex Behaviors will comprise a range of new anatomic and functional approaches that have helped to clarify the basic circuit anatomy of the frontal pole, its functional involvement during performance of cognitively demanding behavioral paradigms in monkeys and humans, and its clinical potential as a target for noninvasive brain stimulation in patients with brain disorders. This review consolidates knowledge about the anatomy and connectivity of frontopolar cortex and provides an integrative summary of its function in primates. We aim to answer the question: what, if anything, does frontopolar cortex contribute to goal-directed cognition and action?

Generalizing the control architecture of the lateral prefrontal cortex

Cognitive control guides non-habitual, goal directed behaviors allowing us to flexibly adapt to ongoing demands. Previous work has suggested that multiple cognitive control processes exist that can be classed according to their action on present-oriented/external information versus future-oriented/internal information. These processes can be mapped onto the lateral prefrontal cortex (LPFC) such that increasingly rostral areas are involved in increasingly future-oriented/internal control processes. Whether and how such processes are organized to support goal-directed behavior remains unclear. On the one hand, the LPFC may flexibly adapt based upon demands. On the other hand, there may be a consistent control architecture such as a control hierarchy that generalizes across demands. Previous work using fMRI in humans during a comprehensive control task that engaged several control processes at once found that an area in mid-LPFC consistently exerted widespread influence throughout the LPFC. These data suggested that the mid-LPFC forms an apex of a putative control hierarchy. However, whether such an architecture generalizes across tasks remains to be tested. Here, we utilized a modified comprehensive control task designed to alter how control processes influence one another to test the generalizability of the LPFC control architecture. Univariate fMRI activations revealed distinct control-related activations relative to past work. Despite these changes, effective connectivity modeling revealed a directed architecture similar to previous findings with the mid-LPFC exerting the most widespread influences throughout LPFC. These results suggest that the fundamental control architecture of the LPFC is relatively fixed, and that different demands are accommodated through modulations of this fixed architecture.

Hierarchical Reinforcement Learning, Sequential Behavior, and the Dorsal Frontostriatal System

To effectively behave within ever-changing environments, biological agents must learn and act at varying hierarchical levels such that a complex task may be broken down into more tractable subtasks. Hierarchical reinforcement learning (HRL) is a computational framework that provides an understanding of this process by combining sequential actions into one temporally extended unit called an option. However, there are still open questions within the HRL framework, including how options are formed and how HRL mechanisms might be realized within the brain. In this review, we propose that the existing human motor sequence literature can aid in understanding both of these questions. We give specific emphasis to visuomotor sequence learning tasks such as the discrete sequence production task and the M × N (M steps × N sets) task to understand how hierarchical learning and behavior manifest across sequential action tasks as well as how the dorsal cortical-subcortical circuitry could support this kind of behavior. This review highlights how motor chunks within a motor sequence can function as HRL options. Furthermore, we aim to merge findings from motor sequence literature with reinforcement learning perspectives to inform experimental design in each respective subfield.

Caught in the ACTS: Defining Abstract Cognitive Task Sequences as an Independent Process

Cognitive neuroscience currently conflates the study of serial responses (e.g., delay match to sample/nonsample, n-back) with the study of sequential operations. In this essay, our goal is to define and disentangle the latter, termed abstract cognitive task sequences (ACTS). Existing literatures address tasks requiring serial events, including procedural learning of implicit motor responses, statistical learning of predictive relationships, and judgments of attributes. These findings do not describe the behavior and underlying mechanism required to succeed at remembering to evaluate color, then shape; or to multiply, then add. A new literature is needed to characterize these sorts of second-order cognitive demands of studying a sequence of operations. Our second goal is to characterize gaps in knowledge related to ACTS that merit further investigation. In the following sections, we define more precisely what we mean by ACTS and suggest research questions' further investigation would be positioned to address.

Reward Value Enhances Sequence Monitoring Ramping Dynamics as Ending Rewards Approach in the Rostrolateral Prefrontal Cortex

Many fundamental human behaviors contain multiple sequences performed to reach a desired outcome, such as cooking. Reward is inherently associated with sequence completion and has been shown to generally enhance cognitive control. However, the impact of reward on cognitive sequence processing remains unexplored. To address this key question, we focused on the rostrolateral prefrontal cortex (RLPFC). This area is necessary and exhibits increasing (“ramping”) activation during sequences, a dynamic that may be related to reward processing in other brain regions. To separate these dynamics, we designed a task where reward was only provided after multiple four-item sequences (“iterations”), rather than each individual sequence. Using fMRI in humans, we investigated three possible interactions of reward and sequential control signals in RLPFC: (1) with the visibility of sequential cues, i.e., memory; (2) equally across individual sequence iterations; and (3) differently across individual sequence iterations (e.g., increasing as reward approaches). Evidence from previous, nonsequential cognitive control experiments suggested that reward would uniformly change RLPFC activity across iterations and may depend on the visibility of cues. However, we found the influence of reward on RLPFC ramping increased across sequence iterations and did not interact with memory. These results suggest an active, predictive, and distinctive role for RLPFC in sequence monitoring and integration of reward information, consistent with extant literature demonstrating similar accelerating reward-related dopamine dynamics in regions connected to RLPFC. These results have implications for understanding sequential behavior in daily life, and when they go awry in disorders such as addiction.

Local and distributed cortical markers of effort expenditure during sustained goal pursuit

The adaptive adjustment of behavior in pursuit of desired goals is critical for survival. To accomplish this complex feat, individuals must weigh the potential benefits of a given action against time, energy, and resource costs. Here, we examine brain responses associated with willingness to exert physical effort during the sustained pursuit of desired goals. Our analyses reveal a distributed pattern of brain activity in aspects of ventral medial prefrontal cortex that tracks with trial-level variability in effort expenditure. Indicating the brain represents echoes of effort at the point of feedback, whole-brain searchlights identified signals reflecting past effort expenditure in medial and lateral prefrontal cortices, encompassing broad swaths of frontoparietal and dorsal attention networks. These data have important implications for our understanding of how the brain's valuation mechanisms contend with the complexity of real-world dynamic environments with relevance for the study of behavior across health and disease.

Neural Representations of Task Context and Temporal Order During Action Sequence Execution

Routine action sequences can share a great deal of similarity in terms of their stimulus response mappings. As a consequence, their correct execution relies crucially on the ability to preserve contextual and temporal information. However, there are few empirical studies on the neural mechanism and the brain areas maintaining such information. To address this gap in the literature, we recently recorded the blood-oxygen level dependent (BOLD) response in a newly developed coffee-tea making task. The task involves the execution of four action sequences that each comprise six consecutive decision states, which allows for examining the maintenance of contextual and temporal information. Here, we report a reanalysis of this dataset using a data-driven approach, namely multivariate pattern analysis, that examines context-dependent neural activity across several predefined regions of interest. Results highlight involvement of the inferior-temporal gyrus and lateral prefrontal cortex in maintaining temporal and contextual information for the execution of hierarchically organized action sequences. Furthermore, temporal information seems to be more strongly encoded in areas over the left hemisphere.

Hierarchical Representation of Multistep Tasks in Multiple-Demand and Default Mode Networks

Task episodes consist of sequences of steps that are performed to achieve a goal. We used fMRI to examine neural representation of task identity, component items, and sequential position, focusing on two major cortical systems—the multiple-demand (MD) and default mode networks (DMN). Human participants (20 males, 22 females) learned six tasks each consisting of four steps. Inside the scanner, participants were cued which task to perform and then sequentially identified the target item of each step in the correct order. Univariate time course analyses indicated that intra-episode progress was tracked by a tonically increasing global response, plus an increasing phasic step response specific to MD regions. Inter-episode boundaries evoked a widespread response at episode onset, plus a marked offset response specific to DMN regions. Representational similarity analysis (RSA) was used to examine representation of task identity and component steps. Both networks represented the content and position of individual steps, however the DMN preferentially represented task identity while the MD network preferentially represented step-level information. Thus, although both MD and DMN networks are sensitive to step-level and episode-level information in the context of hierarchical task performance, they exhibit dissociable profiles in terms of both temporal dynamics and representational content. The results suggest collaboration of multiple brain regions in control of multistep behavior, with MD regions particularly involved in processing the detail of individual steps, and DMN adding representation of broad task context.

SIGNIFICANCE STATEMENT Achieving one's goals requires knowing what to do and when. Tasks are typically hierarchical, with smaller steps nested within overarching goals. For effective, flexible behavior, the brain must represent both levels. We contrast response time courses and information content of two major cortical systems—the multiple-demand (MD) and default mode networks (DMN)—during multistep task episodes. Both networks are sensitive to step-level and episode-level information, but with dissociable profiles. Intra-episode progress is tracked by tonically increasing global responses, plus MD-specific increasing phasic step responses. Inter-episode boundaries evoke widespread responses at episode onset, plus DMN-specific offset responses. Both networks represent content and position of individual steps; however, the DMN and MD networks favor task identity and step-level information, respectively.

The Rostrolateral Prefrontal Cortex Mediates a Preference for High-Agency Environments

The ability to exert flexible instrumental control over one's environment is a defining feature of adaptive decision-making. Here, we investigated neural substrates mediating a preference for environments with greater instrumental divergence, the distance between outcome probability distributions associated with alternative actions. A formal index of agency, instrumental divergence allows an organism to flexibly obtain the currently most desired outcome as preferences change. As such, it may have intrinsic utility, guiding decisions toward environments that maximize instrumental power. Consistent with this notion, we found that a measure of expected value that treats instrumental divergence as a reward surrogate provided a better account of male and female human participants' choice preferences than did a conventional model, sensitive only to monetary reward. Using model-based fMRI, we found that activity in the rostrolateral and ventromedial PFC, regions associated with abstract cognitive inferences and subjective value computations, respectively, scaled with the divergence-based account of expected value. Implications for a neural common currency of information theoretic and motivational variables are discussed.SIGNIFICANCE STATEMENT Agency is a central concept in philosophy and psychology. While research thus far has focused on cognitive and perceptual measures of agency, recent work demonstrating a strong preference for high-agency environments indicates a salient motivational dimension. Here, using instrumental divergence, the distance between outcome distributions associated with alternative actions, as a formal index of agency, we found that brain regions associated with directed exploration and subjective value computations, respectively, were selectively modulated by a model that treated agency as a reward surrogate, over models that assigned utility only to monetary payoffs. In a subset of regions, such effects were predicted by the influence of instrumental divergence on economic choice preferences. Our results elucidate neural mechanisms mediating the utility of agency.

Brain Mechanisms of Concept Learning

Concept learning, the ability to extract commonalities and highlight distinctions across a set of related experiences to build organized knowledge, is a critical aspect of cognition. Previous reviews have focused on concept learning research as a means for dissociating multiple brain systems. The current review surveys recent work that uses novel analytical approaches, including the combination of computational modeling with neural measures, focused on testing theories of specific computations and representations that contribute to concept learning. We discuss in detail the roles of the hippocampus, ventromedial prefrontal, lateral prefrontal, and lateral parietal cortices, and how their engagement is modulated by the coherence of experiences and the current learning goals. We conclude that the interaction of multiple brain systems relating to learning, memory, attention, perception, and reward support a flexible concept-learning mechanism that adapts to a range of category structures and incorporates motivational states, making concept learning a fruitful research domain for understanding the neural dynamics underlying complex behaviors.

Hierarchical cognitive control and the frontal lobes

Cognitive control refers to our ability to choose courses of thought and action that achieve our goals over habitual but contextually inappropriate ones. Hierarchical control problems are those in which multiple goals or contextual contingency must be managed at once and related to one another. In the open-ended complexity of the real world, hierarchical control arguably characterizes most of the problems faced by our control systems. And, it is these cases of hierarchical control where patients with damage to executive systems are most apt to fail, even those that perform well on simplified laboratory tasks. In this chapter, we consider the functional organization of frontal brain systems that support hierarchical cognitive control. We focus on two particular cases of hierarchical control. First, we discuss a line of work testing how managing multiple contingencies en route to a response relates to processing along the rostrocaudal axis of frontal cortex. Second, we consider cases of sequential tasks that require monitoring and behaving according to a series of tasks performed in time. In this latter case, we focus on the particular role of rostrolateral prefrontal cortex. We conclude with considerations of future directions of basic and clinically relevant research in this domain.