Dynamic categorization rules alter representations in human visual cortex

Everyday perceptual tasks require sensory stimuli to be dynamically encoded and analyzed according to changing behavioral goals. For example, when searching for an apple at the supermarket, one might first find the Granny Smith apples by separating all visible apples into the categories "green" and "non-green". However, suddenly remembering that your family actually likes Fuji apples would necessitate reconfiguring the boundary to separate "red" from "red-yellow" objects. This flexible processing enables identical sensory stimuli to elicit varied behaviors based on the current task context. While this phenomenon is ubiquitous in nature, little is known about the neural mechanisms that underlie such flexible computation. Traditionally, sensory regions have been viewed as mainly devoted to processing inputs, with limited involvement in adapting to varying task contexts. However, from the standpoint of efficient computation, it is plausible that sensory regions integrate inputs with current task goals, facilitating more effective information relay to higher-level cortical areas. Here we test this possibility by asking human participants to visually categorize novel shape stimuli based on different linear and non-linear boundaries. Using fMRI and multivariate analyses of retinotopically-defined visual areas, we found that shape representations in visual cortex became more distinct across relevant decision boundaries in a context-dependent manner, with the largest changes in discriminability observed for stimuli near the decision boundary. Importantly, these context-driven modulations were associated with improved categorization performance. Together, these findings demonstrate that codes in visual cortex are adaptively modulated to optimize object separability based on currently relevant decision boundaries.

Proactive selective attention across competition contexts

Selective attention is a cognitive function that helps filter out unwanted information. Theories such as the biased competition model (Desimone & Duncan, 1995) explain how attentional templates bias processing towards targets in contexts where multiple stimuli compete for resources. However, it is unclear how the anticipation of different levels of competition influences the nature of attentional templates, in a proactive fashion. In this study, we used electroencephalography (EEG) to investigate how the anticipated demands of attentional selection (either high or low stimuli competition contexts) modulate target-specific preparatory brain activity and its relationship with task performance. To do so, participants performed a sex/gender judgment task in a cue-target paradigm where, depending on the block, target and distractor stimuli appeared simultaneously (high competition) or sequentially (low competition). Multivariate Pattern Analysis (MVPA) showed that, in both competition contexts, there was a preactivation of the target category to select, with a ramping-up profile at the end of the preparatory interval. However, cross-classification showed no generalization across competition conditions, suggesting different preparatory formats. Notably, time-frequency analyses showed differences between anticipated competition demands, with higher theta band power for high than low competition, which mediated the impact of subsequent stimuli competition on behavioral performance. Overall, our results show that, whereas preactivation of the internal templates associated with the category to select are engaged in advance in high and low competition contexts, their underlying neural patterns differ. In addition, these codes could not be associated with theta power, suggesting that they reflect different preparatory processes. The implications of these findings are crucial to increase our understanding of the nature of top-down processes across different contexts.

Attention switching through text dissimilarity: a cognition research on fragmented reading behavior

People tend to obtain information through fragmented reading. However, this behavior itself might lead to distraction and affect cognitive ability. To address it, it is necessary to understand how fragmented reading behavior influences readers' attention switching. In this study, the researchers first collected online news that had 6 theme words and 60 sentences to compose the experimental material, then defined the degree of text dissimilarity, used to measure the degree of attention switching based on the differences in text content, and conducted an EEG experiment based on P200. The results showed that even after reading the fragmented text content with the same overall content, people in subsequent cognitive tasks had more working memory capacity, lower working memory load, and less negative impact on cognitive ability with the text content with lower text dissimilarity. Additionally, attention switching caused by differences in concept or working memory representation of text content might be the key factor affecting cognitive ability in fragmented reading behavior. The findings disclosed the relation between cognitive ability and fragmented reading and attention switching, opening a new perspective on the method of text dissimilarity. This study provides some references on how to reduce the negative impact of fragmented reading on cognitive ability on new media platforms.

Orthogonal neural encoding of targets and distractors supports multivariate cognitive control

The complex challenges of our mental life require us to coordinate multiple forms of neural information processing. Recent behavioural studies have found that people can coordinate multiple forms of attention, but the underlying neural control process remains obscure. We hypothesized that the brain implements multivariate control by independently monitoring feature-specific difficulty and independently prioritizing feature-specific processing. During functional MRI, participants performed a parametric conflict task that separately tags target and distractor processing. Consistent with feature-specific monitoring, univariate analyses revealed spatially segregated encoding of target and distractor difficulty in the dorsal anterior cingulate cortex. Consistent with feature-specific attentional priority, our encoding geometry analysis revealed overlapping but orthogonal representations of target and distractor coherence in the intraparietal sulcus. Coherence representations were mediated by control demands and aligned with both performance and frontoparietal activity, consistent with top-down attention. Together, these findings provide evidence for the neural geometry necessary to coordinate multivariate cognitive control.

Domain general frontoparietal regions show modality-dependent coding of auditory and visual rules

A defining feature of human cognition is our ability to respond flexibly to what we see and hear, changing how we respond depending on our current goals. In fact, we can rapidly associate almost any input stimulus with any arbitrary behavioural response. This remarkable ability is thought to depend on a frontoparietal "multiple demand" circuit which is engaged by many types of cognitive demand and widely referred to as domain general. However, it is not clear how responses to multiple input modalities are structured within this system. Domain generality could be achieved by holding information in an abstract form that generalises over input modality, or in a modality-tagged form, which uses similar resources but produces unique codes to represent the information in each modality. We used a stimulus-response task, with conceptually identical rules in two sensory modalities (visual and auditory), to distinguish between these possibilities. Multivariate decoding of functional magnetic resonance imaging data showed that representations of visual and auditory rules recruited overlapping neural resources but were expressed in modality-tagged non-generalisable neural codes. Our data suggest that this frontoparietal system may draw on the same or similar resources to solve multiple tasks, but does not create modality-general representations of task rules, even when those rules are conceptually identical between domains.

Attention with or without working memory: mnemonic reselection of attended information

Attention has been regarded as the 'gatekeeper' controlling what information gets selected into working memory. However, a new perspective has emerged with the discovery of attribute amnesia, a phenomenon revealing that people are frequently unable to report information they have just attended to moments ago. This report failure is thought to stem from a lack of consolidating the attended information into working memory, indicating a dissociation between attention and working memory. Building on these findings, a new concept called memory reselection is proposed to describe a secondary round of selection among the attended information. These discoveries challenge the conventional view of how attention and working memory are related and shed new light onto modeling attention and memory as dissociable processes.

Decoding neural patterns for the processing of fearful faces under different visual awareness conditions: A multivariate pattern analysis

Previous studies have provided mixed findings regarding the nonconscious processing of fearful faces. Here, we used multivariate pattern analysis on electroencephalography data from three backward masking experiments to examine the processing of fearful faces under different visual awareness conditions. Three groups of participants were shown pairs of face images presented very briefly (for 16 ms) or for sufficiently long (for 266 ms), and completed tasks where the faces were either relevant to the experimental task (Experiment 1) or not (Experiments 2 and 3). Three main decoding analyses were performed. First, in the visual awareness decoding, the visibility of the faces, and hence participants' awareness of them, was maximally decodable in three time windows: 158-168 ms, 235-260 ms and 400-600 ms where the earlier neural patterns were generalized to the later stage activity. Second, we found that the spatial location of a fearful face in the face pairs was decodable, however only when the faces were consciously seen and task-relevant. Finally, we successfully decoded distinct neural patterns associated with the fearful-face-present conditions, compared to the fearful-face-absent conditions, and these patterns were decodable during both short and long presentations of the faces. Together, our results suggest that, while the processing of the spatial location of fearful faces requires awareness and task-relevancy, the mere presence of fearful faces can be processed even when visual awareness is highly restricted.

Serial dependence in facial identity perception and visual working memory

Serial dependence (SD) refers to the effect in which a person's current perceptual judgment is attracted toward recent stimulus history. Perceptual and memory processes, as well as response and decisional biases, are thought to contribute to SD effects. The current study examined the processing stages of SD facial identity effects in the context of task-related decision processes and how such effects may differ from visual working memory (VWM) interactions. In two experiments, participants were shown a series of two sequentially presented face images. In Experiment 1, the two faces were separated by an interstimulus interval (ISI) of 1, 3, 6, or 10 s, and participants were instructed to reproduce the second face after a varying response delay of 0, 1, 3, 6, or 10 s. Results showed that SD effects occurred most consistently at ISI of 1 s and response delays of 1 and 6 s consistent with early and late stages of processing. In Experiment 2, the ISI was held constant at 1 s, and to separate SD from VWM interactions participants were post-cued to reproduce either the first or the second face. When the second face was the target, SD effects again occurred at response delays of 1 and 6 s, but not when the first face was the target. Together, the results demonstrates that SD facial identity effects occur independently of task-related processes in a distinct temporal fashion and suggest that SD and VWM interactions may rely on separate underlying mechanisms.

Adaptive coding of stimulus information in human frontoparietal cortex during visual classification

The neural mechanisms of how frontal and parietal brain regions support flexible adaptation of behavior remain poorly understood. Here, we used functional magnetic resonance imaging (fMRI) and representational similarity analysis (RSA) to investigate frontoparietal representations of stimulus information during visual classification under varying task demands. Based on prior research, we predicted that increasing perceptual task difficulty should lead to adaptive changes in stimulus coding: task-relevant category information should be stronger, while task-irrelevant exemplar-level stimulus information should become weaker, reflecting a focus on the behaviorally relevant category information. Counter to our expectations, however, we found no evidence for adaptive changes in category coding. We did find weakened coding at the exemplar-level within categories however, demonstrating that task-irrelevant information is de-emphasized in frontoparietal cortex. These findings reveal adaptive coding of stimulus information at the exemplar-level, highlighting how frontoparietal regions might support behavior even under challenging conditions.

Context-independent scaling of neural responses to task difficulty in the multiple-demand network

The multiple-demand (MD) network is sensitive to many aspects of cognitive demand, showing increased activation with more difficult tasks. However, it is currently unknown whether the MD network is modulated by the context in which task difficulty is experienced. Using functional magnetic resonance imaging, we examined MD network responses to low, medium, and high difficulty arithmetic problems within 2 cued contexts, an easy versus a hard set. The results showed that MD activity varied reliably with the absolute difficulty of a problem, independent of the context in which the problem was presented. Similarly, MD activity during task execution was independent of the difficulty of the previous trial. Representational similarity analysis further supported that representational distances in the MD network were consistent with a context-independent code. Finally, we identified several regions outside the MD network that showed context-dependent coding, including the inferior parietal lobule, paracentral lobule, posterior insula, and large areas of the visual cortex. In sum, a cognitive effort is processed by the MD network in a context-independent manner. We suggest that this absolute coding of cognitive demand in the MD network reflects the limited range of task difficulty that can be supported by the cognitive apparatus.

Integrated neural dynamics for behavioural decisions and attentional competition in the prefrontal cortex

In the behaving monkey, complex neural dynamics in the prefrontal cortex contribute to context-dependent decisions and attentional competition. We used demixed principal component analysis to track prefrontal activity dynamics in a cued target detection task. In this task, the animal combined identity of a visual object with a prior instruction cue to determine a target/nontarget decision. From population activity, we extracted principal components for each task feature and examined their time course and sensitivity to stimulus and task variations. For displays containing a single choice object in left or right hemifield, object identity, cue identity and decision were all encoded in population activity, with different dynamics and lateralisation. Object information peaked at 100-200 ms from display onset and was largely confined to the contralateral hemisphere. Cue information was weaker and present even prior to display onset. Integrating information from cue and object, decision information arose more slowly and was bilateral. Individual neurons contributed independently to coding of the three task features. The analysis was then extended to displays with a target in one hemifield and a competing distractor in the other. In this case, the data suggest that each hemisphere initially encoded the identity of the contralateral object. The distractor representation was then rapidly suppressed, with the final target decision again encoded bilaterally. The results show how information is coded along task-related dimensions while competition is resolved and suggest how information flows within and across frontal lobes to implement a learned behavioural decision.

Training modulates memory-driven capture

Attention is captured by information matching the contents of working memory. Though many factors modulate the amount of capture, there is surprising resistance to cognitive control. Capture occurs even when participants are instructed either that an item would never be a target or to drop that item from memory. Does the persistence of capture under these conditions reflect a rigidity in capture, or can properly motivated participants learn to completely suppress distractors and/or completely drop items from memory? Surprisingly, no studies have looked at the influence of extensive training of involuntary capture from working memory items. Here, we addressed whether training leads to a reduction or even elimination of memory-driven capture. After memorizing a single object, participants were cued to remember or to forget this object. Subsequently, they were asked to execute a search task. To measure capture, we compared search performances in displays that did and did not contain a distractor matching the earlier memorized object. Participants completed multiple experimental sessions over four days. The results showed that attentional capture by to-be-remembered distractors was reduced, but not eliminated in subsequent sessions compared with the first session. Training did not impact capture by to-be-forgotten objects. The results suggest observable, but limited, cognitive control over memory-driven capture.

Neural Coding of Visual Objects Rapidly Reconfigures to Reflect Subtrial Shifts in Attentional Focus

Every day, we respond to the dynamic world around us by choosing actions to meet our goals. Flexible neural populations are thought to support this process by adapting to prioritize task-relevant information, driving coding in specialized brain regions toward stimuli and actions that are currently most important. Accordingly, human fMRI shows that activity patterns in frontoparietal cortex contain more information about visual features when they are task-relevant. However, if this preferential coding drives momentary focus, for example, to solve each part of a task in turn, it must reconfigure more quickly than we can observe with fMRI. Here, we used multivariate pattern analysis of magnetoencephalography data to test for rapid reconfiguration of stimulus information when a new feature becomes relevant within a trial. Participants saw two displays on each trial. They attended to the shape of a first target then the color of a second, or vice versa, and reported the attended features at a choice display. We found evidence of preferential coding for the relevant features in both trial phases, even as participants shifted attention mid-trial, commensurate with fast subtrial reconfiguration. However, we only found this pattern of results when the stimulus displays contained multiple objects and not in a simpler task with the same structure. The data suggest that adaptive coding in humans can operate on a fast, subtrial timescale, suitable for supporting periods of momentary focus when complex tasks are broken down into simpler ones, but may not always do so.

Linking the Brain with Behavior: The Neural Dynamics of Success and Failure in Goal-directed Behavior

The human brain is extremely flexible and capable of rapidly selecting relevant information in accordance with task goals. Regions of frontoparietal cortex flexibly represent relevant task information such as task rules and stimulus features when participants perform tasks successfully, but less is known about how information processing breaks down when participants make mistakes. This is important for understanding whether and when information coding recorded with neuroimaging is directly meaningful for behavior. Here, we used magnetoencephalography to assess the temporal dynamics of information processing and linked neural responses with goal-directed behavior by analyzing how they changed on behavioral error. Participants performed a difficult stimulus–response task using two stimulus–response mapping rules. We used time-resolved multivariate pattern analysis to characterize the progression of information coding from perceptual information about the stimulus, cue and rule coding, and finally, motor response. Response-aligned analyses revealed a ramping up of perceptual information before a correct response, suggestive of internal evidence accumulation. Strikingly, when participants made a stimulus-related error, and not when they made other types of errors, patterns of activity initially reflected the stimulus presented, but later reversed, and accumulated toward a representation of the “incorrect” stimulus. This suggests that the patterns recorded at later time points reflect an internally generated stimulus representation that was used to make the (incorrect) decision. These results illustrate the orderly and overlapping temporal dynamics of information coding in perceptual decision-making and show a clear link between neural patterns in the late stages of processing and behavior.

The Effect of Task on Object Processing revealed by EEG decoding

Recent studies showed that task demand affects object representations in higher-level visual areas and beyond, but not so much in earlier areas. There are, however, limitations in those studies including the relatively weak manipulation of task due to the use of familiar real-life objects, the low temporal resolution in fMRI, and the emphasis on the amount and not the source of information carried by brain activations. In the current study, observers categorized images of artificial objects in one of two orthogonal dimensions, shape and texture, while their brain activity was recorded with electroencephalogram (EEG). Results showed that object processing along the texture dimension was affected by task demand starting from a relatively late time (320-370ms time window) after image onset. The findings are consistent with the view that task exerts an effect on the later phases of object processing.

More attention with less working memory: The active inhibition of attended but outdated information

Attention has traditionally been regarded as a gateway to working memory, and almost all theoretical frameworks of attention and working memory assume that individuals always have a better memory for information that has received more attention. Here, we provide a series of counterintuitive demonstrations that show that paying more attention to a piece of information impedes, rather than enhances, the selection of this information into working memory. Experiments 1 to 5 provide converging evidence for an even weaker working memory trace of fully attended but outdated features, compared with baseline irrelevant features that were completely ignored. This indicates that the brain actively inhibits attended but outdated information to prevent it from entering working memory. Experiment 6 demonstrates that this inhibition processing is subject to executive control. These findings lead to a substantial reinterpretation of the relationship between attention and working memory.

Spatial and Feature-selective Attention Have Distinct, Interacting Effects on Population-level Tuning

Attention can be deployed in different ways: When searching for a taxi in New York City, we can decide where to attend (e.g., to the street) and what to attend to (e.g., yellow cars). Although we use the same word to describe both processes, nonhuman primate data suggest that these produce distinct effects on neural tuning. This has been challenging to assess in humans, but here we used an opportunity afforded by multivariate decoding of MEG data. We found that attending to an object at a particular location and attending to a particular object feature produced effects that interacted multiplicatively. The two types of attention induced distinct patterns of enhancement in occipital cortex, with feature-selective attention producing relatively more enhancement of small feature differences and spatial attention producing relatively larger effects for larger feature differences. An information flow analysis further showed that stimulus representations in occipital cortex were Granger-caused by coding in frontal cortices earlier in time and that the timing of this feedback matched the onset of attention effects. The data suggest that spatial and feature-selective attention rely on distinct neural mechanisms that arise from frontal-occipital information exchange, interacting multiplicatively to selectively enhance task-relevant information.

The Synchronized Enhancement Effect of Rhythmic Visual Stimulation of 40 Hz on Selective Attention

Rhythmic visual stimulation (RVS) has been demonstrated to modulate ongoing neuronal oscillations which might be greatly involved in attention processes and thus bring some behavioral consequences. However, there was little knowledge about the effective frequency parameter of RVS which could impact task performance in visuo-spatial selective attention. Thus, here, we addressed this question by investigating the modulating effects of RVSs in different attention-related frequency bands, i.e., alpha (10 Hz) and gamma band (40 Hz). Sixteen participants were recruited to perform a modified visuo-spatial selective attention task. They were required to identify the orientation of target-triangle in visual search arrays while undergoing different RVS backgrounds. By analyzing the acquired behavioral and EEG data, we observed that, compared with control group (no RVS), 40 Hz RVS led to significantly shorter reaction time (RT) while 10 Hz RVS did not bring obvious behavioral consequences. In addition, although both 10 and 40 Hz RVS led to a global enhancement of SSVEP spectrum in the gamma band, 40 Hz RVS led to even larger 40 Hz SSVEP spectrum in prefrontal cortex. Our findings indicate that 40 Hz RVS has an effectively enhancing effect on selective attention and support the crucial role of prefrontal area in selective attention.

Reward motivation increases univariate activity but has limited effect on coding of task-relevant information across the frontoparietal cortex

Selection and integration of information based on current goals is fundamental for goal-directed behavior. Reward motivation has been shown to improve behavioral performance, yet the neural mechanisms that link motivation and control processes, and in particular its effect on context-dependent information processing, remain unclear. We used functional magnetic resonance imaging (fMRI) in 24 human volunteers (13 females) to test whether reward motivation enhances the coding of task-relevant information across the frontoparietal cortex, as would be predicted based on previous experimental evidence and theoretical accounts. In a cued target detection task, participants detected whether an object from a cued visual category was present in a subsequent display. The combination of the cue and the object visual category determined the behavioral status of the objects. To manipulate reward motivation, half of all trials offered the possibility of a monetary reward. We observed an increase with reward in overall univariate activity across the frontoparietal control network when the cue and subsequent object were presented. Multivariate pattern analysis (MVPA) showed that behavioral status information for the objects was conveyed across the network. However, in contrast to our prediction, reward did not increase the discrimination between behavioral status conditions in the stimulus epoch of a trial when object information was processed depending on a current context. In the high-level general-object visual region, the lateral occipital complex, the representation of behavioral status was driven by visual differences and was not modulated by reward. Our study provides useful evidence for the limited effects of reward motivation on task-related neural representations and highlights the necessity to unravel the diverse forms and extent of these effects.

•

Reward motivation leads to increased activity in the frontoparietal control network.

•

Task-relevant information is coded across the control network.

•

Reward does not increase coding of behavioral relevance of task information.

•

Visual category coding in the general-object region does not increase with reward.

Concurrent neuroimaging and neurostimulation reveals a causal role for dlPFC in coding of task-relevant information

Dorsolateral prefrontal cortex (dlPFC) is proposed to drive brain-wide focus by biasing processing in favour of task-relevant information. A longstanding debate concerns whether this is achieved through enhancing processing of relevant information and/or by inhibiting irrelevant information. To address this, we applied transcranial magnetic stimulation (TMS) during fMRI, and tested for causal changes in information coding. Participants attended to one feature, whilst ignoring another feature, of a visual object. If dlPFC is necessary for facilitation, disruptive TMS should decrease coding of attended features. Conversely, if dlPFC is crucial for inhibition, TMS should increase coding of ignored features. Here, we show that TMS decreases coding of relevant information across frontoparietal cortex, and the impact is significantly stronger than any effect on irrelevant information, which is not statistically detectable. This provides causal evidence for a specific role of dlPFC in enhancing task-relevant representations and demonstrates the cognitive-neural insights possible with concurrent TMS-fMRI-MVPA.

Intraoperative mapping of executive function using electrocorticography for patients with low-grade gliomas

Background

Intraoperative functional mapping with direct electrical stimulation during awake surgery for patients with diffuse low-grade glioma has been used in recent years to optimize the balance between surgical resection and quality of life following surgery. Mapping of executive functions is particularly challenging because of their complex nature, with only a handful of reports published so far. Here, we propose the recording of neural activity directly from the surface of the brain using electrocorticography to map executive functions and demonstrate its feasibility and potential utility.

Methods

To track a neural signature of executive function, we recorded neural activity using electrocorticography during awake surgery from the frontal cortex of three patients judged to have an appearance of diffuse low-grade glioma. Based on existing functional magnetic resonance imaging (fMRI) evidence from healthy participants for the recruitment of areas associated with executive function with increased task demands, we employed a task difficulty manipulation in two counting tasks performed intraoperatively. Following surgery, the data were extracted and analyzed offline to identify increases in broadband high-gamma power with increased task difficulty, equivalent to fMRI findings, as a signature of activity related to executive function.

Results

All three patients performed the tasks well. Data were recorded from five electrode strips, resulting in data from 15 channels overall. Eleven out of the 15 channels (73.3%) showed significant increases in high-gamma power with increased task difficulty, 26.6% of the channels (4/15) showed no change in power, and none of the channels showed power decrease. High-gamma power increases with increased task difficulty were more likely in areas that are within the canonical frontoparietal network template.

Conclusions

These results are the first step toward developing electrocorticography as a tool for mapping of executive function complementarily to direct electrical stimulation to guide resection. Further studies are required to establish this approach for clinical use.

Consciousness can overflow report: Novel evidence from attribute amnesia of a single stimulus

There is a long-standing debate on whether visual consciousness is confined to cognitive access measured by reportability, or whether it is rich and overflows reportability. Much of the debate in previous studies concentrated on whether information outside attentional focus could be consciously experienced and reportable. This study sought to address the debate from a new perspective, through testing whether fully attended supraliminal information is necessarily reportable with a variation of attribute amnesia. Participants were asked to judge the parity of a single number or whether a Chinese character referred to furniture. After several trials, they were unexpectedly asked to report the stimulus identity. The results consistently showed that participants could not correctly report the identity, indicating that fully attended information that was consciously perceived could sometimes overflow report. In addition to providing novel overflow evidence, these findings also have crucial implications in understanding the relationship between consciousness and working memory.

Abstract Neural Representations of Category Membership beyond Information Coding Stimulus or Response

For decades, researchers have debated whether mental representations are symbolic or grounded in sensory inputs and motor programs. Certainly, aspects of mental representations are grounded. However, does the brain also contain abstract concept representations that mediate between perception and action in a flexible manner not tied to the details of sensory inputs and motor programs? Such conceptual pointers would be useful when concepts remain constant despite changes in appearance and associated actions. We evaluated whether human participants acquire such representations using fMRI. Participants completed a probabilistic concept learning task in which sensory, motor, and category variables were not perfectly coupled or entirely independent, making it possible to observe evidence for abstract representations or purely grounded representations. To assess how the learned concept structure is represented in the brain, we examined brain regions implicated in flexible cognition (e.g., pFC and parietal cortex) that are most likely to encode an abstract representation removed from sensory-motor details. We also examined sensory-motor regions that might encode grounded sensory-motor-based representations tuned for categorization. Using a cognitive model to estimate participants' category rule and multivariate pattern analysis of fMRI data, we found the left pFC and MT coded for category in the absence of information coding for stimulus or response. Because category was based on the stimulus, finding an abstract representation of category was not inevitable. Our results suggest that certain brain areas support categorization behavior by constructing concept representations in a format akin to a symbol that differs from stimulus-motor codes.

Memory-driven capture occurs for individual features of an object

Items held in working memory (WM) capture attention (memory-driven capture). People can selectively prioritize specific object features in WM. Here, we examined whether feature-specific prioritization within WM modulates memory-driven capture. In Experiment 1, after remembering the color and orientation of a triangle, participants were instructed, via retro-cue, whether the color, the orientation, or both features were relevant. To measure capture, we asked participants to execute a subsequent search task, and we compared performance in displays that did and did not contain the memory-matching feature. Color attracted attention only when it was relevant. No capture by orientation was found. In Experiment 2, we presented the retro-cue at one of the four locations of the search display to direct attention to specific objects. We found capture by color and this capture was larger when it was indicated as relevant. Crucially, orientation also attracted attention, but only when it was relevant. These findings provide evidence for reciprocal interaction between internal prioritization and external attention on the features level. Specifically, internal feature-specific prioritization modulates memory-driven capture but this capture also depends on the salience of the features.

Integrated Intelligence from Distributed Brain Activity

How does organized cognition arise from distributed brain activity? Recent analyses of fluid intelligence suggest a core process of cognitive focus and integration, organizing the components of a cognitive operation into the required computational structure. A cortical 'multiple-demand' (MD) system is closely linked to fluid intelligence, and recent imaging data define nine specific MD patches distributed across frontal, parietal, and occipitotemporal cortex. Wide cortical distribution, relative functional specialization, and strong connectivity suggest a basis for cognitive integration, matching electrophysiological evidence for binding of cognitive operations to their contents. Though still only in broad outline, these data suggest how distributed brain activity can build complex, organized cognition.

The Domain-General Multiple Demand (MD) Network Does Not Support Core Aspects of Language Comprehension: A Large-Scale fMRI Investigation

Aside from the language-selective left-lateralized frontotemporal network, language comprehension sometimes recruits a domain-general bilateral frontoparietal network implicated in executive functions: the multiple demand (MD) network. However, the nature of the MD network's contributions to language comprehension remains debated. To illuminate the role of this network in language processing in humans, we conducted a large-scale fMRI investigation using data from 30 diverse word and sentence comprehension experiments (481 unique participants [female and male], 678 scanning sessions). In line with prior findings, the MD network was active during many language tasks. Moreover, similar to the language-selective network, which is robustly lateralized to the left hemisphere, these responses were stronger in the left-hemisphere MD regions. However, in contrast with the language-selective network, the MD network responded more strongly (1) to lists of unconnected words than to sentences, and (2) in paradigms with an explicit task compared with passive comprehension paradigms. Indeed, many passive comprehension tasks failed to elicit a response above the fixation baseline in the MD network, in contrast to strong responses in the language-selective network. Together, these results argue against a role for the MD network in core aspects of sentence comprehension, such as inhibiting irrelevant meanings or parses, keeping intermediate representations active in working memory, or predicting upcoming words or structures. These results align with recent evidence of relatively poor tracking of the linguistic signal by the MD regions during naturalistic comprehension, and instead suggest that the MD network's engagement during language processing reflects effort associated with extraneous task demands.SIGNIFICANCE STATEMENT Domain-general executive processes, such as working memory and cognitive control, have long been implicated in language comprehension, including in neuroimaging studies that have reported activation in domain-general multiple demand (MD) regions for linguistic manipulations. However, much prior evidence has come from paradigms where language interpretation is accompanied by extraneous tasks. Using a large fMRI dataset (30 experiments/481 participants/678 sessions), we demonstrate that MD regions are engaged during language comprehension in the presence of task demands, but not during passive reading/listening, conditions that strongly activate the frontotemporal language network. These results present a fundamental challenge to proposals whereby linguistic computations, such as inhibiting irrelevant meanings, keeping representations active in working memory, or predicting upcoming elements, draw on domain-general executive resources.

Not-so-working Memory: Drift in Functional Magnetic Resonance Imaging Pattern Representations during Maintenance Predicts Errors in a Visual Working Memory Task

Working memory (WM) is critical to many aspects of cognition, but it frequently fails. Much WM research has focused on capacity limits, but even for single, simple features, the fidelity of individual representations is limited. Why is this? One possibility is that, because of neural noise and interference, neural representations do not remain stable across a WM delay, nor do they simply decay, but instead, they may "drift" over time to a new, less accurate state. We tested this hypothesis in a functional magnetic resonance imaging study of a match/nonmatch WM recognition task for a single item with a single critical feature: orientation. We developed a novel pattern-based index of "representational drift" to characterize ongoing changes in brain activity patterns throughout the WM maintenance period, and we were successfully able to predict performance on the match/nonmatch recognition task using this representational drift index. Specifically, in trials where the target and probe stimuli matched, participants incorrectly reported more nonmatches when their activity patterns drifted away from the target. In trials where the target and probe did not match, participants incorrectly reported more matches when their activity patterns drifted toward the probe. On the basis of these results, we contend that neural noise does not cause WM errors merely by degrading representations and increasing random guessing; instead, one means by which noise introduces errors is by pushing WM representations away from the target and toward other meaningful (yet incorrect) configurations. Thus, we demonstrate that behaviorally meaningful drift within representation space can be indexed by neuroimaging.

Human frontoparietal cortex represents behaviorally relevant target status based on abstract object features

Searching for items that are useful given current goals, or "target" recognition, requires observers to flexibly attend to certain object properties at the expense of others. This could involve focusing on the identity of an object while ignoring identity-preserving transformations such as changes in viewpoint or focusing on its current viewpoint while ignoring its identity. To effectively filter out variation due to the irrelevant dimension, performing either type of task is likely to require high-level, abstract search templates. Past work has found target recognition signals in areas of ventral visual cortex and in subregions of parietal and frontal cortex. However, target status in these tasks is typically associated with the identity of an object, rather than identity-orthogonal properties such as object viewpoint. In this study, we used a task that required subjects to identify novel object stimuli as targets according to either identity or viewpoint, each of which was not predictable from low-level properties such as shape. We performed functional MRI in human subjects of both sexes and measured the strength of target-match signals in areas of visual, parietal, and frontal cortex. Our multivariate analyses suggest that the multiple-demand (MD) network, including subregions of parietal and frontal cortex, encodes information about an object's status as a target in the relevant dimension only, across changes in the irrelevant dimension. Furthermore, there was more target-related information in MD regions on correct compared with incorrect trials, suggesting a strong link between MD target signals and behavior. NEW & NOTEWORTHY Real-world target detection tasks, such as searching for a car in a crowded parking lot, require both flexibility and abstraction. We investigated the neural basis of these abilities using a task that required invariant representations of either object identity or viewpoint. Multivariate decoding analyses of our whole brain functional MRI data reveal that invariant target representations are most pronounced in frontal and parietal regions, and the strength of these representations is associated with behavioral performance.

Adaptive coding in the human brain: Distinct object features are encoded by overlapping voxels in frontoparietal cortex

Our ability to flexibly switch between different tasks is a key component of cognitive control. Non-human primate (NHP) studies (e.g., Freedman, Riesenhuber, Poggio, & Miller, 2001) have shown that prefrontal neurons are re-used across tasks, re-configuring their responses to code currently relevant information. In a similar vein, in the human brain, the "multiple demand" (MD) system is suggested to exert control by adjusting its responses, selectively processing information in line with our current goals (Duncan, 2010). However, whether the same or different resources (underlying neural populations) in the human brain are recruited to solve different tasks remains elusive. In the present study, we aimed to bridge the gap between the NHP and human literature by examining human functional imaging data at an intermediate level of resolution: quantifying the extent to which single voxels contributed to multiple neural codes. Participants alternated between two tasks requiring the selection of feature information from two distinct sets of objects. We examined whether neural codes for the relevant stimulus features in the two different tasks depended on the same or different voxels. In line with the electrophysiological literature, MD voxels were more likely to contribute to multiple neural codes than we predicted based on permutation tests. Comparatively, in the visual system the neural codes depended on distinct sets of voxels. Our data emphasise the flexibility of the MD regions to re-configure their responses and adaptively code relevant information across different tasks.

Attentional Fluctuations Influence the Neural Fidelity and Connectivity of Stimulus Representations

Attention is thought to facilitate both the representation of task-relevant features and the communication of these representations across large-scale brain networks. However, attention is not "all or none," but rather it fluctuates between stable/accurate (in-the-zone) and variable/error-prone (out-of-the-zone) states. Here we ask how different attentional states relate to the neural processing and transmission of task-relevant information. Specifically, during in-the-zone periods: (1) Do neural representations of task stimuli have greater fidelity? (2) Is there increased communication of this stimulus information across large-scale brain networks? Finally, (3) can the influence of performance-contingent reward be differentiated from zone-based fluctuations? To address these questions, we used fMRI and representational similarity analysis during a visual sustained attention task (the gradCPT). Participants ( n = 16) viewed a series of city or mountain scenes, responding to cities (90% of trials) and withholding to mountains (10%). Representational similarity matrices, reflecting the similarity structure of the city exemplars ( n = 10), were computed from visual, attentional, and default mode networks. Representational fidelity (RF) and representational connectivity (RC) were quantified as the interparticipant reliability of representational similarity matrices within (RF) and across (RC) brain networks. We found that being in the zone was characterized by increased RF in visual networks and increasing RC between visual and attentional networks. Conversely, reward only increased the RC between the attentional and default mode networks. These results diverge with analogous analyses using functional connectivity, suggesting that RC and functional connectivity in tandem better characterize how different mental states modulate the flow of information throughout the brain.

Attention Effects on Neural Population Representations for Shape and Location Are Stronger in the Ventral than Dorsal Stream

We examined how attention causes neural population representations of shape and location to change in ventral stream (AIT) and dorsal stream (LIP). Monkeys performed two identical delayed-match-to-sample (DMTS) tasks, attending either to shape or location. In AIT, shapes were more discriminable when directing attention to shape rather than location, measured by an increase in mean distance between population response vectors. In LIP, attending to location rather than shape did not increase the discriminability of different stimulus locations. Even when factoring out the change in mean vector response distance, multidimensional scaling (MDS) still showed a significant task difference in AIT, but not LIP, indicating that beyond increasing discriminability, attention also causes a nonlinear warping of representation space in AIT. Despite single-cell attentional modulations in both areas, our data show that attentional modulations of population representations are weaker in LIP, likely due to a need to maintain veridical representations for visuomotor control.

Multisensory coding in the multiple-demand regions: vibrotactile task information is coded in frontoparietal cortex

At any given moment, our brains receive input from multiple senses. Successful behavior depends on our ability to prioritize the most important information and ignore the rest. A multiple-demand (MD) network of frontal and parietal regions is thought to support this process by adjusting to code information that is currently relevant (Duncan 2010). Accordingly, the network is proposed to encode a range of different types of information, including perceptual stimuli, task rules, and responses, as needed for the current cognitive operation. However, most MD research has used visual tasks, leaving limited information about whether these regions encode other sensory domains. We used multivoxel pattern analysis (MVPA) of functional magnetic resonance imaging (fMRI) data to test whether the MD regions code the details of somatosensory stimuli, in addition to tactile-motor response transformation rules and button-press responses. Participants performed a stimulus-response task in which they discriminated between two possible vibrotactile frequencies and applied a stimulus-response transformation rule to generate a button-press response. For MD regions, we found significant coding of tactile stimulus, rule, and response. Primary and secondary somatosensory regions encoded the tactile stimuli and the button-press responses but did not represent task rules. Our findings provide evidence that MD regions can code nonvisual somatosensory task information, commensurate with a domain-general role in cognitive control.NEW & NOTEWORTHY How does the brain encode the breadth of information from our senses and use this to produce goal-directed behavior? A network of frontoparietal multiple-demand (MD) regions is implicated but has been studied almost exclusively in the context of visual tasks. We used multivariate pattern analysis of fMRI data to show that these regions encode tactile stimulus information, rules, and responses. This provides evidence for a domain-general role of the MD network in cognitive control.

Vibrotactile Discrimination Training Affects Brain Connectivity in Profoundly Deaf Individuals

Early auditory deprivation has serious neurodevelopmental and cognitive repercussions largely derived from impoverished and delayed language acquisition. These conditions may be associated with early changes in brain connectivity. Vibrotactile stimulation is a sensory substitution method that allows perception and discrimination of sound, and even speech. To clarify the efficacy of this approach, a vibrotactile oddball task with 700 and 900 Hz pure-tones as stimuli [counterbalanced as target (T: 20% of the total) and non-target (NT: 80%)] with simultaneous EEG recording was performed by 14 profoundly deaf and 14 normal-hearing (NH) subjects, before and after a short training period (five 1-h sessions; in 2.5–3 weeks). A small device worn on the right index finger delivered sound-wave stimuli. The training included discrimination of pure tone frequency and duration, and more complex natural sounds. A significant P300 amplitude increase and behavioral improvement was observed in both deaf and normal subjects, with no between group differences. However, a P3 with larger scalp distribution over parietal cortical areas and lateralized to the right was observed in the profoundly deaf. A graph theory analysis showed that brief training significantly increased fronto-central brain connectivity in deaf subjects, but not in NH subjects. Together, ERP tools and graph methods depicted the different functional brain dynamic in deaf and NH individuals, underlying the temporary engagement of the cognitive resources demanded by the task. Our findings showed that the index-fingertip somatosensory mechanoreceptors can discriminate sounds. Further studies are necessary to clarify brain connectivity dynamics associated with the performance of vibrotactile language-related discrimination tasks and the effect of lengthier training programs.