Reading positional codes with fMRI: Problems and solutions

Reactivating ordinal position information from auditory sequence memory in human brains

Retaining a sequence of events in their order is a core ability of many cognitive functions, such as speech recognition, movement control, and episodic memory. Although content representations have been widely studied in working memory (WM), little is known about how ordinal position information of an auditory sequence is retained in the human brain as well as its coding characteristics. In fact, there is still a lack of an efficient approach to directly accessing the stored ordinal position code during WM retention. Here, 31 participants performed an auditory sequence WM task with their brain activities recorded using electroencephalography (EEG). We developed new triggering events that could successfully reactivate neural representations of ordinal position during the delay period. Importantly, the ordinal position reactivation is further related to recognition behavior, confirming its indexing of WM storage. Furthermore, the ordinal position code displays an intriguing “stable-dynamic” format, i.e. undergoing the same dynamic neutral trajectory in the multivariate neural space during both encoding and retention (whenever reactivated). Overall, our results provide an effective approach to accessing the behaviorally-relevant ordinal position information in auditory sequence WM and reveal its new temporal characteristics.

Incidental Verbal Semantic Processing Recruits the Fronto-temporal Semantic Control Network

The frontoparietal semantic network, encompassing the inferior frontal gyrus and the posterior middle temporal cortex, is considered to be involved in semantic control processes. The explicit versus implicit nature of these control processes remains however poorly understood. The present study examined this question by assessing regional brain responses to the semantic attributes of an unattended stream of auditory words while participants' top-down attentional control processes were absorbed by a demanding visual search task. Response selectivity to semantic aspects of verbal stimuli was assessed via a functional magnetic resonance imaging response adaptation paradigm. We observed that implicit semantic processing of an unattended verbal stream recruited not only unimodal and amodal cortices in posterior supporting semantic knowledge areas, but also inferior frontal and posterior middle temporal areas considered to be part of the semantic control network. These results indicate that frontotemporal semantic networks support incidental semantic (control) processes.

Neural Representations of Task Context and Temporal Order During Action Sequence Execution

Routine action sequences can share a great deal of similarity in terms of their stimulus response mappings. As a consequence, their correct execution relies crucially on the ability to preserve contextual and temporal information. However, there are few empirical studies on the neural mechanism and the brain areas maintaining such information. To address this gap in the literature, we recently recorded the blood-oxygen level dependent (BOLD) response in a newly developed coffee-tea making task. The task involves the execution of four action sequences that each comprise six consecutive decision states, which allows for examining the maintenance of contextual and temporal information. Here, we report a reanalysis of this dataset using a data-driven approach, namely multivariate pattern analysis, that examines context-dependent neural activity across several predefined regions of interest. Results highlight involvement of the inferior-temporal gyrus and lateral prefrontal cortex in maintaining temporal and contextual information for the execution of hierarchically organized action sequences. Furthermore, temporal information seems to be more strongly encoded in areas over the left hemisphere.

Hierarchical Representation of Multistep Tasks in Multiple-Demand and Default Mode Networks

Task episodes consist of sequences of steps that are performed to achieve a goal. We used fMRI to examine neural representation of task identity, component items, and sequential position, focusing on two major cortical systems—the multiple-demand (MD) and default mode networks (DMN). Human participants (20 males, 22 females) learned six tasks each consisting of four steps. Inside the scanner, participants were cued which task to perform and then sequentially identified the target item of each step in the correct order. Univariate time course analyses indicated that intra-episode progress was tracked by a tonically increasing global response, plus an increasing phasic step response specific to MD regions. Inter-episode boundaries evoked a widespread response at episode onset, plus a marked offset response specific to DMN regions. Representational similarity analysis (RSA) was used to examine representation of task identity and component steps. Both networks represented the content and position of individual steps, however the DMN preferentially represented task identity while the MD network preferentially represented step-level information. Thus, although both MD and DMN networks are sensitive to step-level and episode-level information in the context of hierarchical task performance, they exhibit dissociable profiles in terms of both temporal dynamics and representational content. The results suggest collaboration of multiple brain regions in control of multistep behavior, with MD regions particularly involved in processing the detail of individual steps, and DMN adding representation of broad task context.

SIGNIFICANCE STATEMENT Achieving one's goals requires knowing what to do and when. Tasks are typically hierarchical, with smaller steps nested within overarching goals. For effective, flexible behavior, the brain must represent both levels. We contrast response time courses and information content of two major cortical systems—the multiple-demand (MD) and default mode networks (DMN)—during multistep task episodes. Both networks are sensitive to step-level and episode-level information, but with dissociable profiles. Intra-episode progress is tracked by tonically increasing global responses, plus MD-specific increasing phasic step responses. Inter-episode boundaries evoke widespread responses at episode onset, plus DMN-specific offset responses. Both networks represent content and position of individual steps; however, the DMN and MD networks favor task identity and step-level information, respectively.

Decoding hierarchical control of sequential behavior in oscillatory EEG activity

Despite strong theoretical reasons for assuming that abstract representations organize complex action sequences in terms of subplans (chunks) and sequential positions, we lack methods to directly track such content-independent, hierarchical representations in humans. We applied time-resolved, multivariate decoding analysis to the pattern of rhythmic EEG activity that was registered while participants planned and executed individual elements from pre-learned, structured sequences. Across three experiments, the theta and alpha-band activity coded basic elements and abstract control representations, in particular, the ordinal position of basic elements, but also the identity and position of chunks. Further, a robust representation of higher level, chunk identity information was only found in individuals with above-median working memory capacity, potentially providing a neural-level explanation for working-memory differences in sequential performance. Our results suggest that by decoding oscillatory activity we can track how the cognitive system traverses through the states of a hierarchical control structure.

Short-term memory for spatial, sequential and duration information

•

Analog report methods provide novel insights on STM for space and time.

•

Space and time may be used to bind features in STM.

•

The hippocampus is involved in object-location binding in STM.

Space and time appear to play key roles in the way that information is organized in short-term memory (STM). Some argue that they are crucial contexts within which other stored features are embedded, allowing binding of information that belongs together within STM. Here we review recent behavioral, neurophysiological and imaging studies that have sought to investigate the nature of spatial, sequential and duration representations in STM, and how these might break down in disease. Findings from these studies point to an important role of the hippocampus and other medial temporal lobe structures in aspects of STM, challenging conventional accounts of involvement of these regions in only long-term memory.

A shared representation of order between encoding and recognition in visual short-term memory

Many complex tasks require people to bind individual events into a sequence that can be held in short term memory (STM). For this purpose information about the order of the individual events in the sequence needs to be maintained in an active and accessible form in STM over a period of few seconds. Here we investigated how the temporal order information is shared between the presentation and response phases of an STM task. We trained a classification algorithm on the fMRI activity patterns from the presentation phase of the STM task to predict the order of the items during the subsequent recognition phase. While voxels in a number of brain regions represented positional information during either presentation and recognition phases, only voxels in the lateral prefrontal cortex (PFC) and the anterior temporal lobe (ATL) represented position consistently across task phases. A shared positional code in the ATL might reflect verbal recoding of visual sequences to facilitate the maintenance of order information over several seconds.