Serial position markers in space: visuospatial priming of serial order working memory retrieval

Spatial-positional associations in short-term memory can vanish in long-term memory

Studies on the SPoARC effect have shown that serial information is spatially processed in working memory. However, it remains unknown whether these spatial-positional associations are durable or only temporary. This study aimed at investigating whether spatialization would persist when a sequence presented repeatedly is expected to be chunked. If chunked, the items could be unified spatially and their spatialization could vanish. Thirty-seven participants performed a spatialization task which was remotely inspired by the Hebb repetition paradigm. A sequence of four stimuli presented individually in the middle of a computer screen was repeated throughout the task. After each sequence, participants had to decide whether a probe belonged to the series using two lateralized response keys. The results showed no spatialization for these repetitive sequences, on average. Moreover, further analysis revealed that the effect was detectable at the beginning of the task, suggesting that the more the sequence was repeated, the less participants spatialized information from left to right. These findings show that associations created in working memory between items and space can vanish in repeated sequences: we discuss the idea that working memory progressively saves on spatialization once a sequence is chunked in long-term memory.

The reciprocity of spatial-numerical associations of vocal response codes depends on stimulus mode

Individuals make faster left responses to small/er numbers and faster right responses to large/r numbers than vice versa. This "spatial-numerical association of response codes" (SNARC) effect represents evidence for an overlap between the cognitive representations of number and space. Theories of the SNARC effect differ in whether they predict bidirectional S-R associations between number and space or not. We investigated the reciprocity of S-R priming effects between number and location in three experiments with vocal responses. In Experiments 1 and 2, participants completed a number-location task, with digits as stimuli and location words as responses, and a location-number task, with physical locations as stimuli and number words as responses. In addition, we varied the S-R mapping in each task. Results revealed a strong SNARC effect in the number-location task, but no reciprocal SNARC effect in the location-number task. In Experiment 3, we replaced physical location stimuli with location words and digit stimuli with number words. Results revealed a regular and a reciprocal SNARC effect of similar size. Reciprocal SNARC effects thus seem to emerge with verbal location stimuli and vocal responses, but not with physical location stimuli and vocal responses. The S-R associations underlying the SNARC effect with vocal responses thus appear bidirectional and symmetrical for some combinations of stimulus and response sets, but not for others. This has implications for theoretical accounts of the SNARC effect which need to explain how stimulus mode affects the emergence of reciprocal but not regular SNARC effects.

Temporal and Spatial Information Elicit Different Power and Connectivity Profiles during Working Memory Maintenance

Working memory (WM) is the cognitive ability to store and manipulate information necessary for ongoing tasks. Although frontoparietal areas are involved in the retention of visually presented information, oscillatory neural activity differs for temporal and spatial WM processing. In this study, we corroborated previous findings describing the modulation of neural oscillations and expanded our investigation to the network organization underlying the cognitive processing of temporal and spatial information. We utilized MEG recordings during a Sternberg visual WM task. The spectral oscillatory activity in the maintenance phase revealed increased frontal theta (4-8 Hz) and parietal beta (13-30 Hz) in the temporal condition. Source level coherence analysis delineated the prominent role of parietal areas in all frequency bands during the maintenance of temporal information, whereas frontal and central areas showed major contributions in theta and beta ranges during the maintenance of spatial information. Our study revealed distinct spectral profiles of neural oscillations for separate cognitive subdomains of WM processing. The delineation of specific functional networks might have important implications for clinical applications, enabling the development of stimulation protocols targeting cognitive disabilities associated with WM impairments.

Optimized experimental designs to best detect spatial positional association of response codes in working memory

The SPoARC (Spatial Positional Association of Response Codes) effect refers to spatialization of information in working memory. Among the potential factors that could influence how order is mapped onto a mental space during the recognition process, we selected the following two factors: i) the type of stimuli, in particular their verbal vs. visual aspects and ii) the number of probes. In this study, 137 participants memorized sequences of either words or pictures and subsequently performed a recognition test for which they responded using lateralized keys. For half of the participants, only one probe was presented after each sequence, whereas the other half was administered several probes. A significantly greater number of participants presented a SPoARC using a single probe. We discuss that spatialization is best detected when the sequence is scanned only once. Results also showed no difference between the two types of stimuli (i.e., verbal vs. visual). This finding raises the question of the respective roles of verbalization and visualization in the SPoARC.

Learning and generalization of repetition-based rules in autism

Rule Learning (RL) allows us to extract and generalize high-order rules from a sequence of elements. Despite the critical role of RL in the acquisition of linguistic and social abilities, no study has investigated RL processes in Autism Spectrum Disorder (ASD). Here, we investigated RL in high-functioning autistic adolescents with ASD, examining whether their ability to extract and generalize rules from a sequence of visual elements is affected by the social vs. non-social nature of the stimulus and by visual working memory (WM). Using a forced-choice paradigm, ASD adolescents and typically developing (TD) peers were tested for their ability to detect and generalize high-order, repetition-based rules from visual sequences of simple non-social stimuli (shapes), complex non-social stimuli (inverted faces), and social stimuli (upright face). Both ASD and TD adolescents were able to generalize the rule they had learned to new stimuli, and their ability was modulated by the social nature of the stimuli and the complexity of the rule. Moreover, an association between RL and WM was found in the ASD, but not TD group, suggesting that ASD might have used additional or alternative strategies that relied on visual WM resources.

Sequential versus simultaneous presentation of memoranda in verbal working memory: (How) does it matter?

To-be-memorized information in verbal working memory (WM) can be presented sequentially, like in oral language, and simultaneously, like in written language. Few studies have addressed the importance and implications for verbal WM processing of these two presentation modes. While sequential presentation may favor discrete, temporal encoding processes, simultaneous presentation may favor spatial encoding processes. We compared immediate serial recall tasks for sequential versus simultaneous word list presentation with a specific focus on serial position curves of recall performance, transposition gradients, and the nature of serial order errors. First, we observed higher recall performance in the simultaneous compared to the sequential conditions, with a particularly large effect at end-of-list items. Moreover, results showed more transposition errors between non-adjacent items for the sequential condition, as well as more omission errors especially for start-of-list items. This observation can be explained in terms of differences in refreshing opportunities for start-of-list items during encoding between conditions. This study shows that the presentation mode of sequential material can have a significant impact on verbal WM performance, with an advantage for simultaneous encoding of sequence information.

A SPoARC of Music: Musicians Spatialize Melodies but not All-Comers

Recent studies on the spatial positional associated response codes (SPoARC) effect have shown that when Western adults are asked to keep in mind sequences of verbal items, they mentally spatialize them along the horizontal axis, with the initial items being associated with the left and the last items being associated with the right. The origin of this mental line is still debated, but it has been theorized that it necessitates specific spatial cognitive structures to emerge, which are built through expertise. This hypothesis is examined by testing for the first time whether Western individuals spatialize melodies from left to right and whether expertise in the musical domain is necessary for this effect to emerge. Two groups (musicians and non-musicians) of participants were asked to memorize sequences of four musical notes and to indicate if a subsequent probe was part of the sequence by pressing a "yes" key or a "no" key with the left or right index finger. Left/right-hand key assignment was reversed at mid-experiment. The results showed a SPoARC effect only for the group of musicians. Moreover, no association between pitch and hand responses was observed in either of the two groups. These findings suggest a crucial role of expertise in the SPoARC effect.

Neural Patterns in Parietal Cortex and Hippocampus Distinguish Retrieval of Start versus End Positions in Working Memory

Coding serial order of information is a fundamental ability of our cognitive system, and still, little is known about its neural substrate. This study examined the neural substrates involved in the retrieval of information that is serially stored in verbal working memory task using a sensitive multivariate analysis approach. We compared neural activity for memorized items stemming from the beginning versus the end of a memory list assessing the degree of neural pattern discordance between order positions (beginning vs. end). The present results confirmed and refined the role of the intraparietal sulcus in the processing of serial order information in working memory. An important finding is that the hippocampus showed sensitivity to serial order information. Our results indicate that the representation of serial order information relies on a broader set of neural areas and highlight the role of the intraparietal sulcus and the hippocampus, in addition to the supramarginal gyrus and the SMA. The contribution of different neural regions might reflect the involvement of distinct levels of serial order coding (i.e., spatial, attentional, temporal) that support the representation of serial order information.

Spatialization in working memory and its relation to math anxiety

Working memory (WM) is one of the most important cognitive functions that may play a role in the relation between math anxiety (MA) and math performance. The processing efficiency theory proposes that rumination and worrisome thoughts (induced by MA) result in less available WM resources (which are needed to solve math problems). At the same time, high MA individuals have lower verbal and spatial WM capacity in general. Extending these findings, we found that MA is also linked to the spatial coding of serial order in verbal WM: subjects who organize sequences from left-to-right in verbal WM show lower levels of MA compared with those who do not spatialize. Furthermore, these spatial coders have higher verbal WM capacity, better numerical order judgment abilities, and higher math scores. These findings suggest that spatially structuring the verbal mind is a promising cognitive correlate of MA and opens new avenues for exploring causal links between elementary cognitive processes and MA.

Eye-movements reveal the serial position of the attended item in verbal working memory

The problem of how the mind can retain sequentially organized information has a long research tradition that remains unresolved. While various computational models propose a mechanism of binding serial order information to position markers, the representational nature and processes that operate on these position markers are not clear. Recent behavioral work suggests that space is used to mark positions in serial order and that this process is governed by spatial attention. Based on the assumption that brain areas controlling spatial attention are also involved in saccadic planning, we continuously tracked the eye-movements as a direct measure of the spatial attention during retrieval from a verbal WM sequence. Participants memorized a sequence of auditory numbers. During retention, they heard a number-cue that did or did not belong to the memorized set. After this number-cue, a target-beep could be presented to which they had to respond if the number-cue belonged to the memorized sequence. In Experiment 1, the target-beep was either presented to the left or right ear, and in Experiment 2 bilaterally (removing any spatial aspect). We tested the hypothesis that systematic eye-movements are made when people retrieve items of sequences of auditory words and found that the retrieval of begin items resulted in leftward eye-movements and the retrieval of end items in rightward eye-movements. These observations indicate that the oculomotor system is also involved in the serial order processes in verbal WM thereby providing a promising novel approach to get insight into abstract cognitive processes.

Early is left and up: Saccadic responses reveal horizontal and vertical spatial associations of serial order in working memory

Maintaining serial order in working memory is crucial for cognition. Recent theories propose that serial information is achieved by positional coding of items on a spatial frame of reference. In line with this, an early-left and late-right spatial-positional association of response code (SPoARC) effect has been established. Various theoretical accounts have been put forward to explain the SPoARC effect (the mental whiteboard hypothesis, conceptual metaphor theory, polarity correspondence, or the indirect spatial-numerical association effect). Crucially, while all these accounts predict a left-to-right orientation of the SPoARC effect, they make different predictions regarding the direction of a possible vertical SPoARC effect. In this study, we therefore investigated SPoARC effects along the horizontal and vertical spatial dimension by means of saccadic responses. We replicated the left-to-right horizontal SPoARC effect and established for the first time an up-to-down vertical SPoARC effect. The direction of the vertical SPoARC effect was in contrast to that predicted by metaphor theory, polarity correspondence, or by the indirect spatial-numerical association effect. Rather, our results support the mental whiteboard-hypothesis, according to which positions can be flexibly coded on an internal space depending on the task demands. We also found that the strengths of the horizontal and vertical SPoARC effects were correlated, showing that some people are more prone than others to use spatial references for position coding. Our results therefore suggest that context templates used for position marking are not necessarily spatial in nature but depend on individual strategy preferences.

The development of working memory spatialization revealed by using the cave paradigm in a two-alternative spatial choice

When Western participants are asked to keep in mind a sequence of verbal items, they tend to associate the first items to the left and the last items to the right. This phenomenon, known as the spatial-positional association response codes effect, has been interpreted as showing that individuals spatialize the memoranda by creating a left-to-right mental line with them. One important gap in our knowledge concerns the development of this phenomenon: when do Western individuals start organizing their thought from left to right? To answer this question, 274 participants in seven age groups were tested (kindergarten, Grades 1, 2, 3, 4, and 5, and adults). We used a new protocol meant to be child-friendly, which involves associating two caves with two animals using a two-alternative spatial forced choice. Participants had to guess in which cave a specific animal could be hidden. Results showed that it is from Grade 3 on that participants spatialize information in working memory in a left-to-right fashion like adults.

The Association of Number and Space Under Different Tasks: Insight From a Process Perspective

We investigated the Spatial Numerical Association of Response Codes (SNARC) effect in 240 adults using a parity judgment and a magnitude classification task. Eight numbers from 1 to 9 except 5 were randomly presented one at a time, half of the participants were asked to compare these number with the target number 5 in the magnitude classification task; the other half of the participants were asked to judge whether these numbers were even or odd. It was called a phase when all eight numbers were tested; there were in total 16 phases. Detailed analyses of the changes in response times across the range of numbers and the chronological trend of the SNARC effect size over 16 phases estimated by a curvilinear regression model were reported. This phase-to-phase design and analyses provide an insight into the process of the SNARC effect in different tasks. We found that the SNARC effect emerged earlier and stayed more stable in magnitude classification task than in the parity task during the time course. Furthermore, the size of SNARC effect increased with time in the magnitude classification task, whereas it fluctuated up and down over time in the parity task. These findings indicate that the association of the number and space is dynamic and the process of the SNARC effect varies across tasks.

Hemispatial neglect and serial order in verbal working memory

Working memory refers to our ability to actively maintain and process a limited amount of information during a brief period of time. Often, not only the information itself but also its serial order is crucial for good task performance. It was recently proposed that serial order is grounded in spatial cognition. Here, we compared performance of a group of right hemisphere-damaged patients with hemispatial neglect to healthy controls in verbal working memory tasks. Participants memorized sequences of consonants at span level and had to judge whether a target consonant belonged to the memorized sequence (item task) or whether a pair of consonants were presented in the same order as in the memorized sequence (order task). In line with this idea that serial order is grounded in spatial cognition, we found that neglect patients made significantly more errors in the order task than in the item task compared to healthy controls. Furthermore, this deficit seemed functionally related to neglect severity and was more frequently observed following right posterior brain damage. Interestingly, this specific impairment for serial order in verbal working memory was not lateralized. We advance the hypotheses of a potential contribution to the deficit of serial order in neglect patients of either or both (1) reduced spatial working memory capacity that enables to keep track of the spatial codes that provide memorized items with a positional context, (2) a spatial compression of these codes in the intact representational space.

Grounding Verbal Working Memory: The Case of Serial Order

The maintenance of serial order in verbal working memory (WM) is a major unsolved puzzle in cognitive science. Here we review a series of studies showing that serial order in verbal WM closely interacts with spatial processing. Accordingly, we outline the “mental whiteboard hypothesis,” which postulates that serial order in verbal WM is grounded in the spatial attention system. Specifically, serial context in verbal WM is provided by binding the memoranda to coordinates within an internal, spatially defined system within which (internal) spatial attention is at play to the purpose of searching for and retrieving information. Challenges and opportunities to be considered in future studies are discussed.

Order Information in Verbal Working Memory Shifts the Subjective Midpoint in Both the Line Bisection and the Landmark Tasks

A largely substantiated view in the domain of working memory is that the maintenance of serial order is achieved by generating associations of each item with an independent representation of its position, so-called position markers. Recent studies reported that the ordinal position of an item in verbal working memory interacts with spatial processing. This suggests that position markers might be spatial in nature. However, these interactions were so far observed in tasks implying a clear binary categorization of space (i.e., with left and right responses or targets). Such binary categorizations leave room for alternative interpretations, such as congruency between non-spatial categorical codes for ordinal position (e.g., begin and end) and spatial categorical codes for response (e.g., left and right). Here we discard this interpretation by providing evidence that this interaction can also be observed in a task that draws upon a continuous processing of space, the line bisection task. Specifically, bisections are modulated by ordinal position in verbal working memory, with lines bisected more towards the right after retrieving items from the end compared to the beginning of the memorized sequence. This supports the idea that position markers are intrinsically spatial in nature.

Infants learn better from left to right: a directional bias in infants' sequence learning

A wealth of studies show that human adults map ordered information onto a directional spatial continuum. We asked whether mapping ordinal information into a directional space constitutes an early predisposition, already functional prior to the acquisition of symbolic knowledge and language. While it is known that preverbal infants represent numerical order along a left-to-right spatial continuum, no studies have investigated yet whether infants, like adults, organize any kind of ordinal information onto a directional space. We investigated whether 7-month-olds' ability to learn high-order rule-like patterns from visual sequences of geometric shapes was affected by the spatial orientation of the sequences (left-to-right vs. right-to-left). Results showed that infants readily learn rule-like patterns when visual sequences were presented from left to right, but not when presented from right to left. This result provides evidence that spatial orientation critically determines preverbal infants' ability to perceive and learn ordered information in visual sequences, opening to the idea that a left-to-right spatially organized mental representation of ordered dimensions might be rooted in biologically-determined constraints on human brain development.

How Does Working Memory Enable Number-Induced Spatial Biases?

Number-space associations are a robust observation, but their underlying mechanisms remain debated. Two major accounts have been identified. First, spatial codes may constitute an intrinsic part of number representations stored in the brain – a perspective most commonly referred to as the Mental Number Line account. Second, spatial codes may be generated at the level of working memory when number (or other) representations are coordinated in function of a specific task. The aim of the current paper is twofold. First, whereas a pure Mental Number Line account cannot capture the complexity of observations reported in the literature, we here explore if and how a pure working memory account can suffice. Second, we make explicit (more than in our earlier work) the potential building blocks of such a working memory account, thereby providing clear and concrete foci for empirical efforts to test the feasibility of the account.

Distinctiveness as a function of spatial expansion in verbal working memory: comment on Kreitz, Furley, Memmert, and Simons (2015)

In a recent study, Kreitz et al. (Psychological Research 79:1034-1041, 2015) reported on a relationship between verbal working memory capacity and visuo-spatial attentional breadth. The authors hinted at attentional control to be the major link underlying this relationship. We put forward an alternative explanation by framing it within the context of a recent theory on serial order in memory: verbal item sequences entering in working memory are coded by adding a spatial context that can be derived from reading/writing habits. The observation by Kreitz et al. (Psychological Research 79:1034-1041, 2015) enriches this framework by suggesting that a larger visuo-spatial attentional breadth allows for internal coding of the verbal items in a more (spatially) distinct manner-thereby increasing working memory performance. As such, Kreitz et al. (Psychological Research 79:1034-1041, 2015) is the first study revealing a functional link between visuo-spatial attentional breadth and verbal working memory size, which strengthens spatial accounts of serial order coding in working memory.