Finding the answer in space: the mental whiteboard hypothesis on serial order in working memory

Multiple left-to-right spatial representations of number magnitudes? Evidence from left spatial neglect

The SNARC effect reflects the observation that when healthy observers with left-to-right reading habits are asked to compare the magnitude or to judge the parity of numbers, they provide faster reaction times (RT) to small numbers with left-sided responses and faster RTs to large numbers with right-sided responses. In magnitude comparison (MC), right brain damaged patients with left-sided neglect typically show a pathologically enlarged SNARC for large numbers and selective slowing to numbers that are immediately lower than the numerical reference (e.g. 4 for reference 5). This asymmetry has been taken as evidence that small numbers are mentally positioned to the left of the reference and, therefore, are processed less efficiently by patients neglecting the left side of space. In parity judgement (PJ), on the other hand, the size of the SNARC effect is unaffected by neglect. This dissociation is typically attributed to the disturbed explicit processing of number magnitude in MC and preserved implicit processing of magnitude in PJ. Before accepting this interpretation, however, it remains to be investigated whether neglect patients show the same RT pattern that characterizes the performance of healthy participants (i.e. left-side RTs that increase linearly as a function of number magnitude and right-side RTs that decrease linearly as a function of magnitude). Clarifying this point is crucial, because an equally sized SNARC can originate from different RT patterns. Here we demonstrate that the RT pattern of neglect patients during PJ is entirely comparable to those of patients without neglect and healthy controls, while the same neglect patients show selective slowing to numbers that are immediately lower than the numerical reference in MC. These findings suggest the existence of multiple left-to-right spatial representations of number magnitude and provides an explanation of the functional dissociation between MC and PJ tasks.

Current directions in visual working memory research: An introduction and emerging insights

Visual working memory (VWM) is a core construct in the cognitive (neuro-)sciences, assumed to serve as a hub for information exchange and thus supporting a multitude of cognitive functions related to processing visual information. Here, we give an introduction into key terms and paradigms and an overview of ongoing debates in the field, to which the articles collected in this Special Issue on 'Current Directions in Visual Working Memory Research' contribute. Our aim is to extract, from this overview, some 'emerging' theoretical insights concerning questions such as the optimal way to examine VWM, which types of mental representations contribute to performance on VWM tasks, and how VWM keeps features from the same object together and apart from features of concurrently maintained objects (the binding problem).

Mechanism of the SNARC Effect in Numerical Magnitude, Time Sequence, and Spatial Sequence Tasks: Involvement of LTM and WM

The spatial-numerical association of response codes (SNARC) effect refers to the phenomenon that responses involving small numbers are faster with the left hand and responses involving large numbers are faster with the right hand. Previous studies have investigated the mechanism of the SNARC effect only when the time sequence is induced by centrally presented successive numbers. No study has investigated the mechanism of the SNARC effect when the spatial sequence is induced. Given that the induction of a spatial sequence together with a time sequence provides a new temporary reference for the serial order to be coded in working memory (WM), it would be interesting to examine the SNARC effect when both the time sequence and spatial sequence are induced. Therefore, a novel priming paradigm that simultaneously induced the time sequence and spatial sequence was employed in the present study to investigate the mechanism of the SNARC effect. Specifically, the time sequence and spatial sequence were induced by presenting a series of self-paced successive numbers, centrally or in a left-to-right or right-to-left direction, on the screen. Following the presentation of successive numbers, the probe number was centrally presented on the screen and university students were asked to distinguish to which time sequence or spatial sequence the probe number belonged by pressing a specified key of a qwerty keyboard. The results indicated that (1) the SNARC effect simultaneously appeared in the processing of the number magnitude and time sequence when only the time sequence was induced. (2) The SNARC effect disappeared in the processing of the time sequence; however, the SNARC effect coexisted in the processing of the numerical magnitude and spatial sequence when the spatial sequence was induced and participants performed a time sequence relevant task. (3) The SNARC effect coexisted in the processing of the numerical magnitude, time sequence, and spatial sequence when the spatial sequence was induced and participants performed a spatial sequence relevant task. Based on these results, we conclude that whether the SNARC effect coexists in the processing of the numerical magnitude, the time sequence and spatial sequence were influenced by the spatial sequence and relevant task. The results further support the mental whiteboard hypothesis and extended the WM account. Implications for theories on the SNARC effect were discussed.

Systematic spatial patterns of the sense of familiarity: Hierarchical modelling based on eye-tracking experiments

Using different types of stimuli, such as pictures, horizontally written Japanese words, and vertically written Japanese words, this study investigated the spatial patterns of the sense of familiarity within the visual field. The perceptual asymmetry theory predicted that stimuli in the lower visual field would be processed more fluently and would therefore be perceived as more familiar. The working memory theory, originally proposed in space-number research, envisaged type-specific spatial patterns for different stimuli. Participants made old/new recognition memory judgements for stimuli, presented at random positions, while their eye movements were recorded. The observed spatial patterns changed according to the stimulus type (e.g., "more left = older" for horizontally written words and "upper = older" for vertically written words), and this flexibility is encapsulated by the working memory theory as follows: (a) stimulus-type-specific spatial configurations are encoded in long-term memory on the basis of one's experience (e.g., vertically written words are empirically associated with the "upper = older" spatial configuration), (b) the presentation of a stimulus automatically cues the temporal activation of the associated spatial configuration in working memory, and (c) the referential process between the stimulus and configuration unconsciously affects the viewer's sense of familiarity.

Visuospatial bootstrapping: Binding useful visuospatial information during verbal working memory encoding does not require set-shifting executive resources

Immediate serial recall of digits is better when the digits are shown by highlighting them in a familiar array, such as a phone keypad, compared with presenting them serially in a single location, a pattern referred to as "visuospatial bootstrapping." This pattern implies the establishment of temporary links between verbal and spatial working memory, alongside access to information in long-term memory. However, the role of working memory control processes like those implied by the "Central Executive" in bootstrapping has not been directly investigated. Here, we report a study addressing this issue, focusing on executive processes of attentional shifting. Tasks in which information has to be sequenced are thought to be heavily dependent on shifting. Memory for digits presented in keypads versus single locations was assessed under two secondary task load conditions, one with and one without a sequencing requirement, and hence differing in the degree to which they invoke shifting. Results provided clear evidence that multimodal binding (visuospatial bootstrapping) can operate independently of this form of executive control process.

Impaired Processing of Serial Order Determines Working Memory Impairments in Alzheimer's Disease

BACKGROUND

Working memory (WM) problems are commonly observed in Alzheimer's disease (AD), but the affected mechanisms leading to impaired WM are still insufficiently understood. The ability to efficiently process serial order in WM has been demonstrated to be fundamental to fluent daily life functioning. The decreased capability to mentally process serial position in WM has been put forward as the underlying explanation for generally compromised WM performance.

OBJECTIVE

Determine which mechanisms, such as order processing, are responsible for deficient WM functioning in AD.

METHOD

A group of AD patients (n = 32) and their partners (n = 25), assigned to the control group, were submitted to an extensive battery of neuropsychological and experimental tasks, assessing general cognitive state and functioning of several aspects related to serial order WM.

RESULTS

The results revealed an impaired ability to bind item information to serial position within WM in AD patients compared to controls. It was additionally observed that AD patients experienced specific difficulties with directing spatial attention when searching for item information stored in WM.

CONCLUSION

The processing of serial order and the allocation of attentional resources are both disrupted, explaining the generally reduced WM functioning in AD patients. Further studies should now clarify whether this observation could explain disease-related problems for other cognitive functions such as verbal expression, auditory comprehension, or planning.

Working Memory Updating: Load and Binding

In the present study, we aimed to examine how specific objects are updated in working memory. We compared conditions in which contents or content-context bindings from working memory were both encoded and updated (Experiment 1). In addition, for bindings, we manipulated the memory load (i.e., number of contents) to maintain during updating. Results indicated that memory load did not specifically affect the process; rather, the content-context binding (vs. single contents) was critical in determining the increase in response latencies. Results were replicated even in Experiment 2, in which we manipulated the spatial locations of the to-be-recognized probes. Results showed evidence of a potential dissociation between updating of memory contents-only and content-context bindings. In addition, memory load and spatial coherence between phases and probe recognition did not interact with updating performance. Overall, results were taken as a contribution toward mapping the complex nature of the updating mechanism.

Hemispatial neglect and serial order in verbal working memory

Working memory refers to our ability to actively maintain and process a limited amount of information during a brief period of time. Often, not only the information itself but also its serial order is crucial for good task performance. It was recently proposed that serial order is grounded in spatial cognition. Here, we compared performance of a group of right hemisphere-damaged patients with hemispatial neglect to healthy controls in verbal working memory tasks. Participants memorized sequences of consonants at span level and had to judge whether a target consonant belonged to the memorized sequence (item task) or whether a pair of consonants were presented in the same order as in the memorized sequence (order task). In line with this idea that serial order is grounded in spatial cognition, we found that neglect patients made significantly more errors in the order task than in the item task compared to healthy controls. Furthermore, this deficit seemed functionally related to neglect severity and was more frequently observed following right posterior brain damage. Interestingly, this specific impairment for serial order in verbal working memory was not lateralized. We advance the hypotheses of a potential contribution to the deficit of serial order in neglect patients of either or both (1) reduced spatial working memory capacity that enables to keep track of the spatial codes that provide memorized items with a positional context, (2) a spatial compression of these codes in the intact representational space.

Grounding Verbal Working Memory: The Case of Serial Order

The maintenance of serial order in verbal working memory (WM) is a major unsolved puzzle in cognitive science. Here we review a series of studies showing that serial order in verbal WM closely interacts with spatial processing. Accordingly, we outline the “mental whiteboard hypothesis,” which postulates that serial order in verbal WM is grounded in the spatial attention system. Specifically, serial context in verbal WM is provided by binding the memoranda to coordinates within an internal, spatially defined system within which (internal) spatial attention is at play to the purpose of searching for and retrieving information. Challenges and opportunities to be considered in future studies are discussed.

Order Information in Verbal Working Memory Shifts the Subjective Midpoint in Both the Line Bisection and the Landmark Tasks

A largely substantiated view in the domain of working memory is that the maintenance of serial order is achieved by generating associations of each item with an independent representation of its position, so-called position markers. Recent studies reported that the ordinal position of an item in verbal working memory interacts with spatial processing. This suggests that position markers might be spatial in nature. However, these interactions were so far observed in tasks implying a clear binary categorization of space (i.e., with left and right responses or targets). Such binary categorizations leave room for alternative interpretations, such as congruency between non-spatial categorical codes for ordinal position (e.g., begin and end) and spatial categorical codes for response (e.g., left and right). Here we discard this interpretation by providing evidence that this interaction can also be observed in a task that draws upon a continuous processing of space, the line bisection task. Specifically, bisections are modulated by ordinal position in verbal working memory, with lines bisected more towards the right after retrieving items from the end compared to the beginning of the memorized sequence. This supports the idea that position markers are intrinsically spatial in nature.

Infants learn better from left to right: a directional bias in infants' sequence learning

A wealth of studies show that human adults map ordered information onto a directional spatial continuum. We asked whether mapping ordinal information into a directional space constitutes an early predisposition, already functional prior to the acquisition of symbolic knowledge and language. While it is known that preverbal infants represent numerical order along a left-to-right spatial continuum, no studies have investigated yet whether infants, like adults, organize any kind of ordinal information onto a directional space. We investigated whether 7-month-olds' ability to learn high-order rule-like patterns from visual sequences of geometric shapes was affected by the spatial orientation of the sequences (left-to-right vs. right-to-left). Results showed that infants readily learn rule-like patterns when visual sequences were presented from left to right, but not when presented from right to left. This result provides evidence that spatial orientation critically determines preverbal infants' ability to perceive and learn ordered information in visual sequences, opening to the idea that a left-to-right spatially organized mental representation of ordered dimensions might be rooted in biologically-determined constraints on human brain development.

Domain-Generality of Timing-Based Serial Order Processes in Short-Term Memory: New Insights from Musical and Verbal Domains

Several models in the verbal domain of short-term memory (STM) consider a dissociation between item and order processing. This view is supported by data demonstrating that different types of time-based interference have a greater effect on memory for the order of to-be-remembered items than on memory for the items themselves. The present study investigated the domain-generality of the item versus serial order dissociation by comparing the differential effects of time-based interfering tasks, such as rhythmic interference and articulatory suppression, on item and order processing in verbal and musical STM domains. In Experiment 1, participants had to maintain sequences of verbal or musical information in STM, followed by a probe sequence, this under different conditions of interference (no-interference, rhythmic interference, articulatory suppression). They were required to decide whether all items of the probe list matched those of the memory list (item condition) or whether the order of the items in the probe sequence matched the order in the memory list (order condition). In Experiment 2, participants performed a serial order probe recognition task for verbal and musical sequences ensuring sequential maintenance processes, under no-interference or rhythmic interference conditions. For Experiment 1, serial order recognition was not significantly more impacted by interfering tasks than was item recognition, this for both verbal and musical domains. For Experiment 2, we observed selective interference of the rhythmic interference condition on both musical and verbal order STM tasks. Overall, the results suggest a similar and selective sensitivity to time-based interference for serial order STM in verbal and musical domains, but only when the STM tasks ensure sequential maintenance processes.

Counteracting Implicit Conflicts by Electrical Inhibition of the Prefrontal Cortex

Cognitive conflicts and distractions by task-irrelevant information often counteract effective and goal-directed behaviors. In some cases, conflicting information can even emerge implicitly, without an overt distractor, by the automatic activation of mental representations. For instance, during number processing, magnitude information automatically elicits spatial associations resembling a mental number line. This spatial–numerical association of response codes (SNARC) effect can modulate cognitive-behavioral performance but is also highly flexible and context-dependent, which points toward a critical involvement of working memory functions. Transcranial direct current stimulation to the PFC, in turn, has been effective in modulating working memory-related cognitive performance. In a series of experiments, we here demonstrate that decreasing activity of the left PFC by cathodal transcranial direct current stimulation consistently and specifically eliminates implicit cognitive conflicts based on the SNARC effect, but explicit conflicts based on visuospatial distraction remain unaffected. This dissociation is polarity-specific and appears unrelated to functional magnitude processing as classified by regular numerical distance effects. These data demonstrate a causal involvement of the left PFC in implicit cognitive conflicts based on the automatic activation of spatial–numerical processing. Corroborating the critical interaction of brain stimulation and neurocognitive functions, our findings suggest that distraction from goal-directed behavior by automatic activation of implicit, task-irrelevant information can be blocked by the inhibition of prefrontal activity.

Early but not late blindness leads to enhanced arithmetic and working memory abilities

Behavioural and neurophysiological evidence suggest that vision plays an important role in the emergence and development of arithmetic abilities. However, how visual deprivation impacts on the development of arithmetic processing remains poorly understood. We compared the performances of early (EB), late blind (LB) and sighted control (SC) individuals during various arithmetic tasks involving addition, subtraction and multiplication of various complexities. We also assessed working memory (WM) performances to determine if they relate to a blind person's arithmetic capacities. Results showed that EB participants performed better than LB and SC in arithmetic tasks, especially in conditions in which verbal routines and WM abilities are needed. Moreover, EB participants also showed higher WM abilities. Together, our findings demonstrate that the absence of developmental vision does not prevent the development of refined arithmetic skills and can even trigger the refinement of these abilities in specific tasks.

Becoming an expert: Ontogeny of expertise as an example of neural reuse

In this commentary, we discuss an important pattern of results in the literature on the neural basis of expertise: (a) decrease of cerebral activation at the beginning of acquisition of expertise and (b) functional cerebral reorganization as a consequence of years of practice. We show how these two results can be integrated with the neural reuse framework.

How Does Working Memory Enable Number-Induced Spatial Biases?

Number-space associations are a robust observation, but their underlying mechanisms remain debated. Two major accounts have been identified. First, spatial codes may constitute an intrinsic part of number representations stored in the brain – a perspective most commonly referred to as the Mental Number Line account. Second, spatial codes may be generated at the level of working memory when number (or other) representations are coordinated in function of a specific task. The aim of the current paper is twofold. First, whereas a pure Mental Number Line account cannot capture the complexity of observations reported in the literature, we here explore if and how a pure working memory account can suffice. Second, we make explicit (more than in our earlier work) the potential building blocks of such a working memory account, thereby providing clear and concrete foci for empirical efforts to test the feasibility of the account.

Hemispatial Neglect Shows That "Before" Is "Left"

Recent research has led to the hypothesis that events which unfold in time might be spatially represented in a left-to-right fashion, resembling writing direction. Here we studied fourteen right-hemisphere damaged patients, with or without neglect, a disorder of spatial awareness affecting contralesional (here left) space processing and representation. We reasoned that if the processing of time-ordered events is spatial in nature, it should be impaired in the presence of neglect and spared in its absence. Patients categorized events of a story as occurring before or after a central event, which acted as a temporal reference. An asymmetric distance effect emerged in neglect patients, with slower responses to events that took place before the temporal reference. The event occurring immediately before the reference elicited particularly slow responses, closely mirroring the pattern found in neglect patients performing numerical comparison tasks. Moreover, the first item elicited significantly slower responses than the last one, suggesting a preference for a left-to-right scanning/representation of events in time. Patients without neglect showed a regular and symmetric distance effect. These findings further suggest that the representation of events order is spatial in nature and provide compelling evidence that ordinality is similarly represented within temporal and numerical domains.

Distinctiveness as a function of spatial expansion in verbal working memory: comment on Kreitz, Furley, Memmert, and Simons (2015)

In a recent study, Kreitz et al. (Psychological Research 79:1034-1041, 2015) reported on a relationship between verbal working memory capacity and visuo-spatial attentional breadth. The authors hinted at attentional control to be the major link underlying this relationship. We put forward an alternative explanation by framing it within the context of a recent theory on serial order in memory: verbal item sequences entering in working memory are coded by adding a spatial context that can be derived from reading/writing habits. The observation by Kreitz et al. (Psychological Research 79:1034-1041, 2015) enriches this framework by suggesting that a larger visuo-spatial attentional breadth allows for internal coding of the verbal items in a more (spatially) distinct manner-thereby increasing working memory performance. As such, Kreitz et al. (Psychological Research 79:1034-1041, 2015) is the first study revealing a functional link between visuo-spatial attentional breadth and verbal working memory size, which strengthens spatial accounts of serial order coding in working memory.

Inhibition of return and attentional facilitation: Numbers can be counted in, letters tell a different story

Prior research has provided strong evidence for spatial-numerical associations. Single digits can for instance act as attentional cues, orienting visuo-spatial attention to the left or right hemifield depending on the digit's magnitude, thus facilitating target detection in the cued hemifield (left/right hemifield after small/large digits, respectively). Studies using other types of behaviourally or biologically relevant central cues known to elicit automated symbolic attention orienting effects such as arrows or gaze have shown that the initial facilitation of cued target detection can turn into inhibition at longer stimulus onset asynchronies (SOAs). However, no studies so far investigated whether inhibition of return (IOR) is also observed using digits as uninformative central cues. To address this issue we designed an attentional cueing paradigm using SOAs ranging from 500 ms to 1650 ms. As expected, the results showed a facilitation effect at the relatively short 650 ms SOA, replicating previous findings. At the long 1650 ms SOA, however, participants were faster to detect targets in the uncued hemifield compared to the cued hemifield, showing an IOR effect. A control experiment with letters showed no such congruency effects at any SOA. These findings provide the first evidence that digits not only produce facilitation effects at shorter intervals, but also induce inhibitory effects at longer intervals, confirming that Arabic digits engage automated symbolic orienting of attention.

Spatial displacement of numbers on a vertical number line in spatial neglect

Previous studies that investigated the association of numbers and space in humans came to contradictory conclusions about the spatial character of the mental number magnitude representation and about how it may be influenced by unilateral spatial neglect. The present study aimed to disentangle the debated influence of perceptual vs. representational aspects via explicit mapping of numbers onto space by applying the number line estimation paradigm with vertical orientation of stimulus lines. Thirty-five acute right-brain damaged stroke patients (6 with neglect) were asked to place two-digit numbers on vertically oriented lines with 0 marked at the bottom and 100 at the top. In contrast to the expected, nearly linear mapping in the control patient group, patients with spatial neglect overestimated the position of numbers in the lower middle range. The results corroborate spatial characteristics of the number magnitude representation. In neglect patients, this representation seems to be biased towards the ipsilesional side, independent of the physical orientation of the task stimuli.

Serial position markers in space: visuospatial priming of serial order working memory retrieval

Most general theories on serial order working memory (WM) assume the existence of position markers that are bound to the to-be-remembered items to keep track of the serial order. So far, the exact cognitive/neural characteristics of these markers have remained largely underspecified, while direct empirical evidence for their existence is mostly lacking. In the current study we demonstrate that retrieval from verbal serial order WM can be facilitated or hindered by spatial cuing: begin elements of a verbal WM sequence are retrieved faster after cuing the left side of space, while end elements are retrieved faster after cuing the right side of space. In direct complement to our previous work--where we showed the reversed impact of WM retrieval on spatial processing--we argue that the current findings provide us with a crucial piece of evidence suggesting a direct and functional involvement of space in verbal serial order WM. We outline the idea that serial order in verbal WM is coded within a spatial coordinate system with spatial attention being involved when searching through WM, and we discuss how this account can explain several hallmark observations related to serial order WM.