Differentiating between Verbal and Spatial Encoding using Eye-Movement Recordings

Navigating the mind's eye: Understanding gaze shifts in visuospatial bootstrapping

Visuospatial bootstrapping refers to the well-replicated phenomena in which serial recall in a purely verbal task is boosted by presenting digits within the familiar spatial layout of a typical telephone keypad. The visuospatial bootstrapping phenomena indicates that additional support comes from long-term knowledge of a fixed spatial pattern, and prior experimentation supports the idea that access to this benefit depends on the availability of the visuospatial motor system. We investigate this by tracking participants' eye movements during encoding and retention of verbal lists to learn whether gaze patterns support verbal memory differently when verbal information is presented in the familiar visual layout. Participants' gaze was recorded during attempts to recall lists of seven digits in three formats: centre of the screen, typical telephone keypad, or a spatially identical layout with randomised number placement. Performance was better with the typical than with the novel layout. Our data show that eye movements differ when encoding and retaining verbal information that has a familiar layout compared with the same verbal information presented in a novel layout, suggesting recruitment of different spatial rehearsal strategies. However, no clear link between gaze pattern and recall accuracy was observed, which suggests that gazes play a limited role in retention, at best.

Dynamic resource allocation in spatial working memory during full and partial report tasks

Serial position effects are well-documented in working memory literature. Studies of spatial short-term memory that rely on binary response; full report tasks tend to report stronger primacy than recency effects. In contrast, studies that utilize a continuous response, partial report task report stronger recency than primacy effects (Gorgoraptis, Catalao, Bays, & Husain, 2011; Zokaei, Gorgoraptis, Bahrami, Bays, & Husain, 2011). The current study explored the idea that probing spatial working memory using full and partial continuous response tasks would produce different distributions of visuospatial working memory resources across spatial sequences and, therefore, explain the conflicting results in the literature. Experiment 1 demonstrated that primacy effects were observed when memory was probed with a full report task. Experiment 2 confirmed this finding while controlling eye movements. Critically, Experiment 3 demonstrated that switching from a full to a partial report task abolished the primacy effect and produced a recency effect, consistent with the idea that the distribution of resources in visuospatial working memory depends on the type of recall required. It is argued that the primacy effect in the whole report task arose from the accumulation of noise caused by the execution of multiple spatially directed actions during recall, whereas the recency effect in the partial report task reflects the redistribution of preallocated resources when an anticipated item is not presented. These data show that it is possible to reconcile apparently contradictory findings within the resource theory of spatial working memory and the importance of considering how memory is probed when interpreting behavioral data through the lens of resource theories of spatial working memory.

Encoding specificity instead of online integration of real-world spatial regularities for objects in working memory

Most objects show high degrees of spatial regularity (e.g. beach umbrellas appear above, not under, beach chairs). The spatial regularities of real-world objects benefit visual working memory (VWM), but the mechanisms behind this spatial regularity effect remain unclear. The "encoding specificity" hypothesis suggests that spatial regularity will enhance the visual encoding process but will not facilitate the integration of information online during VWM maintenance. The "perception-alike" hypothesis suggests that spatial regularity will function in both visual encoding and online integration during VWM maintenance. We investigated whether VWM integrates sequentially presented real-world objects by focusing on the existence of the spatial regularity effect. Throughout five experiments, we manipulated the presentation (simultaneous vs. sequential) and regularity (with vs. without regularity) of memory arrays among pairs of real-world objects. The spatial regularity of memory objects presented simultaneously, but not sequentially, improved VWM performance. We also examined whether memory load, verbal suppression and masking, and memory array duration hindered the spatial regularity effect in sequential presentation. We found a stable absence of the spatial regularity effect, suggesting that the participants were unable to integrate real-world objects based on spatial regularities online. Our results support the encoding specificity hypothesis, wherein the spatial regularity of real-world objects can enhance the efficiency of VWM encoding, but VWM cannot exploit spatial regularity to help organize sampled sequential information into meaningful integrations.

Charting the Diversity of Strategic Processes in Visuospatial Short-Term Memory

Despite the abundant literature on visuospatial short-term memory, researchers have devoted little attention to strategic processes: What procedures do subjects implement to memorize visuospatial material? Evidence for various strategies exists, but it is spread across a variety of fields. This integrative review of the literature brings together scattered evidence to provide an overview of strategic processes in visuospatial memory tasks. The diversity of strategies and their proposed operating mechanisms are reviewed and discussed. The evidence leads to proposing seven broad strategic processes used in visuospatial short-term memory, each with multiple variants. Strategies can vary across individuals, but the same subjects also appear to use multiple strategies depending on the perceptual features of to-be-remembered displays. These results point to a view of visuospatial strategies as a functional library of facilitatory processes on which subjects can draw to support visuospatial short-term memory performance. Implications are discussed for the difference between visual and spatial tasks, for the appropriate measurement of strategic behaviors, and for the interpretation of performance in visuospatial memory tasks.

On-item fixations during serial encoding do not affect spatial working memory

Ample evidence suggests that there is overlap between the eye-movement system and spatial working memory. Such overlapping structures or capacities may result in interference on the one hand and beneficial support on the other. We investigated eye-movement control during encoding of verbal or spatial information, keeping the display the same between tasks. Saccades to to-be-encoded items were scarce during spatial encoding in comparison with verbal encoding. However, despite replicating this difference across different tasks (serial, free recall) and presentation modalities (simultaneous, sequential presentation), we found no relation between item fixations and memory performance-that is, no costs or benefits. Inducing a change from covert to overt encoding did not affect spatial memory performance as well. In contrast, regressive fixations on prior items, that were no longer on the screen, were associated with increased spatial memory performance. Regressions occurred mainly at the end of the encoding period and were targeted at the first presented item. Our results suggest a dissociation between two types of fixations that accompany serial spatial memory: On-item fixations are epiphenomenal; regressions indicate rehearsal or output preparation.

Recalling what was where when seeing nothing there

So-called "looks-at-nothing" have previously been used to show that recalling what also elicits the recall of where this was. Here, we present evidence from an eye-tracking study which shows that disrupting looks to "there" does not disrupt recalling what was there, nor do (anticipatory) looks to "there" facilitate recalling what was there. Therefore, our results suggest that recalling where does not recall what.

The reliability of continuous brain responses during naturalistic listening to music

Low-level (timbral) and high-level (tonal and rhythmical) musical features during continuous listening to music, studied by functional magnetic resonance imaging (fMRI), have been shown to elicit large-scale responses in cognitive, motor, and limbic brain networks. Using a similar methodological approach and a similar group of participants, we aimed to study the replicability of previous findings. Participants' fMRI responses during continuous listening of a tango Nuevo piece were correlated voxelwise against the time series of a set of perceptually validated musical features computationally extracted from the music. The replicability of previous results and the present study was assessed by two approaches: (a) correlating the respective activation maps, and (b) computing the overlap of active voxels between datasets at variable levels of ranked significance. Activity elicited by timbral features was better replicable than activity elicited by tonal and rhythmical ones. These results indicate more reliable processing mechanisms for low-level musical features as compared to more high-level features. The processing of such high-level features is probably more sensitive to the state and traits of the listeners, as well as of their background in music.