A spatial modality effect in serial memory

Auditory and visual short-term memory: influence of material type, contour, and musical expertise

Short-term memory has mostly been investigated with verbal or visuospatial stimuli and less so with other categories of stimuli. Moreover, the influence of sensory modality has been explored almost solely in the verbal domain. The present study compared visual and auditory short-term memory for different types of materials, aiming to understand whether sensory modality and material type can influence short-term memory performance. Furthermore, we aimed to assess if music expertise can modulate memory performance, as previous research has reported better auditory memory (and to some extent, visual memory), and better auditory contour recognition for musicians than non-musicians. To do so, we adapted the same recognition paradigm (delayed-matching to sample) across different types of stimuli. In each trial, participants (musicians and non-musicians) were presented with two sequences of events, separated by a silent delay, and had to indicate whether the two sequences were identical or different. The performance was compared for auditory and visual materials belonging to three different categories: (1) verbal (i.e., syllables); (2) nonverbal (i.e., that could not be easily denominated) with contour (based on loudness or luminance variations); and (3) nonverbal without contour (pink noise sequences or kanji letters sequences). Contour and no-contour conditions referred to whether the sequence can entail (or not) a contour (i.e., a pattern of up and down changes) based on non-pitch features. Results revealed a selective advantage of musicians for auditory no-contour stimuli and for contour stimuli (both visual and auditory), suggesting that musical expertise is associated with specific short-term memory advantages in domains close to the trained domain, also extending cross-modally when stimuli have contour information. Moreover, our results suggest a role of encoding strategies (i.e., how the material is represented mentally during the task) for short-term-memory performance.

Commonalities of visual and auditory working memory in a spatial-updating task

Although visual and auditory inputs are initially processed in separate perception systems, studies have built on the idea that to maintain spatial information these modalities share a component of working memory. The present study used working memory navigation tasks to examine functional similarities and dissimilarities in the performance of updating tasks. Participants mentally updated the spatial location of a target in a virtual array in response to sequential pictorial and sonant directional cues before identifying the target's final location. We predicted that if working memory representations are modality-specific, mixed-modality cues would demonstrate a cost of modality switching relative to unimodal cues. The results indicate that updating performance using visual unimodal cues positively correlated with that using auditory unimodal cues. Task performance using unimodal cues was comparable to that using mixed modality cues. The results of a subsequent experiment involving updating of target traces were consistent with those of the preceding experiments and support the view of modality-nonspecific memory.

Familiarity with visual stimuli boosts recency bias in macaques

To probe how non-human primates (NHPs) decode temporal dynamic stimuli, we used a two-alternative forced choice task (2AFC), where the cue was dynamic: a movie snippet drawn from an animation that transforms one image into another. When the cue was drawn from either the beginning or end of the animation, thus heavily weighted towards one (the target) of both images (the choice pair), then primates performed at high levels of accuracy. For a subset of trials, however, the cue was ambiguous, drawn from the middle of the animation, containing information that could be associated to either image. Those trials, rewarded randomly and independent of choice, offered an opportunity to study the strategy the animals used trying to decode the cue. Despite being ambiguous, the primates exhibited a clear strategy, suggesting they were not aware that reward was given non-differentially. More specifically, they relied more on information provided at the end than at the beginning of those cues, consistent with the recency effect reported by numerous serial position studies. Interestingly and counterintuitively, this effect became stronger for sessions where the primates were already familiar with the stimuli. In other words, despite having rehearsed with the same stimuli in a previous session, the animals relied even more on a decision strategy that did not yield any benefits during a previous session. In the discussion section we speculate on what might cause this behavioral shift towards stronger bias, as well as why this behavior shows similarities with a repetition bias in humans known as the illusory truth effect.

Short-term memory stores organized by information domain

Vision and audition have complementary affinities, with vision excelling in spatial resolution and audition excelling in temporal resolution. Here, we investigated the relationships among the visual and auditory modalities and spatial and temporal short-term memory (STM) using change detection tasks. We created short sequences of visual or auditory items, such that each item within a sequence arose at a unique spatial location at a unique time. On each trial, two successive sequences were presented; subjects attended to either space (the sequence of locations) or time (the sequence of inter item intervals) and reported whether the patterns of locations or intervals were identical. Each subject completed blocks of unimodal trials (both sequences presented in the same modality) and crossmodal trials (Sequence 1 visual, Sequence 2 auditory, or vice versa) for both spatial and temporal tasks. We found a strong interaction between modality and task: Spatial performance was best on unimodal visual trials, whereas temporal performance was best on unimodal auditory trials. The order of modalities on crossmodal trials also mattered, suggesting that perceptual fidelity at encoding is critical to STM. Critically, no cost was attributable to crossmodal comparison: In both tasks, performance on crossmodal trials was as good as or better than on the weaker unimodal trials. STM representations of space and time can guide change detection in either the visual or the auditory modality, suggesting that the temporal or spatial organization of STM may supersede sensory-specific organization.

Increased distractibility in schizotypy: Independent of individual differences in working memory capacity?

Individuals with schizophrenia typically show increased levels of distractibility. This has been attributed to impaired working memory capacity (WMC), since lower WMC is typically associated with higher distractibility, and schizophrenia is typically associated with impoverished WMC. Here, participants performed verbal and spatial serial recall tasks that were accompanied by to-be-ignored speech tokens. For the few trials wherein one speech token was replaced with a different token, impairment was produced to task scores (a deviation effect). Participants subsequently completed a schizotypy questionnaire and a WMC measure. Higher schizotypy scores were associated with lower WMC (as measured with operation span, OSPAN), but WMC and schizotypy scores explained unique variance in relation to the mean magnitude of the deviation effect. These results suggest that schizotypy is associated with heightened domain-general distractibility, but that this is independent of its relationship with WMC.

Enumeration Produces Poor Primacy for Tactile Presentation Relative to Visual and Auditory Presentation as the Only Modality Effect

Methodological biases may help explain the modality effect, which is superior recall of auditory recency (end of list) items relative to visual recency items. In 1985 Nairne and McNabb used a counting procedure to reduce methodological biases, and they produced modality-like effects, such that recall of tactile recency items was superior to recall of visual recency items. The present study extended Nairne and McNabb's counting procedure and controlled several variables which may have enhanced recall of tactile end items or disrupted recall of visual end items in their study. Although the results of the present study indicated general serial position effects across tactile, visual, and auditory presentation modalities, the tactile condition showed lower recall for the initial items in the presentation list than the other two conditions. Moreover, recall of the final list item did not differ across the three presentation modalities; modality effects were not found. These results did not replicate the findings of Nairne and McNabb, or much of the past research showing superior recall of auditory recency items. Implications of these findings are discussed.