Tracing the emergence of the memorability benefit

Not all objects are created equal: The object benefit in visual working memory is supported by greater recollection-like memory, but only for memorable objects

Visual working memory is thought to have a fixed capacity limit. However, recent evidence suggests that a greater number of real-world objects than simple features (i.e., colors) can be maintained, an effect termed the object benefit. Here, we examined whether this object benefit in visual working memory is due to qualitatively different memory processes employed for meaningful stimuli compared to simple features. In online samples of young adults, real-world objects were better remembered than colors, had higher measures of recollection, and showed a greater proportion of high-confidence responses (Exp. 1). Objects were also remembered better than their scrambled counterparts (Exp. 2), suggesting that this benefit is related to semantic information, rather than visual complexity. Critically, the specific objects that were likely to be remembered with high confidence were highly correlated across experiments, consistent with the idea that some objects are more memorable than others. Visual working memory performance for the least-memorable objects was worse than that of colors and scrambled objects. These findings suggest that real-world objects give rise to recollective, or at least high-confidence, responses at retrieval that may depend on activation of semantic features, but that this effect is limited to certain objects.

Memory augmentation with an adaptive cognitive interface

What we remember reflects both what we encounter, such as the intrinsic memorability of a stimulus, and our internal attentional state when we encounter that stimulus. Our memories are better for memorable images and images encountered in an engaged attentional state. Here, in an effort to modulate long-term memory performance, we manipulated these factors in combination by selecting the memorability of presented images contingent on individuals' natural fluctuations in sustained attention. Can image memorability and attentional state be strategically combined to improve memory? Are memorable images still well remembered during lapses in sustained attention, and conversely, can attentive states rescue memory performance for forgettable images? We designed a procedure to monitor participants' sustained attention dynamics on the fly via their response time fluctuations during a continuous performance task with trial-unique scene images. When high- or low-attentional states were detected, our algorithm triggered the presentation of high- or low-memorability images. Afterwards, participants completed a surprise recognition memory test for the attention-triggered images. Results demonstrated that memory performance for memorable items is not only resistant to lapses in sustained attention but also that memory cannot be further improved by encoding memorable items in engaged attentional states. On the other hand, memory performance for low-memorability images can be rescued by attentive encoding states. In sum, we show that both memorability and sustained attention can be leveraged in real time to maximize memory performance. This approach suggests that adaptive cognitive interfaces can tailor what information appears when to best support overall memory.

Memorability shapes perceived time (and vice versa)

Visual stimuli are known to vary in their perceived duration. Some visual stimuli are also known to linger for longer in memory. Yet, whether these two features of visual processing are linked is unknown. Despite early assumptions that time is an extracted or higher-order feature of perception, more recent work over the past two decades has demonstrated that timing may be instantiated within sensory modality circuits. A primary location for many of these studies is the visual system, where duration-sensitive responses have been demonstrated. Furthermore, visual stimulus features have been observed to shift perceived duration. These findings suggest that visual circuits mediate or construct perceived time. Here we present evidence across a series of experiments that perceived time is affected by the image properties of scene size, clutter and memorability. More specifically, we observe that scene size and memorability dilate time, whereas clutter contracts it. Furthermore, the durations of more memorable images are also perceived more precisely. Conversely, the longer the perceived duration of an image, the more memorable it is. To explain these findings, we applied a recurrent convolutional neural network model of the ventral visual system, in which images are progressively processed over time. We find that more memorable images are processed faster, and that this increase in processing speed predicts both the lengthening and the increased precision of perceived durations. These findings provide evidence for a link between image features, time perception and memory that can be further explored with models of visual processing.

Investigating the effects of perceptual complexity versus conceptual meaning on the object benefit in visual working memory

Previous research has demonstrated greater visual working memory (VWM) performance for real-world objects compared with simple features. Greater amplitudes of the contralateral delay activity (CDA)-a sustained event-related potential measured during the delay period of a VWM task-have also been noted for meaningful stimuli, despite being thought of as a neural marker of a fixed working memory capacity. The current study aimed to elucidate the factors underlying improved memory performance for real-world objects by isolating the relative contributions of perceptual complexity (i.e., number of visual features) and conceptual meaning (i.e., availability of semantic, meaningful features). Participants (N = 22) performed a lateralized VWM task to test their memory of intact real-world objects, scrambled real-world objects and colours. The CDA was measured during both encoding and WM retention intervals (600-1000 ms and 1300-1700 ms poststimulus onset, respectively), and behavioural performance was estimated by using d' (memory strength in a two-alternative forced choice task). Behavioural results revealed significantly better performance within-subjects for real-world objects relative to scrambled objects and colours, with no difference between colours and scrambled objects. The amplitude of the CDA was also largest for intact real-world objects, with no difference in magnitude for scrambled objects and colours, during working memory maintenance. However, during memory encoding, both the colours and intact real-world objects had significantly greater amplitudes than scrambled objects and were comparable in magnitude. Overall, findings suggest that conceptual meaning (semantics) supports the memory benefit for real-world objects.

Memory for artwork is predictable

Viewing art is often seen as a highly personal and subjective experience. However, are there universal factors that make a work of art memorable? We conducted three experiments, where we recorded online memory performance for 4,021 paintings from the Art Institute of Chicago, tested in-person memory after an unconstrained visit to the Art Institute, and obtained abstract attribute measures such as beauty and emotional valence for these pieces. Participants showed significant agreement in their memories both online and in-person, suggesting that pieces have an intrinsic "memorability" based solely on their visual properties that is predictive of memory in a naturalistic museum setting. Importantly, ResMem, a deep learning neural network designed to estimate image memorability, could significantly predict memory both online and in-person based on the images alone, and these predictions could not be explained by other low- or high-level attributes like color, content type, aesthetics, and emotion. A regression comprising ResMem and other stimulus factors could predict as much as half of the variance of in-person memory performance. Further, ResMem could predict the fame of a piece, despite having no cultural or historical knowledge. These results suggest that perceptual features of a painting play a major role in influencing its success, both in memory for a museum visit and in cultural memory over generations.