Image memorability is predicted by discriminability and similarity in different stages of a convolutional neural network

Memorability shapes perceived time (and vice versa)

Visual stimuli are known to vary in their perceived duration. Some visual stimuli are also known to linger for longer in memory. Yet, whether these two features of visual processing are linked is unknown. Despite early assumptions that time is an extracted or higher-order feature of perception, more recent work over the past two decades has demonstrated that timing may be instantiated within sensory modality circuits. A primary location for many of these studies is the visual system, where duration-sensitive responses have been demonstrated. Furthermore, visual stimulus features have been observed to shift perceived duration. These findings suggest that visual circuits mediate or construct perceived time. Here we present evidence across a series of experiments that perceived time is affected by the image properties of scene size, clutter and memorability. More specifically, we observe that scene size and memorability dilate time, whereas clutter contracts it. Furthermore, the durations of more memorable images are also perceived more precisely. Conversely, the longer the perceived duration of an image, the more memorable it is. To explain these findings, we applied a recurrent convolutional neural network model of the ventral visual system, in which images are progressively processed over time. We find that more memorable images are processed faster, and that this increase in processing speed predicts both the lengthening and the increased precision of perceived durations. These findings provide evidence for a link between image features, time perception and memory that can be further explored with models of visual processing.

Context reinstatement requires a schema relevant virtual environment to benefit object recall

How does our environment impact what we will later remember? Early work in real-world environments suggested that having matching encoding/retrieval contexts improves memory. However, some laboratory-based studies have not replicated this advantageous context-dependent memory effect. Using virtual reality methods, we find support for context-dependent memory effects and examine an influence of memory schema and dynamic environments. Participants (N = 240) remembered more objects when in the same virtual environment (context) as during encoding. This traded-off with falsely "recognizing" more similar lures. Experimentally manipulating the virtual objects and environments revealed that a congruent object/environment schema aids recall (but not recognition), though a dynamic background does not. These findings further our understanding of when and how context affects our memory through a naturalistic approach to studying such effects.

Differential effects of intrinsic properties of natural scenes and interference mechanisms on recognition processes in long-term visual memory

Humans display remarkable long-term visual memory (LTVM) processes. Even though images may be intrinsically memorable, the fidelity of their visual representations, and consequently the likelihood of successfully retrieving them, hinges on their similarity when concurrently held in LTVM. In this debate, it is still unclear whether intrinsic features of images (perceptual and semantic) may be mediated by mechanisms of interference generated at encoding, or during retrieval, and how these factors impinge on recognition processes. In the current study, participants (32) studied a stream of 120 natural scenes from 8 semantic categories, which varied in frequencies (4, 8, 16 or 32 exemplars per category) to generate different levels of category interference, in preparation for a recognition test. Then they were asked to indicate which of two images, presented side by side (i.e. two-alternative forced-choice), they remembered. The two images belonged to the same semantic category but varied in their perceptual similarity (similar or dissimilar). Participants also expressed their confidence (sure/not sure) about their recognition response, enabling us to tap into their metacognitive efficacy (meta-d'). Additionally, we extracted the activation of perceptual and semantic features in images (i.e. their informational richness) through deep neural network modelling and examined their impact on recognition processes. Corroborating previous literature, we found that category interference and perceptual similarity negatively impact recognition processes, as well as response times and metacognitive efficacy. Moreover, images semantically rich were less likely remembered, an effect that trumped a positive memorability boost coming from perceptual information. Critically, we did not observe any significant interaction between intrinsic features of images and interference generated either at encoding or during retrieval. All in all, our study calls for a more integrative understanding of the representational dynamics during encoding and recognition enabling us to form, maintain and access visual information.

Symbol superiority: Why $ is better remembered than 'dollar'

Memory typically is better for information presented in picture format than in word format. Dual-coding theory (Paivio, 1969) proposes that this is because pictures are spontaneously labelled, leading to the creation of two representational codes-image and verbal-whereas words often lead to only a single (verbal) code. With this perspective as motivation, the present investigation asked whether common graphic symbols (e.g.,!@#$%&) are afforded primarily verbal coding, akin to words, or whether they also invoke visual imagery, as do pictures. Across four experiments, participants were presented at study with graphic symbols or words (e.g., $ or 'dollar'). In Experiment 1, memory was assessed using free recall; in Experiment 2, memory was assessed using old-new recognition. In Experiment 3, the word set was restricted to a single category. In Experiment 4, memory for graphic symbols, pictures, and words was directly compared. All four experiments demonstrated a memory benefit for symbols relative to words. In a fifth experiment, machine learning estimations of inherent stimulus memorability were found to predict memory performance in the earlier experiments. This study is the first to present evidence that, like pictures, graphic symbols are better remembered than words, in line with dual-coding theory and with a distinctiveness account. We reason that symbols offer a visual referent for abstract concepts that are otherwise unlikely to be spontaneously imaged.

The features underlying the memorability of objects

What makes certain images more memorable than others? While much of memory research has focused on participant effects, recent studies using a stimulus-centric perspective have sparked debate on the determinants of memory, including the roles of semantic and visual features and whether the most prototypical or atypical items are best remembered. Prior studies have typically relied on constrained stimulus sets, limiting a generalized view of the features underlying what we remember. Here, we collected more than 1 million memory ratings for a naturalistic dataset of 26,107 object images designed to comprehensively sample concrete objects. We establish a model of object features that is predictive of image memorability and examined whether memorability could be accounted for by the typicality of the objects. We find that semantic features exert a stronger influence than perceptual features on what we remember and that the relationship between memorability and typicality is more complex than a simple positive or negative association alone.

Radiogenomics of C9orf72 Expansion Carriers Reveals Global Transposable Element Derepression and Enables Prediction of Thalamic Atrophy and Clinical Impairment

Hexanucleotide repeat expansion (HRE) within C9orf72 is the most common genetic cause of frontotemporal dementia (FTD). Thalamic atrophy occurs in both sporadic and familial FTD but is thought to distinctly affect HRE carriers. Separately, emerging evidence suggests widespread derepression of transposable elements (TEs) in the brain in several neurodegenerative diseases, including C9orf72 HRE-mediated FTD (C9-FTD). Whether TE activation can be measured in peripheral blood and how the reduction in peripheral C9orf72 expression observed in HRE carriers relates to atrophy and clinical impairment remain unknown. We used FreeSurfer software to assess the effects of C9orf72 HRE and clinical diagnosis (n = 78 individuals, male and female) on atrophy of thalamic nuclei. We also generated a novel, human, whole-blood RNA-sequencing dataset to determine the relationships among peripheral C9orf72 expression, TE activation, thalamic atrophy, and clinical severity (n = 114 individuals, male and female). We confirmed global thalamic atrophy and reduced C9orf72 expression in HRE carriers. Moreover, we identified disproportionate atrophy of the right mediodorsal lateral nucleus in HRE carriers and showed that C9orf72 expression associated with clinical severity, independent of thalamic atrophy. Strikingly, we found global peripheral activation of TEs, including the human endogenous LINE-1 element L1HS L1HS levels were associated with atrophy of multiple pulvinar nuclei, a thalamic region implicated in C9-FTD. Integration of peripheral transcriptomic and neuroimaging data from human HRE carriers revealed atrophy of specific thalamic nuclei, demonstrated that C9orf72 levels relate to clinical severity, and identified marked derepression of TEs, including L1HS, which predicted atrophy of FTD-relevant thalamic nuclei.SIGNIFICANCE STATEMENT Pathogenic repeat expansion in C9orf72 is the most frequent genetic cause of FTD and amyotrophic lateral sclerosis (ALS; C9-FTD/ALS). The clinical, neuroimaging, and pathologic features of C9-FTD/ALS are well characterized, whereas the intersections of transcriptomic dysregulation and brain structure remain largely unexplored. Herein, we used a novel radiogenomic approach to examine the relationship between peripheral blood transcriptomics and thalamic atrophy, a neuroimaging feature disproportionately impacted in C9-FTD/ALS. We confirmed reduction of C9orf72 in blood and found broad dysregulation of transposable elements-genetic elements typically repressed in the human genome-in symptomatic C9orf72 expansion carriers, which associated with atrophy of thalamic nuclei relevant to FTD. C9orf72 expression was also associated with clinical severity, suggesting that peripheral C9orf72 levels capture disease-relevant information.

Content-specific vulnerability of recent episodic memories in Alzheimer's disease

Endel Tulving's episodic memory framework emphasizes the multifaceted re-experiencing of personal events. Indeed, decades of research focused on the experiential nature of episodic memories, usually treating recent episodic memory as a coherent experiential quality. However, recent insights into the functional architecture of the medial temporal lobe show that different types of mnemonic information are segregated into distinct neural pathways in brain circuits empirically associated with episodic memory. Moreover, recent memories do not fade as a whole under conditions of progressive neurodegeneration in these brain circuits, notably in Alzheimer's disease. Instead, certain memory content seem particularly vulnerable from the moment of their encoding while other content can remain memorable consistently across individuals and contexts. We propose that these observations are related to the content-specific functional architecture of the medial temporal lobe and consequently to a content-specific impairment of memory at different stages of the neurodegeneration. To develop Endel Tulving's inspirational legacy further and to advance our understanding of how memory function is affected by neurodegenerative conditions such as Alzheimer's disease, we postulate that it is compelling to focus on the representational content of recent episodic memories.

•

The functional anatomy of episodic memory segregates different memory content.

•

Alzheimer's disease may cause content-specific loss of recent memories

•

Content-specific memorability across individuals changes with Alzheimer's disease.

•

Content-specific assessment could provide new insights into episodic memory in health and disease