The dynamics of fidelity over the time course of long-term memory

From silos to synergy: Integrating approaches to investigate the role of prior knowledge and expectations on episodic memory

Significant progress in the investigation of how prior knowledge influences episodic memory has been made using three sometimes isolated (but not mutually exclusive) approaches: strictly adult behavioral investigations, computational models, and investigations into the development of the system. Here we point out that these approaches are complementary, each approach informs and is informed by the other. Thus, a natural next step for research is to combine all three approaches to further our understanding of the role of prior knowledge in episodic memory. Here we use studies of memory for expectation-congruent and incongruent information from each of these often disparate approaches to illustrate how combining approaches can be used to test and revise theories from the other. This domain is particularly advantageous because it highlights important features of more general memory processes, further differentiates models of memory, and can shed light on developmental change in the memory system. We then present a case study to illustrate the progress that can be made from integrating all three approaches and highlight the need for more endeavors in this vein. As a first step, we also propose a new computational model of memory that takes into account behavioral and developmental factors that can influence prior knowledge and episodic memory interactions. This integrated approach has great potential for offering novel insights into the relationship between prior knowledge and episodic memory, and cognition more broadly.

Disruption of Anterior Temporal Lobe Reduces Distortions in Memory From Category Knowledge

Memory retrieval does not provide a perfect recapitulation of past events, but instead an imperfect reconstruction of event-specific details and general knowledge. However, it remains unclear whether this reconstruction relies on mixtures of signals from different memory systems, including one supporting general knowledge. Here, we investigate whether the anterior temporal lobe (ATL) distorts new memories because of prior category knowledge. In this preregistered experiment (n = 36), participants encoded and retrieved image-location associations. Most images' locations were clustered according to their category, but some were in random locations. With this protocol, we previously demonstrated that randomly located images were retrieved closer to their category cluster relative to their encoded locations, suggesting an influence of category knowledge. We combined this procedure with TMS delivered to the left ATL before retrieval. We separately examined event-specific details (error) and category knowledge (bias) to identify distinct signals attributable to different memory systems. We found that TMS to ATL attenuated bias in location memory, but this effect was limited to exploratory analyses of atypical category members of animal categories. The magnitude of error was not impacted, suggesting that a memory's fidelity can be decoupled from its distortion by category knowledge. This raises the intriguing possibility that retrieval is jointly supported by separable memory systems.

A spatially continuous diffusion model of visual working memory

We present results from five visual working memory (VWM) experiments in which participants were briefly shown between 2 and 6 colored squares. They were then cued to recall the color of one of the squares and they responded by choosing the color on a continuous color wheel. The experiments provided response proportions and response time (RT) measures as a function of angle for the choices. Current VWM models for this task include discrete models that assume an item is either within working memory or not and resource models that assume that memory strength varies as a function of the number of items. Because these models do not include processes that allow them to account for RT data, we implemented them within the spatially continuous diffusion model (SCDM, Ratcliff, 2018) and use the experimental data to evaluate these combined models. In the SCDM, evidence retrieved from memory is represented as a spatially continuous normal distribution and this drives the decision process until a criterion (represented as a 1-D line) is reached, which produces a decision. Noise in the accumulation process is represented by continuous Gaussian process noise over spatial position. The models that fit best from the discrete and resource-based classes converged on a common model that had a guessing component and that allowed the height of the normal memory-strength distribution to vary with number of items. The guessing component was implemented as a regular decision process driven by a flat evidence distribution, a zero-drift process. The combination of choice and RT data allows models that were not identifiable based on choice data alone to be discriminated.

Episodic memory processes modulate how schema knowledge is used in spatial memory decisions

Schema knowledge can dramatically affect how we encode and retrieve memories. Current models propose that schema information is combined with episodic memory at retrieval to influence memory decisions, but it is not known how the strength or type of episodic memory (i.e., unconscious memory versus familiarity versus recollection) influences the extent to which schema information is incorporated into memory decisions. To address this question, we had participants search for target objects in semantically expected (i.e., congruent) locations or in unusual (i.e., incongruent) locations within scenes. In a subsequent test, participants indicated where in each scene the target had been located previously, then provided confidence-based recognition memory judgments that indexed recollection, familiarity strength, and unconscious memory for the scenes. In both an initial online study (n = 133) and replication (n = 59), target location recall was more accurate for targets that had been located in schema-congruent rather than incongruent locations; importantly, this effect was strongest for new scenes, decreased with unconscious memory, decreased further with familiarity strength, and was eliminated entirely for recollected scenes. Moreover, when participants recollected an incongruent scene but did not correctly remember the target location, they were still biased away from congruent regions-suggesting that detrimental schema bias was suppressed in the presence of recollection even when precise target location information was not remembered. The results indicate that episodic memory modulates how schemas are used: Schema knowledge contributes to spatial memory judgments primarily when episodic memory fails to provide precise information, and recollection can override schema bias completely.

The persistence of remote visual semantic memory following ocular blindness

Subjects with complete ocular blindness in both eyes provide a unique opportunity to study the long-term durability of visual semantic memory. In this cross-sectional study we recruited eleven subjects who had acquired blindness for between 1 and 36 years. For comparison, we studied four subjects with congenital blindness and seventeen age- and sex-matched sighted control subjects. We administered ten forced-choice questionnaires that probed one auditory category and four visual categories, namely object shape and size; object hue and lightness; word and letter shape; and the shape and features of famous faces. Subjects with congenital blindness performed worse than controls on all visual categories, but nevertheless performed better than chance on object structure or colour, suggesting that the answers to some questions about visual properties can be derived from haptic or non-visual semantic information. Subjects with acquired blindness performed similarly to controls on all categories except for facial memory, particularly for facial features. We conclude that there is a substantial "permastore" of visual semantic memory but that facial memories are less durable, perhaps indicating that they are either less over-learned or more dependent on visual representations than other forms of visual object information.

Why are the batteries in the microwave?: Use of semantic information under uncertainty in a search task

A major problem in human cognition is to understand how newly acquired information and long-standing beliefs about the environment combine to make decisions and plan behaviors. Over-dependence on long-standing beliefs may be a significant source of suboptimal decision-making in unusual circumstances. While the contribution of long-standing beliefs about the environment to search in real-world scenes is well-studied, less is known about how new evidence informs search decisions, and it is unclear whether the two sources of information are used together optimally to guide search. The present study expanded on the literature on semantic guidance in visual search by modeling a Bayesian ideal observer's use of long-standing semantic beliefs and recent experience in an active search task. The ability to adjust expectations to the task environment was simulated using the Bayesian ideal observer, and subjects' performance was compared to ideal observers that depended on prior knowledge and recent experience to varying degrees. Target locations were either congruent with scene semantics, incongruent with what would be expected from scene semantics, or random. Half of the subjects were able to learn to search for the target in incongruent locations over repeated experimental sessions when it was optimal to do so. These results suggest that searchers can learn to prioritize recent experience over knowledge of scenes in a near-optimal fashion when it is beneficial to do so, as long as the evidence from recent experience was learnable.

Hi-def memories of lo-def scenes

The study of visual memory is typically concerned with an image's content: How well, and with what precision, we can recall which objects, people, or features we have seen in the past. But images also vary in their quality: The same object or scene may appear in an image that is sharp and highly resolved, or it may appear in an image that is blurry and faded. How do we remember those properties? Here six experiments demonstrate a new phenomenon of "vividness extension": a tendency to (mis)remember images as though they are "enhanced" versions of themselves - that is, sharper and higher quality than they actually appeared at the time of encoding. Subjects briefly saw images of scenes that varied in how blurry they were, and then adjusted a new image to be as blurry as the original. Unlike an old photograph that fades and blurs, subjects misremembered scenes as more vivid (i.e., less blurry) than those scenes had actually appeared moments earlier. Follow-up experiments extended this phenomenon to saturation and pixelation - with subjects recalling scenes as more colorful and resolved - and ruled out various forms of response bias. We suggest that memory misrepresents the quality of what we have seen, such that the world is remembered as more vivid than it is.

Evaluating recall error in preschoolers: Category expectations influence episodic memory for color

Despite limited memory capacity, children are exceptional learners. How might children engage in meaningful learning despite limited memory systems? Past research suggests that adults integrate category knowledge and noisy episodic traces to aid recall when episodic memory is noisy or incomplete (e.g. Hemmer & Steyvers, 2009a,b). We suspect children utilize a similar process but integrate category and episodic traces in recall to a different degree. Here we conduct two experiments to empirically assess children's color category knowledge (Study 1) and recall of target hue values (Study 2). In Study 1, although children's generated hue values appear to be noisier than adults, we found no significant difference between children and adult's generated color category means (prototypes), suggesting that preschool-aged children's color categories are well established. In Study 2, we found that children's (like adult's) free recall of target hue values regressed towards color category means. We implemented three probabilistic memory models: one that combines category knowledge and specific target information (Integrative), a category only (Noisy Prototype) model, and a target only (Noisy Target) model to computationally evaluate recall performance. Consistent with previous studies with older children (Duffy, Huttenlocher, & Crawford, 2006), quantitative fits of the models to aggregate group-level data provided strong support for the Integrative process. However, at the individual subject level, a greater proportion of preschoolers' recall was better fit by a Prototype only model. Our results provide evidence that the integration of category knowledge in episodic memory comes online early and strongly. Implications for how the greater reliance on category knowledge by preschoolers relative to adults might track with developmental shifts in relational episodic memory are discussed.

Schematic memories develop quickly, but are not expressed unless necessary

Episodic memory retrieval is increasingly influenced by schematic information as memories mature, but it is unclear whether this is due to the slow formation of schemas over time, or the slow forgetting of the episodes. To address this, we separately probed memory for newly learned schemas as well as their influence on episodic memory decisions. In this experiment, participants encoded images from two categories, with the location of images in each category drawn from a different spatial distribution. They could thus learn schemas of category locations by encoding specific episodes. We found that images that were more consistent with these distributions were more precisely retrieved, and this schematic influence increased over time. However, memory for the schema distribution, measured using generalization to novel images, also became less precise over time. This incongruity suggests that schemas form rapidly, but their influence on episodic retrieval is dictated by the need to bolster fading memory representations.

Material constancy in perception and working memory

A key challenge for the visual system entails the extraction of constant properties of objects from sensory information that varies moment by moment due to changes in viewing conditions. Although successful performance in constancy tasks requires cooperation between perception and working memory, the function of the memory system has been under-represented in recent material perception literature. Here, we addressed the limits of material constancy by elucidating if and how working memory is involved in constancy tasks by using a variety of material stimuli, such as metals, glass, and translucent objects. We conducted experiments with a simultaneous and a successive matching-to-sample paradigm in which participants matched the perceived material properties of objects with or without a temporal delay under varying illumination contexts. The current study combined a detailed analysis of matching errors, data on the strategy use obtained via a self-report questionnaire, and the statistical image analysis of diagnostic image cues used for material discrimination. We found a comparable material constancy between simultaneous and successive matching conditions, and it was suggested that, in both matching conditions, participants used similar information processing strategies for the discrimination of materials. The study provides converging evidence on the critical role of working memory in material constancy, where working memory serves as a shared processing bottleneck that constrains both simultaneous and successive material constancy.

How Do People Code Information in Working Memory When Items Share Features?

Abstract. A large literature suggests that the way we process information is influenced by the categories that we have learned. We examined whether, when we try to uniquely encode items in working memory, the information encoded depends on the other stimuli being simultaneously learned. Participants were required to memorize unknown aliens, presented one at the time, for immediate recognition of their features. Some aliens, called twins, were organized into pairs that shared every feature (nondiscriminative feature) except one (discriminative feature), while some other aliens, called hermits, did not share feature. We reasoned that if people develop unsupervised categories by creating a category for a pair of aliens, we should observe better feature identification performance for nondiscriminative features compared to hermit features, but not compared to discriminative features. On the contrary, if distinguishing features draw attention, we should observe better performance when a discriminative rather than nondiscriminative feature was probed. Overall, our results suggest that when items share features, people code items in working memory by focusing on similarities between items, establishing clusters of items in an unsupervised fashion not requiring feedback on cluster membership.

Adaptive allocation of human visual working memory capacity during statistical and categorical learning

Human brains are finite, and thus have bounded capacity. An efficient strategy for a capacity-limited agent is to continuously adapt by dynamically reallocating capacity in a task-dependent manner. Here we study this strategy in the context of visual working memory (VWM). People use their VWM stores to remember visual information over seconds or minutes. However, their memory performances are often error-prone, presumably due to VWM capacity limits. We hypothesize that people attempt to be flexible and robust by strategically reallocating their limited VWM capacity based on two factors: (a) the statistical regularities (e.g., stimulus feature means and variances) of the to-be-remembered items, and (b) the requirements of the task that they are attempting to perform. The latter specifies, for example, which types of errors are costly versus irrelevant for task performance. These hypotheses are formalized within a normative computational modeling framework based on rate-distortion theory, an extension of conventional Bayesian approaches that uses information theory to study rate-limited (or capacity-limited) processes. Using images of plants that are naturalistic and precisely controlled, we carried out two sets of experiments. Experiment 1 found that when a stimulus dimension (the widths of plants' leaves) was assigned a distribution, subjects adapted their VWM performances based on this distribution. Experiment 2 found that when one stimulus dimension (e.g., leaf width) was relevant for distinguishing plant categories but another dimension (leaf angle) was irrelevant, subjects' responses in a memory task became relatively more sensitive to the relevant stimulus dimension. Together, these results illustrate the task-dependent robustness of VWM, thereby highlighting the dependence of memory on learning.

Working Memory Has Better Fidelity Than Long-Term Memory: The Fidelity Constraint Is Not a General Property of Memory After All

How detailed are long-term-memory representations compared with working memory representations? Recent research has found an equal fidelity bound for both memory systems, suggesting a novel general constraint on memory. Here, we assessed the replicability of this discovery. Participants (total N = 72) were presented with colored real-life objects and were asked to recall the colors using a continuous color wheel. Deviations from study colors were modeled to generate two estimates of color memory: the variability of remembered colors-fidelity-and the probability of forgetting the color. Estimating model parameters using both maximum-likelihood estimation and Bayesian hierarchical modeling, we found that working memory had better fidelity than long-term memory (Experiments 1 and 2). Furthermore, within each system, fidelity worsened as a function of time-correlated mechanisms (Experiments 2 and 3). We conclude that fidelity is subject to decline across and within memory systems. Thus, the justification for a general fidelity constraint in memory does not seem to be valid.

The interplay of language and visual perception in working memory

How do perception and language interact to form the representations that guide our thoughts and actions over the short-term? Here, we provide a first examination of this question by investigating the role of verbal labels in a continuous visual working memory (WM) task. Across four experiments, participants retained in memory the continuous color of a set of dots which were presented sequentially (Experiments 1-3) or simultaneously (Experiment 4). At test, they reproduced the colors of all dots using a color wheel. During stimulus presentation participants were required to either label the colors (color labeling) or to repeat "bababa" aloud (articulatory suppression), hence prompting or preventing verbal labeling, respectively. We tested four competing hypotheses of the labeling effect: (1) labeling generates a verbal representation that overshadows the visual representation; (2) labeling yields a verbal representation in addition to the visual one; (3) the labels function as a retrieval cue, adding distinctiveness to items in memory; and (4) labels activate visual categorical representations in long-term memory. Collectively, our experiments show that labeling does not overshadow the visual input; it augments it. Mixture modeling showed that labeling increased the quantity and quality of information in WM. Our findings are consistent with the hypothesis that labeling activates visual long-term categorical representations which help in reducing the noise in the internal representations of the visual stimuli in WM.