Models that allow us to perceive the world more accurately also allow us to remember past events more accurately via differentiation

Integrating word-form representations with global similarity computation in recognition memory

In recognition memory, retrieval is thought to occur by computing the global similarity of the probe to each of the studied items. However, to date, very few global similarity models have employed perceptual representations of words despite the fact that false recognition errors for perceptually similar words have consistently been observed. In this work, we integrate representations of letter strings from the reading literature with global similarity models. Specifically, we employed models of absolute letter position (slot codes and overlap models) and relative letter position (closed and open bigrams). Each of the representations was used to construct a global similarity model that made contact with responses and RTs at the individual word level using the linear ballistic accumulator (LBA) model (Brown & Heathcote Cognitive Psychology, 57 , 153-178, 2008). Relative position models were favored in three of the four datasets and parameter estimates suggested additional influence of the initial letters in the words. When semantic representations from the word2vec model were incorporated into the models, results indicated that orthographic representations were almost equally consequential as semantic representations in determining inter-item similarity and false recognition errors, which undermines previous suggestions that long-term memory is primarily driven by semantic representations. The model was able to modestly capture individual word variability in the false alarm rates, but there were limitations in capturing variability in the hit rates that suggest that the underlying representations require extension.

Representational formats of human memory traces

Neural representations are internal brain states that constitute the brain's model of the external world or some of its features. In the presence of sensory input, a representation may reflect various properties of this input. When perceptual information is no longer available, the brain can still activate representations of previously experienced episodes due to the formation of memory traces. In this review, we aim at characterizing the nature of neural memory representations and how they can be assessed with cognitive neuroscience methods, mainly focusing on neuroimaging. We discuss how multivariate analysis techniques such as representational similarity analysis (RSA) and deep neural networks (DNNs) can be leveraged to gain insights into the structure of neural representations and their different representational formats. We provide several examples of recent studies which demonstrate that we are able to not only measure memory representations using RSA but are also able to investigate their multiple formats using DNNs. We demonstrate that in addition to slow generalization during consolidation, memory representations are subject to semantization already during short-term memory, by revealing a shift from visual to semantic format. In addition to perceptual and conceptual formats, we describe the impact of affective evaluations as an additional dimension of episodic memories. Overall, these studies illustrate how the analysis of neural representations may help us gain a deeper understanding of the nature of human memory.

Retrieving effectively from source memory: Evidence for differentiation and local matching processes

The ability to distinguish between different explanations of human memory abilities continues to be the subject of many ongoing theoretical debates. These debates attempt to account for a growing corpus of empirical phenomena in item-memory judgments, which include the list strength effect, the strength-based mirror effect, and output interference. One of the main theoretical contenders is the Retrieving Effectively from Memory (REM) model. We show that REM, in its current form, has difficulties in accounting for source-memory judgments - a situation that calls for its revision. We propose an extended REM model that assumes a local-matching process for source judgments alongside source differentiation. We report a first evaluation of this model's predictions using three experiments in which we manipulated the relative source-memory strength of different lists of items. Analogous to item-memory judgments, we observed a null list strength effect and a strength-based mirror effect in the case of source memory. In a second evaluation, which relied on a novel experiment alongside two previously published datasets, we evaluated the model's predictions regarding the manifestation of output interference in item and lack of it in source memory judgments. Our results showed output interference severely affecting the accuracy of item-memory judgments but having a null or negligible impact when it comes to source-memory judgments. Altogether, these results support REM's core notion of differentiation (for both item and source information) as well as the concept of local matching proposed by the present extension.

Partial Repetition Costs are Reduced but not Eliminated with Practice

Often, we depart from an intended course of events to react to sudden situational demands (an intervening event) before resuming the originally planned action. Executing an action to an intervening event can be delayed if the features of this action plan partly overlap with an action plan retained in working memory (WM) compared to when they completely overlap or do not overlap. This delay is referred to as a partial repetition cost (PRC). PRCs are typically attributed to code confusion between action plans in WM. We tested this by training the component action plans extensively to reduce their reliance on WM. If PRCs are caused by code confusion within WM, then PRCs should be reduced and possibly eliminated with extensive practice. To test this, participants performed a partial repetition (PR) task after 0, 4 and 8.5 sessions of stimulus-response (S-R) training. In the PR task, participants saw two visual events. They retained an action to the first event while executing a speeded action to a second (intervening) event; afterwards, they executed the retained action. The two action plans either partly overlapped or did not overlap. Results showed that extensive (S-R and PR task) practice reduced but did not eliminate PRCs. A reduction in PRCs (code confusion) with practice is compatible with memory models that assume action events become more specific and less reliant on WM with practice. These findings merit expansions of PR tasks to other domains and broader conceptions of action plans that incorporate the formal structure of memory models.

Examining the transition of novel information toward familiarity

Throughout their lives, humans encounter multiple instances of new information that can be inconsistent with prior knowledge (novel). Over time, the once-novel information becomes integrated into their established knowledge base, shifting from novelty to familiarity. In this study, we investigated the processes by which the first steps of this transition take place. We hypothesized that the neural representations of initially novel items gradually change over the course of repeated presentations, expressing a shift toward familiarity. We further assumed that this shift could be traced by examining neural patterns using fMRI. In two experiments, while being scanned, participants read noun-adjective word pairs that were either consistent or inconsistent with their prior knowledge. Stimuli were repeated 3-6 times within the scans. Employing mass univariate and multivariate similarity analyses, we showed that the neural representations associated with the initial presentation of familiar versus novel objects differed in lateral frontal and temporal regions, the medial prefrontal cortex, and the medial temporal lobe. Importantly, the neural representations of novel stimuli gradually changed throughout repetitions until they became indistinguishable from their respective familiar items. We interpret these findings as indicating that an early phase of familiarization can be completed within a few repetitions. This initial familiarization can then serve as the prerequisite to the integration of novel items into existing knowledge. Future empirical and theoretical works can build on the current findings to develop a comprehensive model of the transition from novelty to familiarity.

Exemplar-based judgment or direct recall: On a problematic procedure for estimating parameters in exemplar models of quantitative judgment

Exemplar models are often used in research on multiple-cue judgments to describe the underlying process of participants' responses. In these experiments, participants are repeatedly presented with the same exemplars (e.g., poisonous bugs) and instructed to memorize these exemplars and their corresponding criterion values (e.g., the toxicity of a bug). We propose that there are two possible outcomes when participants judge one of the already learned exemplars in some later block of the experiment. They either have memorized the exemplar and their respective criterion value and are thus able to recall the exact value, or they have not learned the exemplar and thus have to judge its criterion value, as if it was a new stimulus. We argue that psychologically, the judgments of participants in a multiple-cue judgment experiment are a mixture of these two qualitatively distinct cognitive processes: judgment and recall. However, the cognitive modeling procedure usually applied does not make any distinction between these processes and the data generated by them. We investigated potential effects of disregarding the distinction between these two processes on the parameter recovery and the model fit of one exemplar model. We present results of a simulation as well as the reanalysis of five experimental data sets showing that the current combination of experimental design and modeling procedure can bias parameter estimates, impair their validity, and negatively affect the fit and predictive performance of the model. We also present a latent-mixture extension of the original model as a possible solution to these issues.

Location and temporal memory of objects declines in aged marmosets (Callithrix jacchus)

Episodic memory decline is an early marker of cognitive aging in human. Although controversial in animals and called “episodic-like memory”, several models have been successfully developed, however they rarely focused on ageing. While marmoset is an emerging primate model in aging science, episodic-like memory has never been tested in this species and importantly in aged marmosets. Here, we examined if the recall of the what-when and what-where building blocks of episodic-like memory declines in ageing marmosets. We developed a naturalistic approach using spontaneous exploration of real objects by young and old marmosets in the home cage. We implemented a three-trial task with 1 week inter-trial interval. Two different sets of identical objects were presented in sample trials 1 and 2, respectively. For the test trial, two objects from each set were presented in a former position and two in a new one. We quantified the exploratory behaviour and calculated discrimination indices in a cohort of 20 marmosets. Young animals presented a preserved memory for combined what-where, and what-when components of the experiment, which declined with aging. These findings lead one to expect episodic-like memory deficits in aged marmosets.

In search of the proximal cause of the animacy effect on memory: Attentional resource allocation and semantic representations

People recall and recognize animate words better than inanimate words, perhaps because memory systems were shaped by evolution to prioritize memory for predators, people, and food sources. Attentional paradigms show an animacy advantage that suggests that the animacy advantage in memory stems from a prioritization of animate items when allocating attentional resources during encoding. According to the attentional prioritization hypothesis, the animacy effect should be even larger when attention is divided during encoding. Alternatively, the animacy effect could be due to more controlled processing during encoding, and so should be reduced when attention is divided during encoding. We tested the attentional prioritization hypothesis and the controlled processing hypothesis by manipulating attention during encoding in free recall (Experiment 1) and recognition (Experiment 2) but failed to find interactions between word type and attentional load in either free recall or recognition, contrary to the predictions from both hypotheses. We then tested whether the semantic representations of animate and inanimate items differ in terms of number of semantic features, using existing recall data from an item-level megastudy by Lau, Goh, and Yap (Quarterly Journal of Experimental Psychology, 71 (10), 2207-2222, 2018). Animate items have more semantic features, which partially mediated the relationship between animacy status and recall.

Eye movements and the label feedback effect: Speaking modulates visual search via template integrity

The label-feedback hypothesis (Lupyan, 2012) proposes that language modulates low- and high-level visual processing, such as priming visual object perception. Lupyan and Swingley (2012) found that repeating target names facilitates visual search, resulting in shorter response times (RTs) and higher accuracy. In the present investigation, we conceptually replicated and extended their study, using additional control conditions and recording eye movements during search. Our goal was to evaluate whether self-directed speech influences target locating (i.e. attentional guidance) or object perception (i.e., distractor rejection and target appreciation). In three experiments, during object search, people spoke target names, nonwords, irrelevant (absent) object names, or irrelevant (present) object names (all within-participants). Experiments 1 and 2 examined search RTs and accuracy: Speaking target names improved performance, without differences among the remaining conditions. Experiment 3 incorporated eye-tracking: Gaze fixation patterns suggested that language does not affect attentional guidance, but instead affects both distractor rejection and target appreciation. When search trials were conditionalized according to distractor fixations, language effects became more orderly: Search was fastest while people spoke target names, followed in linear order by the nonword, distractor-absent, and distractor-present conditions. We suggest that language affects template maintenance during search, allowing fluent differentiation of targets and distractors. Materials, data, and analyses can be retrieved here: https://osf.io/z9ex2/.

Testing a Strategy-Disruption Account of the List-Strength Effect

Presenting items multiple times on a study list increases their memorability, a process known as item strengthening. The list-strength effect (LSE) refers to the finding that, compared to unstrengthened (pure) lists, lists for which a subset of the items have been strengthened produce enhanced memory for the strengthened items and depressed memory for the unstrengthened items. Although the LSE is found in free recall (Tulving & Hastie, 1972), it does not occur in recognition (Ratcliff et al., 1990). In free recall, the LSE in mixed lists is attributed to a sampling bias promoting priority recall of strong items and consequent output interference affecting weak items. We suggest that, in recognition, the disruption of this pattern through the randomization of test probes is responsible for the null LSE. We present several pilot experiments consistent with this account; however, the registered experiment, which had more statistical power, did not support this account.

How Do People Code Information in Working Memory When Items Share Features?

Abstract. A large literature suggests that the way we process information is influenced by the categories that we have learned. We examined whether, when we try to uniquely encode items in working memory, the information encoded depends on the other stimuli being simultaneously learned. Participants were required to memorize unknown aliens, presented one at the time, for immediate recognition of their features. Some aliens, called twins, were organized into pairs that shared every feature (nondiscriminative feature) except one (discriminative feature), while some other aliens, called hermits, did not share feature. We reasoned that if people develop unsupervised categories by creating a category for a pair of aliens, we should observe better feature identification performance for nondiscriminative features compared to hermit features, but not compared to discriminative features. On the contrary, if distinguishing features draw attention, we should observe better performance when a discriminative rather than nondiscriminative feature was probed. Overall, our results suggest that when items share features, people code items in working memory by focusing on similarities between items, establishing clusters of items in an unsupervised fashion not requiring feedback on cluster membership.

Modeling list-strength and spacing effects using version 3 of the retrieving effectively from memory (REM.3) model and its superimposition-of-similar-images assumption

Shiffrin and Steyvers (1997) introduced a model of recognition memory called retrieving effectively from memory (REM) and successfully applied it to a number of basic memory phenomena. REM incorporates differentiation, wherein item repetitions are accumulated in a single mnemonic trace rather than separate traces. This allows REM to account for several benchmark findings, including the null list-strength effect in recognition (Ratcliff, Clark, & Shiffrin, 1990). The original REM treated massed and spaced repetitions identically, which prevents it from predicting a mnemonic advantage for spaced over massed repetitions (i.e., the spacing effect). However, Shiffrin and Steyvers discussed the possibility that repetitions might be represented in a single trace only if the subject identifies that the repeated item was previously studied. It is quite plausible that subjects would notice repetitions more for massed than for spaced items. Here we show that incorporating this idea allows REM to predict three important findings in the recognition memory literature: (1) the spacing effect, (2) the finding of slightly positive list-strength effects with spaced repetitions, as opposed to massed repetitions or increased study time, and (3) list-strength effects that have been observed using very large strong-to-weak ratios (see Norman, 2002).

A systematic review of brain functional connectivity patterns involved in episodic and semantic memory

The study of functional connectivity and declarative memory has lately been focused on finding biomarkers of neuropsychological diseases. However, little is known about its patterns in healthy brains. Thus, in this systematic review we analyze and integrate the findings of 81 publications regarding functional connectivity (measured by fMRI during both task and resting-state) and semantic and episodic memory in healthy adults. Moreover, we discriminate and analyze the main areas and links found in specific memory phases (encoding, storage or retrieval) based on several criteria, such as time length, depth of processing, rewarding value of the information, vividness and amount or kind of details retrieved. There is a certain degree of overlap between the networks of episodic and semantic memory and between the encoding and retrieval stages. Although several differences are pointed out during the article, this calls to attention the need for further empirical studies that actively compare both types of memory, particularly using other baseline conditions apart from the traditional resting state. Indeed, the active involvement of the default mode network in both declarative memory and resting condition suggests the possibility that during rest there is an on-going memory processing. We find support for the 'attention to memory' hypothesis, the memory differentiation model and the appropriate transfer hypothesis, but some evidence is inconsistent with the traditional hub-and-spoke model.

Spaced Learning Enhances Episodic Memory by Increasing Neural Pattern Similarity Across Repetitions

Spaced learning has been shown consistently to benefit memory compared with massed learning, yet the neural representations and processes underlying the spacing effect are still poorly understood. In particular, two influential models (i.e., the encoding variability hypothesis and the study-phase retrieval hypothesis) could both model behavioral performance very well, but they make opposite hypotheses regarding the spacing effect's neural mechanisms. The present study attempted to provide empirical neural evidence to adjudicate these competing hypotheses. Using spatiotemporal pattern similarity (STPS) analysis of EEG data, this study investigated whether and how repetition lags (massed/short-spaced/long-spaced) modulated the STPS's contribution to episodic memory encoding in male and female human participants. The results revealed that greater item-specific STPS in the right frontal electrodes at 543-727 ms after stimulus onset was associated with better memory performance. More importantly, this STPS was larger under the spaced-learning condition than the massed-learning condition and partially mediated the spacing effect on memory performance. In addition, we found that massed learning was associated with stronger repetition suppression in the N400 component that reflected momentary retrieval strength, but reduced activity in the late positive component that was associated with memory retrieval. These results suggest that spaced learning improves long-term memory by increasing retrieval effort and enhancing the pattern reinstatement of prior neural representations, which may be achieved by reducing the momentary retrieval strength as the extended repetition lags might help to eliminate the residual representation in working memory.SIGNIFICANCE STATEMENT As one of the most ubiquitous and fundamental phenomena in the history of memory research, the spacing effect provides an important window into understanding how enduring memory is formed in the brain and how different practice strategies could modulate these mechanisms to affect memory performance. By leveraging the neural representational analysis on scalp EEG data, the current study provides the first empirical data to show that spaced learning enhances memory by improving the spatiotemporal similarity that occurs at a late time window. Our results support the study-phase retrieval hypothesis but not the encoding variability hypothesis and emphasize the role of neural pattern reinstatement in strengthening memory via repeated study.

Neural activity reveals interactions between episodic and semantic memory systems during retrieval

Whereas numerous findings support a distinction between episodic and semantic memory, it is now widely acknowledged that these two forms of memory interact during both encoding and retrieval. The precise nature of this interaction, however, remains poorly understood. To examine the role of semantic organization during episodic encoding and retrieval, we recorded intracranial encephalographic signals as 69 neurosurgical patients studied and subsequently recalled categorized and unrelated word lists. Applying multivariate classifiers to neural recordings, we were able to reliably predict encoding success, retrieval success, and temporal and categorical clustering during recall. By assessing how these classifiers generalized across list types, we identified specific retrieval processes that predicted recall of categorized lists and distinguished between recall transitions within and between category clusters. These results particularly implicate retrieval (rather than encoding) processes in the categorical organization of episodic memories. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

A strength-based mirror effect persists even when criterion shifts are unlikely

In single-item recognition, the strength-based mirror effect (SBME) is reliably obtained when encoding strength is manipulated between lists or participants. Debate surrounds the degree to which this effect is due to differentiation (e.g., Criss Journal of Memory and Language, 55, 461-478, 2006) or criterion shifts (e.g., Hicks & Starns Memory & Cognition, 42, 742-754, 2014). Problematically, differing underlying control processes may be equally capable of producing an SBME. The ability of criterion shifts to produce an SBME has been shown in prior work where differentiation was unlikely. The present work likewise produces an SBME under conditions where criterion shifts are unlikely. Specifically, we demonstrate that an SBME can be elicited without the typical number of trials needed to adjust one's decision criterion (Experiments 1, 2, and 5) and using encoding manipulations that do not explicitly alert participants that their memory quality has changed (Experiments 3 and 4). When taken in the context of the broader literature, these results demonstrate the need to prioritize memory models that can predict SBMEs via multiple underlying processes.

Why seemingly more difficult test conditions produce more accurate recognition of semantic prototype words: A recognition memory paradox?

Subjects studied Deese-Roediger-McDermott semantic-associate lists and took a recognition test. The makeup and number of test probes were manipulated. In Experiments 1 and 2A, one of three or all three distractors were semantically related to the list theme. In Experiment 2B, 6 or 30 related probes were used at test. Results showed that semantically related distractors and a longer list of test words both had a beneficial effect on the accurate discrimination of the prototype lures from the studied semantic associates and on the discrimination of studied from unstudied prototype words. These findings are inconsistent with predictions of memory interference and activation theories. We propose that the counterintuitive findings can be explained by the notion of old/new recognition as categorization learning and that relatedness and a larger number of test probes provide more accurate information about the prototype lure as a distractor, thereby improving its classification as a distractor.

Modeling the dynamics of recognition memory testing with an integrated model of retrieval and decision making

A robust finding in recognition memory is that performance declines monotonically across test trials. Despite the prevalence of this decline, there is a lack of consensus on the mechanism responsible. Three hypotheses have been put forward: (1) interference is caused by learning of test items (2) the test items cause a shift in the context representation used to cue memory and (3) participants change their speed-accuracy thresholds through the course of testing. We implemented all three possibilities in a combined model of recognition memory and decision making, which inherits the memory retrieval elements of the Osth and Dennis (2015) model and uses the diffusion decision model (DDM: Ratcliff, 1978) to generate choice and response times. We applied the model to four datasets that represent three challenges, the findings that: (1) the number of test items plays a larger role in determining performance than the number of studied items, (2) performance decreases less for strong items than weak items in pure lists but not in mixed lists, and (3) lexical decision trials interspersed between recognition test trials do not increase the rate at which performance declines. Analysis of the model's parameter estimates suggests that item interference plays a weak role in explaining the effects of recognition testing, while context drift plays a very large role. These results are consistent with prior work showing a weak role for item noise in recognition memory and that retrieval is a strong cause of context change in episodic memory.

Release from output interference in recognition memory: A test of the attention hypothesis

Retrieval results in both costs and benefits to episodic memory. Output interference (OI) refers to the finding that episodic memory accuracy decreases with increasing test trials. Release from OI is the restoration of original accuracy at some point during the test. For example, a release from OI in recognition memory testing occurs when the semantic similarity between stimuli decreases midway through testing, suggesting that item representations stored on early trials cause interference on tests occurring on later trials to the extent that the earlier items share features with the latter items. In two recognition memory experiments, we demonstrate release from OI for words and faces. We also test whether release from OI is the result of interference or is due to a boost in attention caused by reorienting to a novel stimulus type. A test for the foils presented during the initial test list supports the interference account of OI. Implications for models of memory are discussed.