Recognition Memory Strength is Predicted by Pupillary Responses at Encoding While Fixation Patterns Distinguish Recollection from Familiarity

Pupil old/new effect as an objective measure of recognition memory: a meta-analysis of 17 eye-tracking experiments

Recognition memory, the ability to recognise previously encountered information, correlates with pupil diameter changes during the recognition period. This physiological response, known as the pupil old/new effect, generally reflects the variation in pupil dilation when encountering previously studied (old) stimuli compared to new stimuli. To develop a more precise understanding of the pupil old/new effect, we conducted a meta-analysis of 17 eye-tracking experiments (across 12 articles spanning from 2008 to 2023) involving 560 healthy adults with a mean age of 22.31 years. Analysis of publication bias showed a rather low risk of bias in the selected articles. The main meta-analysis revealed a significant and large pooled pupil old/new effect (Cohen's dz = 0.73, 95% CI [0.50, 0.95]). Further analysis of moderators showed that the number of participants included in the experiments and the criteria for selecting trials (only correct trials vs. all trials) had a significant impact on the meta-analytic results. In general, the analyses revealed a robust pupil old/new effect across all selected articles. This finding underscores its potential utility as a marker of recognition memory across different stimuli type, and various experimental designs.

Eyes on Memory: Pupillometry in Encoding and Retrieval

This review critically examines the contributions of pupillometry to memory research, primarily focusing on its enhancement of our understanding of memory encoding and retrieval mechanisms mainly investigated with the recognition memory paradigm. The evidence supports a close link between pupil response and memory formation, notably influenced by the type of novelty detected. This proposal reconciles inconsistencies in the literature regarding pupil response patterns that may predict successful memory formation, and highlights important implications for encoding mechanisms. The review also discusses the pupil old/new effect and its significance in the context of recollection and in reflecting brain signals related to familiarity or novelty detection. Additionally, the capacity of pupil response to serve as a true memory signal and to distinguish between true and false memories is evaluated. The evidence provides insights into the nature of false memories and offers a novel understanding of the cognitive mechanisms involved in memory distortions. When integrated with rigorous experimental design, pupillometry can significantly refine theoretical models of memory encoding and retrieval. Furthermore, combining pupillometry with neuroimaging and pharmacological interventions is identified as a promising direction for future research.

Glossiness perception and its pupillary response

Recent studies have revealed that pupillary response changes depend on perceptual factors such as subjective brightness caused by optical illusions and luminance. However, the manner in which the perceptual factor that is derived from the glossiness perception of object surfaces affects the pupillary response remains unclear. We investigated the relationship between the glossiness perception and pupillary response through a glossiness rating experiment that included recording the pupil diameter. We prepared general object images (original) and randomized images (shuffled) that comprised the same images with randomized small square regions as stimuli. The image features were controlled by matching the luminance histogram. The observers were asked to rate the perceived glossiness of the stimuli presented for 3,000 ms and the changes in their pupil diameters were recorded. Images with higher glossiness ratings constricted the pupil size more than those with lower glossiness ratings at the peak constriction of the pupillary responses during the stimulus duration. The linear mixed-effects model demonstrated that the glossiness rating, image category (original/shuffled), variance of the luminance histogram, and stimulus area were most effective in predicting the pupillary responses. These results suggest that the illusory brightness obtained by the image regions of high-glossiness objects, such as specular highlights, induce pupil constriction.

Differential contributions of the hippocampal dentate gyrus and CA1 subfield to mnemonic discrimination

Evidence suggests that individual hippocampal subfields are preferentially involved in various memory-related processes. Here, we demonstrated dissociations in these memory processes in two unique individuals with near-selective bilateral damage within the hippocampus, affecting the dentate gyrus (DG) in case BL and the cornu ammonis 1 (CA1) subfield in case BR. BL was impaired in discriminating highly similar objects in memory (i.e., mnemonic discrimination) but exhibited preserved overall recognition of studied objects, regardless of similarity. Conversely, BR demonstrated impaired general recognition. These results provide evidence for the DG in discrimination processes, likely related to underlying pattern separation computations, and the CA1 in retention/retrieval.

Eye tracking evidence for the reinstatement of emotionally negative and neutral memories

Recent eye tracking studies have linked gaze reinstatement-when eye movements from encoding are reinstated during retrieval-with memory performance. In this study, we investigated whether gaze reinstatement is influenced by the affective salience of information stored in memory, using an adaptation of the emotion-induced memory trade-off paradigm. Participants learned word-scene pairs, where scenes were composed of negative or neutral objects located on the left or right side of neutral backgrounds. This allowed us to measure gaze reinstatement during scene memory tests based on whether people looked at the side of the screen where the object had been located. Across two experiments, we behaviorally replicated the emotion-induced memory trade-off effect, in that negative object memory was better than neutral object memory at the expense of background memory. Furthermore, we found evidence that gaze reinstatement was related to recognition memory for the object and background scene components. This effect was generally comparable for negative and neutral memories, although the effects of valence varied somewhat between the two experiments. Together, these findings suggest that gaze reinstatement occurs independently of the processes contributing to the emotion-induced memory trade-off effect.

An examination of individual differences in levels of processing

The present study examined individual differences in levels of processing. Participants completed a cued recall task in which they made either rhyme or semantic judgements on pairs of items. Pupillary responses during encoding were recorded as a measure of the allocation of attentional effort and participants completed multiple measures of working and long-term memory. The results suggested levels of processing effect in both accuracy and pupillary responses with deeper levels of processing demonstrating higher accuracy and larger pupillary responses than shallower levels of processing. Most participants demonstrated levels of processing effect, but there was substantial variability in the size of the effect. Variation in levels of processing was positively related to individual differences in long-term memory and the magnitude of the pupillary levels of processing effect, but not working memory. These results suggest that some of the variation in levels of processing is likely due to individual differences in the allocation of attentional effort (particularly to items processed deeply) during encoding.

What does preferential viewing tell us about the neurobiology of recognition memory?

The two tests most widely used in nonhuman primates to assess the neurobiology of recognition memory produce conflicting results. Preferential viewing tests (e.g., visual paired comparison) produce robust impairments following hippocampal lesions, whereas matching tests (e.g., delayed nonmatching-to-sample) often show complete sparing. Here, we review the data, the proposed explanations for this discrepancy, and then critically evaluate those explanations. The most likely explanation is that preferential viewing tests are not a process-pure assessment of recognition memory, but also test elements of novelty-seeking, habituation, and motivation. These confounds likely explain the conflicting results. Thus, we propose that memory researchers should prefer explicit matching tests and readers interested in the neural substrates of recognition memory should give explicit matching tests greater interpretive weight.

From pre-processing to advanced dynamic modeling of pupil data

The pupil of the eye provides a rich source of information for cognitive scientists, as it can index a variety of bodily states (e.g., arousal, fatigue) and cognitive processes (e.g., attention, decision-making). As pupillometry becomes a more accessible and popular methodology, researchers have proposed a variety of techniques for analyzing pupil data. Here, we focus on time series-based, signal-to-signal approaches that enable one to relate dynamic changes in pupil size over time with dynamic changes in a stimulus time series, continuous behavioral outcome measures, or other participants' pupil traces. We first introduce pupillometry, its neural underpinnings, and the relation between pupil measurements and other oculomotor behaviors (e.g., blinks, saccades), to stress the importance of understanding what is being measured and what can be inferred from changes in pupillary activity. Next, we discuss possible pre-processing steps, and the contexts in which they may be necessary. Finally, we turn to signal-to-signal analytic techniques, including regression-based approaches, dynamic time-warping, phase clustering, detrended fluctuation analysis, and recurrence quantification analysis. Assumptions of these techniques, and examples of the scientific questions each can address, are outlined, with references to key papers and software packages. Additionally, we provide a detailed code tutorial that steps through the key examples and figures in this paper. Ultimately, we contend that the insights gained from pupillometry are constrained by the analysis techniques used, and that signal-to-signal approaches offer a means to generate novel scientific insights by taking into account understudied spectro-temporal relationships between the pupil signal and other signals of interest.

The effect of reward-induced arousal on the success and precision of episodic memory retrieval

Moment-to-moment fluctuations in arousal can have large effects on learning and memory. For example, when neutral items are predictive of a later reward, they are often remembered better than neutral items without a reward association. This reward anticipation manipulation is thought to induce a heightened state of arousal, resulting in stronger encoding. It is unclear, however, whether these arousal-induced effects on encoding are 'all-or-none', or whether encoding precision varies from trial to trial with degree of arousal. Here, we examined whether trial-to-trial variability in reward-related pupil-linked arousal might correspond to variability in participants' long-term memory encoding precision. We tested this using a location memory paradigm in which half of the to-be-encoded neutral items were linked to later monetary reward, while the other half had no reward association. After the encoding phase, we measured immediate item location memory on a continuous scale, allowing us to assess both memory success and memory precision. We found that pre-item baseline pupil size and pupil size during item encoding were not related to subsequent memory performance. In contrast, the anticipation of instrumental reward increased pupil size, and a smaller anticipatory increase in pupil size was linked to greater subsequent memory success but not memory precision.

Task-Dependent Visual Behavior in Immersive Environments: A Comparative Study of Free Exploration, Memory and Visual Search

Visual behavior depends on both bottom-up mechanisms, where gaze is driven by the visual conspicuity of the stimuli, and top-down mechanisms, guiding attention towards relevant areas based on the task or goal of the viewer. While this is well-known, visual attention models often focus on bottom-up mechanisms. Existing works have analyzed the effect of high-level cognitive tasks like memory or visual search on visual behavior; however, they have often done so with different stimuli, methodology, metrics and participants, which makes drawing conclusions and comparisons between tasks particularly difficult. In this work we present a systematic study of how different cognitive tasks affect visual behavior in a novel within-subjects design scheme. Participants performed free exploration, memory and visual search tasks in three different scenes while their eye and head movements were being recorded. We found significant, consistent differences between tasks in the distributions of fixations, saccades and head movements. Our findings can provide insights for practitioners and content creators designing task-oriented immersive applications.

Pupil response patterns distinguish true from false memories

Memory is reconstructive and error-prone, which make memory illusions very common in everyday life. However, studying memory illusions can provide valuable insights into how memory works. Pupil response has emerged, in recent years, as an indicator of memory encoding and retrieval, however its validity as a measure of memory success is debated. In this study, we explored whether pupil response patterns can differentiate true from false memories and whether variations in the temporal dynamics of pupil response can elucidate the mechanisms underlying false memory creation. The Deese-Roediger-McDermott (DRM) paradigm was employed to generate false memories in two separate experiments involving visual and auditory stimuli. Pupil responses effectively differentiated true from false memories based on variations in pupil amplitude at different temporal components. This discrimination remained consistent across both experiments, with slightly stronger effects in the auditory condition, aligning with the more pronounced false memory effects in this condition. Notably, differential pupil responses between true and false memories varied based on the type of memory involved at recognition. These findings provide valuable insights into the cognitive processes underlying memory distortions, with implications for theoretical frameworks and real-world contexts.

Effect of encoding variability on rejection of non-corresponding lures: Role of retrieval processes

The primary goal of the current study was to assess whether different retrieval processes moderated the effect of encoding variability on accurate performance on the A-B' test format of the forced-choice Mnemonic Similarity Task (MST). Young adults completed the A-B' test format of the forced-choice MST and determined whether their recognition memory judgment was based on recall-to-reject processing, recall-to-accept processing, familiarity, or guessing. When guesses were excluded from correct trials, fixation counts were higher to the A stimulus than the B stimulus at encoding. Contrary to our expectation that greater fixations to the A versus B stimulus at encoding would be associated with reliance on familiarity or recall-to-accept processing and greater fixations to the B versus A stimulus would be associated with recall-to-reject processing, no interaction between the stimulus and the subsequently reported memory process was observed.

Pupil Dilation During Encoding, But Not Type of Auditory Stimulation, Predicts Recognition Success in Face Memory

We encounter and process information from multiple sensory modalities in our daily lives, and research suggests that learning can be more efficient when contexts are multisensory. In this study, we were interested in whether face identity recognition memory might be improved in multisensory learning conditions, and to explore associated changes in pupil dilation during encoding and recognition. In two studies participants completed old/new face recognition tasks wherein visual face stimuli were presented in the context of sounds. Faces were learnt alongside no sound, low arousal sounds (Experiment 1), high arousal non-face relevant, or high arousal face relevant (Experiment 2) sounds. We predicted that the presence of sounds during encoding would improve later recognition accuracy, however, the results did not support this with no effect of sound condition on memory. Pupil dilation, however, was found to predict later successful recognition both at encoding and during recognition. While these results do not provide support to the notion that face learning is improved under multisensory conditions relative to unisensory conditions, they do suggest that pupillometry may be a useful tool to further explore face identity learning and recognition.

The effect of target-related and target-irrelevant novel stimuli on response behaviour

Novel events catch our attention, which can influence performance of a task. Whether this attentional capture by novelty benefits or impairs performance depends on several factors, such as the relevance of the stimulus, task requirements, and the timing of the event. Additionally, it has been argued that novel stimuli can hold intrinsic reward value, which may directly affect approach motivation, similar to positive valence stimuli. This link between novelty and approach/avoid behaviour has not been investigated directly. Here, we investigated whether stimulus novelty interacts with response behaviour in an approach/avoidance task, and whether these effects depend on the task relevance of novelty and stimulus timing. In experiment 1, participants gave an approach or avoid response dependent on a shape (diamond or square) presented at different stimulus onset asynchronies (SOA) following a novel or familiar scene (target-irrelevant novelty). In experiment 2, participants had to approach or avoid a novel or familiar image depending on the content (indoor/outdoor; target-related novelty). A shape was presented at different SOA. Results of a linear mixed model showed novelty-induced performance costs as demonstrated by longer RT and lower accuracy when novelty was target-relevant, likely due to attentional lingering at novel images. When images were target-irrelevant, approach but not avoid responses were faster for familiar versus novel images at 200 ms SOA only. Thus, novelty had a differentially pronounced detrimental effect on performance. These observations confirm that processing of novel stimuli generally depends on stimulus relevance, and tentatively suggests that differential processing of novel and familiar images is intensified by motivated approach behaviour.

Eye movements dissociate between perceiving, sensing, and unconscious change detection in scenes

Detecting visual changes can be based on perceiving, whereby one can identify a specific detail that has changed, on sensing, whereby one knows that there is a change but is unable to identify what changed, or on unconscious change detection, whereby one is unaware of any change even though the change influences one's behavior. Prior work has indicated that the processes underlying these different types of change detection are functionally and neurally distinct, but the attentional mechanisms that are related to these different types of change detection remain largely unknown. In the current experiment, we examined eye movements during a change detection task in globally manipulated scenes, and participants indicated their change detection confidence on a scale that allowed us to isolate perceiving, sensing, and unconscious change detection. For perceiving-based change detection, but not sensing-based or unconscious change detection, participants were more likely to preferentially revisit highly changed scene regions across the first and second presentation of the scene (i.e., resampling). This increase in resampling started within 250 ms of the test scene onset, suggesting that the effect began within the first two fixations. In addition, changed scenes were related to more clustered (i.e., less dispersed) eye movements than unchanged scenes, particularly when the subjects were highly confident that no change had occurred - providing evidence for change detection outside of conscious awareness. The results indicate that perceiving, sensing, and unconscious change detection responses are related to partially distinct patterns of eye movements.

The extent and specificity of visual exploration determines the formation of recollected memories in complex scenes

Our visual memories of complex scenes often appear as robust, detailed records of the past. Several studies have demonstrated that active exploration with eye movements improves recognition memory for scenes, but it is unclear whether this improvement is due to stronger feelings of familiarity or more detailed recollection. We related the extent and specificity of fixation patterns at encoding and retrieval to different recognition decisions in an incidental memory paradigm. After incidental encoding of 240 real-world scene photographs, participants (N = 44) answered a surprise memory test by reporting whether an image was new, remembered (indicating recollection), or just known to be old (indicating familiarity). To assess the specificity of their visual memories, we devised a novel report procedure in which participants selected the scene region that they specifically recollected, that appeared most familiar, or that was particularly new to them. At encoding, when considering the entire scene,subsequently recollected compared to familiar or forgotten scenes showed a larger number of fixations that were more broadly distributed, suggesting that more extensive visual exploration determines stronger and more detailed memories. However, when considering only the memory-relevant image areas, fixations were more dense and more clustered for subsequently recollected compared to subsequently familiar scenes. At retrieval, the extent of visual exploration was more restricted for recollected compared to new or forgotten scenes, with a smaller number of fixations. Importantly, fixation density and clustering was greater in memory-relevant areas for recollected versus familiar or falsely recognized images. Our findings suggest that more extensive visual exploration across the entire scene, with a subset of more focal and dense fixations in specific image areas, leads to increased potential for recollecting specific image aspects.

Arousal-modulated memory encoding and retrieval in adults with autism spectrum disorder

Recently, we have shown that pupil dilation during a recognition memory task can serve as an index of memory retrieval difficulties in autism. At the time of publication, we were unaware of specific data-analysis methods that can be used to shed further light on the origins of such memory related pupil dilation. Specifically, by distinguishing "tonic" from "phasic" changes in pupil dilation and considering their temporal progression, it is possible to draw inferences about the functional integrity of a locus coeruleus-norepinephrine system (LC-NE) that is known to play a key role in regulating memory encoding and retrieval processes. We therefore apply these analyses to our previously published eye-tracking data of adults with ASD (N = 24) and neurotypical development (TD, N = 30) during the recognition memory task. In this re-analysis, we related pupil dilation during encoding and retrieval to recognition accuracy in a per-trial analysis of linear mixed models. In ASD, we replicated attenuated recognition accuracy, which was accompanied by attenuated pupil dilation during encoding and retrieval. Group differences in pupil dilation during retrieval occurred late during the trial (after 1.75 s) and indicated an altered top-down processing like attenuated attribution of semantic salience in response to previously encoded stimuli. In addition, only in the ASD group were higher pupil dilation during encoding and lower pupil dilation during retrieval associated with decreased recognition accuracy. This supports altered modulation of memory encoding and retrieval in ASD, with LC-NE phasic activity as promising underlying mechanism. LAY SUMMARY: We investigated the changes of pupil size during memory testing in autism spectrum disorder. Adults with ASD remembered fewer items correctly than neurotypical individuals (TD). This reduced memory was related to increased pupillary responses at study and decreased pupil dilation at test only for adults with ASD indicating a different modulation of memory by the locus coeruleus.

A corollary discharge mediates saccade-related inhibition of single units in mnemonic structures of the human brain

Despite the critical link between visual exploration and memory, little is known about how neuronal activity in the human mesial temporal lobe (MTL) is modulated by saccades. Here, we characterize saccade-associated neuronal modulations, unit-by-unit, and contrast them to image onset and to occipital lobe neurons. We reveal evidence for a corollary discharge (CD)-like modulatory signal that accompanies saccades, inhibiting/exciting a unique population of broad-/narrow-spiking units, respectively, before and during saccades and with directional selectivity. These findings comport well with the timing, directional nature, and inhibitory circuit implementation of a CD. Additionally, by linking neuronal activity to event-related potentials (ERPs), which are directionally modulated following saccades, we recontextualize the ERP associated with saccades as a proxy for both the strength of inhibition and saccade direction, providing a mechanistic underpinning for the more commonly recorded saccade-related ERP in the human brain.

Eye-movement replay supports episodic remembering

When we bring to mind something we have seen before, our eyes spontaneously unfold in a sequential pattern strikingly similar to that made during the original encounter, even in the absence of supporting visual input. Oculomotor movements of the eye may then serve the opposite purpose of acquiring new visual information; they may serve as self-generated cues, pointing to stored memories. Over 50 years ago Donald Hebb, the forefather of cognitive neuroscience, posited that such a sequential replay of eye movements supports our ability to mentally recreate visuospatial relations during episodic remembering. However, direct evidence for this influential claim is lacking. Here we isolate the sequential properties of spontaneous eye movements during encoding and retrieval in a pure recall memory task and capture their encoding-retrieval overlap. Critically, we show that the fidelity with which a series of consecutive eye movements from initial encoding is sequentially retained during subsequent retrieval predicts the quality of the recalled memory. Our findings provide direct evidence that such scanpaths are replayed to assemble and reconstruct spatio-temporal relations as we remember and further suggest that distinct scanpath properties differentially contribute depending on the nature of the goal-relevant memory.

Damage to the human dentate gyrus impairs the perceptual discrimination of complex, novel objects

The hippocampus (HPC), and the dentate gyrus (DG) subregion in particular, is purported to be a pattern separator, orthogonally representing similar information so that distinct memories may be formed. The HPC may also be involved in complex perceptual discrimination. It is unclear if this role is limited to spatial/scene stimuli or extends to the discrimination of objects. Also unclear is whether the DG itself contributes to pattern separation beyond memory. BL, an individual with bilateral DG lesions, was previously shown to have poor discrimination of similar, everyday objects in memory. Here, we demonstrate that BL's deficit extends to complex perceptual discrimination of novel objects. Specifically, BL was presented with closely matched possible and impossible objects, which give rise to fundamentally different 3D perceptual representations despite being visually similar. BL performed significantly worse than controls when asked to select an odd object (e.g., impossible) amongst three identical counterpart objects (e.g., possible) presented at different rotations. His deficit was also evident in an atypical eye fixation pattern during this task. In contrast, BL's performance was indistinguishable from that of controls on other tasks involving the same objects, indicating that he could visually differentiate the object pairs, that he perceived the objects holistically in 3D, and that he has only a mild weakness in categorizing object possibility. Furthermore, his performance on standardized neuropsychological measures indicated intact mental rotation, visual-spatial attention, and working memory (visual and auditory). Collectively, these results provide evidence that the DG is necessary for complex perceptual discrimination of novel objects, indicating that the DG might function as a generic pattern separator of a wide range of stimuli within high-level perception, and that its role is not limited to memory.

Pupillary correlates of individual differences in long-term memory

The present study is the first to examine individual differences in long-term memory, arousal dysregulation, and intensity of attention within the same experiment. Participants (N = 106) completed 28 lists of an immediate free-recall task while their pupil diameter was recorded via an eye-tracker during the encoding period. Two main pupillary measures were extracted: intraindividual variability in pre-list pupil diameter and evoked pupillary responses during item encoding. Variability in pre-list pupil diameter served as a measure of arousal dysregulation, and evoked pupillary responses served as a measure of intensity of attention. Based on prior work, we hypothesized that there would be a positive association between intensity of attention and recall ability, and that there would be a negative association between arousal dysregulation and recall ability. Collectively these two measures accounted for 19% of interindividual variance in recall, with 5% attributable uniquely to intensity of attention and 12% attributable uniquely to arousal regulation. The findings demonstrate that there are sources of individual differences in long-term memory that can be revealed via pupillometry, notably the amount of effort deployed during item encoding and the degree to which people exhibit dysregulated arousal. Both findings are consistent with recent theorizing regarding the role of the locus coeruleus (LC)-norepinephrine (NE) system's role in goal-directed cognition. Specifically, the LC governs both moment-to-moment arousal and NE release to cortical regions subserving cognitive processing. Among people for whom this system operates most optimally, long-term memory retention is superior.

Individual pupil size changes as a robust indicator of cognitive familiarity differences

Cognitive psychology has a long history of using physiological measures, such as pupillometry. However, their susceptibility to confounds introduced by stimulus properties, such as color and luminance, has limited their application. Pupil size measurements, in particular, require sophisticated experimental designs to dissociate relatively small changes in pupil diameter due to cognitive responses from larger ones elicited by changes in stimulus properties or the experimental environment. Here, building on previous research, we present a pupillometry paradigm that adapts the pupil to stimulus properties during the baseline period without revealing stimulus meaning or context by using a pixel-scrambled image mask around an intact image. We demonstrate its robustness in the context of pupillary responses to branded product familiarity. Results show larger average and peak pupil dilation for passively viewed familiar product images and an extended later temporal component representing differences in familiarity across participants (starting around 1400 ms post-stimulus onset). These amplitude differences are present for almost all participants at the single-participant level, and vary somewhat by product category. However, amplitude differences were absent during the baseline period. These findings demonstrate that involuntary pupil size measurements combined with the presented paradigm are successful in dissociating cognitive effects of familiarity from physical stimulus confounds.

Memory after visual search: Overlapping phonology, shared meaning, and bilingual experience influence what we remember

How we remember the things that we see can be shaped by our prior experiences. Here, we examine how linguistic and sensory experiences interact to influence visual memory. Objects in a visual search that shared phonology (cat-cast) or semantics (dog-fox) with a target were later remembered better than unrelated items. Phonological overlap had a greater influence on memory when targets were cued by spoken words, while semantic overlap had a greater effect when targets were cued by characteristic sounds. The influence of overlap on memory varied as a function of individual differences in language experience -- greater bilingual experience was associated with decreased impact of overlap on memory. We conclude that phonological and semantic features of objects influence memory differently depending on individual differences in language experience, guiding not only what we initially look at, but also what we later remember.

Pupil responses to implied motion in figurative and abstract paintings

Motion can be perceived in static images, such as photos and figurative paintings, representing realistic subjects in motion, with or without directional information (e.g., motion blur or speed lines). Motion impression can be achieved even in non-realistic static images such as motion illusions and abstract paintings. It has been shown that visual motion processing affects the diameter of the pupil, responding differently to real, illusory, and implied motion in photographs (IM). It has been suggested that these different effects might be due to top-down modulations from different cortical areas underlying their processing. It is worthwhile to investigate pupillary response to figurative paintings, since they require an even higher level of interpretation than photos representing the same kind of subjects, given the complexity of cognitive processes involved in the aesthetic experience. Also, pupil responses to abstract paintings allows to study the effect of IM perception in representations devoid of real-life motion cues. We measured pupil responses to IM in figurative and abstract artworks depicting static and dynamic scenes, as rated by a large group of individuals not participating in the following experiment. Since the pupillary response is modulated by the subjective image interpretation, a motion rating test has been used to correct individual pupil data according to whether participants actually perceived the presence of motion in the paintings. Pupil responses to movies showing figurative and abstract subjects, and to motion illusions were also measured, to compare real and illusory motion with painted IM. Movies, both figurative and abstract, elicit the largest pupillary dilation of all static stimuli, whereas motion illusions cause the smallest pupil size, as previously shown. Interestingly, pupil responses to IM depend on the paintings' style. Figurative paintings depicting moving subjects cause more dilation than those representing static figures, and pupil size increases with the strength of IM, as already found with realistic photos. The opposite effect is obtained with abstract artworks. Abstract paintings depicting motion produce less dilation than those depicting stillness. In any case, these results reflect the individual subjective perception of dynamism, as the very same paintings can induce opposite responses in observer which interpreted it as static or dynamic. Overall, our data show that pupil size depends on high-level interpretation of motion in paintings, even when they do not represent real-world scenes. Our findings further suggest that the pupil is modulated by multiple top-down cortical mechanisms, involving the processing of motion, attention, memory, imagination, and other cognitive functions necessary for enjoying a complete aesthetic experience.

Gaze Behavior During Navigation and Visual Search of an Open-World Virtual Environment

Eye tracking has been an essential tool within the vision science community for many years. However, the majority of studies involving eye-tracking technology employ a relatively passive approach through the use of static imagery, prescribed motion, or video stimuli. This is in contrast to our everyday interaction with the natural world where we navigate our environment while actively seeking and using task-relevant visual information. For this reason, an increasing number of vision researchers are employing virtual environment platforms, which offer interactive, realistic visual environments while maintaining a substantial level of experimental control. Here, we recorded eye movement behavior while subjects freely navigated through a rich, open-world virtual environment. Within this environment, subjects completed a visual search task where they were asked to find and count occurrence of specific targets among numerous distractor items. We assigned each participant into one of four target conditions: Humvees, motorcycles, aircraft, or furniture. Our results show a statistically significant relationship between gaze behavior and target objects across Target Conditions with increased visual attention toward assigned targets. Specifically, we see an increase in the number of fixations and an increase in dwell time on target relative to distractor objects. In addition, we included a divided attention task to investigate how search changed with the addition of a secondary task. With increased cognitive load, subjects slowed their speed, decreased gaze on objects, and increased the number of objects scanned in the environment. Overall, our results confirm previous findings and support that complex virtual environments can be used for active visual search experimentation, maintaining a high level of precision in the quantification of gaze information and visual attention. This study contributes to our understanding of how individuals search for information in a naturalistic (open-world) virtual environment. Likewise, our paradigm provides an intriguing look into the heterogeneity of individual behaviors when completing an un-timed visual search task while actively navigating.

Oculomotor Behaviors and Integrative Memory Functions in the Alzheimer's Clinical Syndrome

BACKGROUND

Biological information drawn from eye-tracking metrics is providing evidence regarding drivers of cognitive decline in Alzheimer's disease. In particular, pupil size has proved useful to investigate cognitive performance during online activities.

OBJECTIVE

To investigate the oculomotor correlates of impaired performance of patients with mild Alzheimer's Clinical Syndrome (ACS) on a recently developed memory paradigm, namely the Short-Term Memory Binding Test (STMBT).

METHODS

We assessed a sample of eighteen healthy controls (HC) and eighteen patients with a diagnosis of mild ACS with the STMBT while we recorded their oculomotor behaviors using pupillometry and eye-tracking.

RESULTS

As expected, a group (healthy controls versus ACS) by condition (Unbound Colours versus Bound Colours) interaction was found whereby behavioral group differences were paramount in the Bound Colours condition. Healthy controls' pupils dilated significantly more in the Bound Colours than in the Unbound Colours condition, a discrepancy not observed in ACS patients. Furthermore, ROC analysis revealed the abnormal pupil behaviors distinguished ACS patients from healthy controls with values of sensitivity and specify of 100%, thus outperforming both recognition scores and gaze duration.

CONCLUSION

The biological correlates of Short-Term Memory Binding impairments appear to involve a network much wider than we have thought to date, which expands across cortical and subcortical structures. We discuss these findings focusing on their implications for our understanding of neurocognitive phenotypes in the preclinical stages of Alzheimer's disease and potential development of cognitive biomarkers that can support ongoing initiatives to prevent dementia.

Expectation-driven novelty effects in episodic memory

Novel and unexpected stimuli are often prioritised in memory, given their inherent salience. Nevertheless, not all forms of novelty show such an enhancement effect. Here, we discuss the role expectation plays in modulating the way novelty affects memory processes, circuits, and subsequent performance. We first review independent effects of expectation on memory, and then consider how different types of novelty are characterised by expectation. We argue that different types of novelty defined by expectation implicate differential neurotransmission in memory formation brain regions and may also result in the creation of different types of memory. Contextual novelty, which is unexpected by definition, is often associated with better recollection, supported by dopaminergic-hippocampal interactions. On the other hand, expected stimulus novelty is supported by engagement of medial temporal cortices, as well as the hippocampus, through cholinergic modulation. Furthermore, when expected stimulus novelty results in enhanced memory, it is predominantly driven by familiarity. The literature reviewed here highlights the complexity of novelty-sensitive memory systems, the distinction between types of novelty, and how they are differentially affected by expectancy.

Eye-movements reveal semantic interference effects during the encoding of naturalistic scenes in long-term memory

Similarity-based semantic interference (SI) hinders memory recognition. Within long-term visual memory paradigms, the more scenes (or objects) from the same semantic category are viewed, the harder it is to recognize each individual instance. A growing body of evidence shows that overt attention is intimately linked to memory. However, it is yet to be understood whether SI mediates overt attention during scene encoding, and so explain its detrimental impact on recognition memory. In the current experiment, participants watched 372 photographs belonging to different semantic categories (e.g., a kitchen) with different frequency (4, 20, 40 or 60 images), while being eye-tracked. After 10 minutes, they were presented with the same 372 photographs plus 372 new photographs and asked whether they recognized (or not) each photo (i.e., old/new paradigm). We found that the more the SI, the poorer the recognition performance, especially for old scenes of which memory representations existed. Scenes more widely explored were better recognized, but for increasing SI, participants focused on more local regions of the scene in search for its potentially distinctive details. Attending to the centre of the display, or to scene regions rich in low-level saliency was detrimental to recognition accuracy, and as SI increased participants were more likely to rely on visual saliency. The complexity of maintaining faithful memory representations for increasing SI also manifested in longer fixation durations; in fact, a more successful encoding was also associated with shorter fixations. Our study highlights the interdependence between attention and memory during high-level processing of semantic information.

Can pupillometry distinguish accurate from inaccurate familiarity?

Pupillometry, the measurement of pupil diameter, has become increasingly popular as a tool to investigate human memory. It has long been accepted that the pupil is able to distinguish familiar from completely novel items, a phenomenon known as "pupil old/new effect". Surprisingly, most pupillometric studies on the pupil old/new effect tend to disregard the possibility that the pupillary response to familiarity memory may not be entirely exclusive. Here, we investigated whether the pupillary response to old items correctly judged familiar (hits; accurate familiarity) can be differentiated from the pupillary response to new items wrongly judged familiar (false alarms; inaccurate familiarity). We found no evidence that the two processes could be isolated, as both accurate and inaccurate familiarity showed nearly identical mean and across-time pupillary responses. However, both familiarity hits and false alarms showed pupillary responses unequivocally distinct from those observed during either recollection or novelty detection, which suggests that the pupil measure of familiarity hits and/or false alarms was sufficiently sensitive. The pupillary response to false alarms may have been partially driven by perceptual fluency, since novel objects incorrectly judged to be old (i.e., false alarms) showed a higher degree of similarity to studied images than items correctly judged as novel (i.e., correct rejections). Thus, our results suggest that pupil dilation may not be able to distinguish accurate from inaccurate familiarity using standard recognition memory paradigms, and they also suggest that the pupillary response during familiarity feelings may also partly reflect perceptual fluency.

Pupillary response to representations of light in paintings

It is known that, although the level of light is the primary determinant of pupil size, cognitive factors can also affect pupil diameter. It has been demonstrated that photographs of the sun produce pupil constriction independently of their luminance and other low-level features, suggesting that high-level visual processing may also modulate pupil response.

Here, we measure pupil response to artistic paintings of the sun, moon, or containing a uniform lighting, that, being mediated by the artist's interpretation of reality and his technical rendering, require an even higher level of interpretation compared with photographs. We also study how chromatic content and spatial layout affect the results by presenting grey-scale and inverted versions of each painting. Finally, we assess directly with a categorization test how subjective image interpretation affects pupil response.

We find that paintings with the sun elicit a smaller pupil size than paintings with the moon, or paintings containing no visible light source. The effect produced by sun paintings is reduced by disrupting contextual information, such as by removing color or manipulating the relations between paintings features that make more difficult to identify the source of light. Finally, and more importantly, pupil diameter changes according to observers’ interpretation of the scene represented in the same stimulus.

In conclusion, results show that the subcortical pupillary response to light is modulated by subjective interpretation of luminous objects, suggesting the involvement of cortical systems in charge of cognitive processes, such as attention, object recognition, familiarity, memory, and imagination.

Encoding-linked pupil response is modulated by expected and unexpected novelty: Implications for memory formation and neurotransmission

Whether a novel stimulus is expected or unexpected may have implications for the kind of ensuing encoding and the type of subsequent memory. Pupil response was used in the present study to explore the way expected and unexpected stimuli are encoded and whether encoding-linked pupil response is modulated by expectation. Participants first established a contingency relationship between a series of symbols and the type of stimulus (man-made or natural) that followed each one. At encoding, some of the target stimuli violated the previously established relationship (i.e., unexpected), while the majority conformed to this relationship (i.e., expected). Expectation at encoding had opposite effects on familiarity and recollection, the two types of memory that support recognition, and modulated differently the way pupil response predicted subsequent memory. Encoding of unexpected novel stimuli was associated with increased pupil dilation as a predictor of subsequent memory type and strength. In contrast, encoding of expected novel stimuli was associated with decreased pupil response (constriction), which was predictive of subsequent memory type and strength. The findings support the close link between pupil response and memory formation, but critically indicate that this is modulated by the type of novelty as defined by expectation. These novel findings have important implications for the encoding mechanisms involved when different types of novelty are detected and is proposed to indicate the operation of different neurotransmitters during memory formation.

Pupil Dilation as an Index of Verbal Fluency

Verbal fluency tasks are widely used as a neuropsychological test of language production. We assessed pupil dilation during a verbal fluency task and during a control task. On the verbal fluency task, we asked 45 healthy participants (mean age = 23.55 years) to generate as many words as possible beginning with the letter “P,” whereas on the control task we asked them to count aloud. In both tasks we recorded pupil dilation with eye-tracking glasses. Results demonstrated that, compared with counting, verbal fluency resulted in a larger pupil dilation. The larger pupil dilation observed during verbal fluency compared with counting can be attributed to the cognitive load of verbal fluency, which involves both linguistic processing and executive function. By highlighting how verbal fluency can increase pupil dilation, our findings pave the way for the physiological assessment of verbal processing in healthy and pathological populations.

The link between detail generation and eye movements when encoding and retrieving complex images

Examining eye movement patterns when encoding and retrieving visually complex memories is useful to understand the link between visuo-perceptual processes and how associated details are represented within these memories. Here, we used images of real-world scenes (e.g., a couple grocery shopping) to examine how encoding and retrieval eye movements are linked to the details used to describe complex images during these two phases of memory. Given that memories are often elaborated upon during retrieval, we also examined whether eye-movements at retrieval related to details that were the same as those described when encoding the image (reinstated details) as well as details about the image event that were not initially described at encoding (newly generated details). Testing young healthy participants, we found that retrieval eye movements, specifically eye fixation rate, predicted reinstated details, but not newly generated details. This suggests that visuo-perceptual processes are preferentially engaged at retrieval to reactivate perceived information. At encoding, we found a relationship between eye movements and detail generation that changed over time. This relationship was positive early on in the encoding phase but changed to a negative relationship later in the phase, indicating that a unique relationship exists between activating visuo-perceptual processes during early encoding versus late encoding. Overall, our results provide new insights into how visuo-perceptual processes contribute to different components of complex memory.

The spatial distribution of attention predicts familiarity strength during encoding and retrieval

The memories we form are determined by what we attend to, and conversely, what we attend to is influenced by our memory for past experiences. Although we know that shifts of attention via eye movements are related to memory during encoding and retrieval, the role of specific memory processes in this relationship is unclear. There is evidence that attention may be especially important for some forms of memory (i.e., conscious recollection), and less so for others (i.e., familiarity-based recognition and unconscious influences of memory), but results are conflicting with respect to both the memory processes and eye movement patterns involved. To address this, we used a confidence-based method of isolating eye movement indices of spatial attention that are related to different memory processes (i.e., recollection, familiarity strength, and unconscious memory) during encoding and retrieval of real-world scenes. We also developed a new method of measuring the dispersion of eye movements, which proved to be more sensitive to memory processing than previously used measures. Specifically, in 2 studies, we found that familiarity strength-that is, changes in subjective reports of memory confidence-increased with (a) more dispersed patterns of viewing during encoding, (b) less dispersed viewing during retrieval, and (c) greater overlap in regions viewed between encoding and retrieval (i.e., resampling). Recollection was also related to these eye movements in a similar manner, though the associations with recollection were less consistent across experiments. Furthermore, we found no evidence for effects related to unconscious influences of memory. These findings indicate that attentional processes during viewing may not preferentially relate to recollection, and that the spatial distribution of eye movements is directly related to familiarity-based memory during encoding and retrieval. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Eye Movements of Spatial Working Memory Encoding in Children with and without Autism: Chunking Processing and Reference Preference

Individuals with autism spectrum disorder (ASD) experience spatial working memory deficits and show different encoding mechanisms from typical developing (TD) peers. To effectively describe the encoding strategies of those with ASD and highlight their characteristics in cognitive processing, we adopted improved change detection tasks and added eye-movement indicators to investigate the chunking function and reference preference of children with and without ASD. The current study included 20 participants with ASD aged 8-16 and 20 TD children matched for age, gender, and intelligence. Experiment 1 used high/low-structured change detection tasks, and eye-movement indexes were recorded as they memorized the locations of the items to investigate spatial chunking strategies. In Experiment 2, changes in eye movement patterns were observed by adding a frame of reference. The results suggested different encoding strategies in ASD and TD individuals. The ASD group showed local processing bias and had difficulty adopting chunking strategies in spatial working memory. Eye-movement analysis suggested that they rarely showed integrated information processing tendency observed in TD children. Moreover, as a compensatory processing, they were more likely to use the frame of reference. In this study, we compared the spatial chunking strategies and reference preference of children with and without ASD, and eye-movement analysis was used to investigate the processing mechanism. These findings are significant for research on cognitive characteristics of ASD and provide a new focus for working memory training in children with ASD. LAY SUMMARY: The current study suggests that children with autism spectrum disorder are poorer at organizing items into chunks in spatial working memory, but rely more on reference frames. If the purpose of location memory is to strengthen the adaptability of children with autism, it should provide them with more clues or references. If it is for the purpose of intervention such as cognitive training, it should guide them to integrate information to improve the basic cognitive processing efficiency. Autism Res 2021, 14: 897-910. © 2020 International Society for Autism Research and Wiley Periodicals LLC.

Encoding dynamics in free recall: Examining attention allocation with pupillometry

In four experiments pupillary responses were used to examine attention allocation and encoding dynamics in free recall. In Experiment 1, pupillary responses increased (and then decreased) across serial position suggesting that attention was increasingly allocated to items during learning until working memory was overloaded. In Experiment 2, manipulating presentation duration resulted in larger and more sustained pupillary responses with increased presentation duration, suggesting that participants were likely engaging in more elaborative and attention-demanding processes. In Experiment 3a, manipulating list-length resulted in decreased pupillary responses across serial position suggesting that participants were prioritizing early list items and less attention was allocated to later items. In Experiment 3b, when list-length was known, pupillary responses in the long-list length condition tended to decrease across serial position whereas pupillary responses in the short list-length condition tended to increase and decrease across serial positon. These results suggest that participants flexibly allocate attention to items during encoding depending on the nature of the task and the types of processes that are engaged in. These results further suggest the potential of utilizing pupillary responses to track attention allocation during learning.

Visual Exploration at Higher Fixation Frequency Increases Subsequent Memory Recall

Only a small proportion of what we see can later be recalled. Up to date it is unknown how far differences in visual exploration during encoding affect the strength of episodic memories. Here, we identified individual gaze characteristics by analyzing eye tracking data in a picture encoding task performed by 967 healthy subjects during fMRI. We found a positive correlation between fixation frequency during visual exploration and subsequent free recall performance. Brain imaging results showed a positive correlation of fixation frequency with activations in regions related to vision and memory, including the medial temporal lobe. To investigate if higher fixation frequency is causally linked to better memory, we experimentally manipulated visual exploration patterns in an independent population of 64 subjects. Doubling the number of fixations within a given exploration time increased subsequent free recall performance by 19%. Our findings provide evidence for a causal relationship between fixation frequency and episodic memory for visual information.

Pupil dilation as an indicator of future thinking

BACKGROUND

The pupil typically dilates in reaction to cognitive load. In this study, we, for the first time, investigated whether future thinking (i.e., the ability to generate hypothetical scenarios in the future) would result in pupil dilation.

METHODS

We recorded pupil dilation of participants during two conditions: past and future thinking. In past thinking, we invited participants to retrieve past personal events, while in future thinking, we invited them to imagine an event that may occur in the future.

RESULTS

Analysis demonstrated a larger pupil size during future than past thinking. Results also demonstrated longer retrieval time of future events compared with past ones, suggesting that future thinking perhaps requires more cognitive load than for past thinking. Interestingly, retrieval times during past and future thinking were positively correlated with pupil size.

DISCUSSION

The finding that future thinking activates pupil dilation could be due to the fact that while both past and future thinking require retrieving information from memory, future, but not past, thinking additionally requires the ability to recombine this information into novel scenarios.

Tracking visual search demands and memory load through pupil dilation

Continuously tracking cognitive demands via pupil dilation is a desirable goal for the monitoring and investigation of cognitive performance in applied settings where the exact time point of mental engagement in a task is often unknown. Yet, hitherto no experimentally validated algorithm exists for continuously estimating cognitive demands based on pupil size. Here, we evaluated the performance of a continuously operating algorithm that is agnostic of the onset of the stimuli and derives them by way of retrospectively modeling attentional pulses (i.e., onsets of processing). We compared the performance of this algorithm to a standard analysis of stimulus-locked pupil data. The pupil data were obtained while participants performed visual search (VS) and visual working memory (VWM) tasks with varying cognitive demands. In Experiment 1, VS was performed during the retention interval of the VWM task to assess interactive effects between search and memory load on pupil dilation. In Experiment 2, the tasks were performed separately. The results of the stimulus-locked pupil data demonstrated reliable increases in pupil dilation due to high VWM load. VS difficulty only affected pupil dilation when simultaneous memory demands were low. In the single task condition, increased VS difficulty resulted in increased pupil dilation. Importantly, online modeling of pupil responses was successful on three points. First, there was good correspondence between the modeled and stimulus locked pupil dilations. Second, stimulus onsets could be approximated from the derived attentional pulses to a reasonable extent. Third, cognitive demands could be classified above chance level from the modeled pupil traces in both tasks.

What eye movements reveal about strategy encoding of words in younger and older adults

The main goal of this study was to explore the organizational strategies used by younger and older adults when encoding words, using eye-tracking. Participants had to learn a set of organizable words and then a set of non-organizable words, each presented on a single display. Participants were then asked to recall the words of each set in the order in which they came to their mind. Hence, the participants' encoding strategies revealed by eye-tracking could be directly related to their subsequent memory performance. The results confirmed the detrimental impact of aging on memory and the weaker use of organizational strategies by older adults during the recall phase. The eye-tracking data showed that when they encode the words, older adults do not look at them for as long as younger adults, probably because of slower eye movements. They also revealed that compared to younger adults, older adults were much less able to adapt their word scanning strategy according to whether the words to encode were organizable or not. Finally, the relationships that were found between the recall scores and the eye-tracking data suggest that the eye movement pattern at learning can predict how people will recall the words.

How the size of the to-be-learned material influences the encoding and later retrieval of associative memories: A pupillometric assessment

Pupillometry have recently added valuable insights about the cognitive and possible neurobiological processes underlying episodic memory. Most of the studies, however, investigated recognition memory, which only partially relies on cue-driven recollection, the hallmark feature of episodic memory. Here we measured pupil size during a paired associate learning task, where participants encoded word-pairs, and after a short delay they took part in a cued recall. Importantly, we manipulated the size of the learning set: participants either learnt two, four or eight word-pairs in a row. As expected, increasing set size resulted in larger forgetting, assumingly as a consequence of weaker memory strength for the word-pairs. Our results show an important difference between pupil size changes observed during encoding and retrieval. During retrieval, the pupil instantly begun to dilate, as a sign of increased processing load accompanying the retrieval of the target memory. Importantly, large set size was also associated with larger pupil dilation during retrieval. This supports the notion that pupil dilation can be regarded as a marker of memory strength. In contrast, during encoding, pupil dilation decreased with increasing amount of encoded information, which might be due to the overuse of attentional resources. Furthermore, we also found that serial position during encoding modulated subsequent memory effects: for the first three serial positions, successful recall was predicted by larger pupil dilation during encoding, whereas such subsequent memory effect was absent for later serial positions. These results suggest that the amount of information independently modulates pupil dilation during encoding and retrieval, and support the assumption that pupillometric investigation of paired associate learning could be an informative way to investigate the cognitive and neurobiological processes of episodic memory.

J. Janssen S, Gallouj K, Lenoble Q. Autobiographical Memory Increases Pupil Dilation

BACKGROUND

Pupil activity has been widely considered as a "summed index" of physiological activities during cognitive processing.

METHODOLOGY

We investigated pupil dilation during retrieval of autobiographical memory and compared pupil diameter with a control condition in which participants had to count aloud. We also measured pupil diameters retrieval of free (i.e., first memory that comes to mind), positive, and negative memories (memories associated, respectively, with the words "happy" and "sad").

RESULTS

Analyses demonstrated larger pupil diameters during the free, positive, and negative autobiographical memory retrieval than during the control task. Analyses also demonstrated no significant differences in pupil diameters across the three autobiographical memory conditions.

CONCLUSION

These outcomes demonstrate that, compared with counting, autobiographical retrieval results in a larger pupil size. However, the emotional valence of memories yields non-significant effect on pupil diameters. Our findings demonstrate how autobiographical memory retrieval yields pupil dilation.

Gaze direction reveals implicit item and source memory in older adults

This study looked at eye movements in relation to source memory in older adults. Participants first studied images of common objects appearing in different quadrants of a screen. After a delay, they were shown cues one at a time presented in all four quadrants. Participants stated whether or not the cue had been seen before and in which location. Participants also rated level of confidence in their responses. In trials where participants either claimed they have not seen a previously presented cue or placed it in an incorrect location, they looked significantly more at the correct quadrant. The proportion of time looking at the correct quadrants during incorrect responses was not related to confidence ratings. These results suggest that eye gaze during the memory task does not reflect memory retrieval below the threshold of verbal report. They instead point to an implicit form of source memory in humans that is accessible to eye movements but not to verbal responses.

Rapid Brain Responses to Familiar vs. Unfamiliar Music - an EEG and Pupillometry study

Human listeners exhibit marked sensitivity to familiar music, perhaps most readily revealed by popular “name that tune” games, in which listeners often succeed in recognizing a familiar song based on extremely brief presentation. In this work, we used electroencephalography (EEG) and pupillometry to reveal the temporal signatures of the brain processes that allow differentiation between a familiar, well liked, and unfamiliar piece of music. In contrast to previous work, which has quantified gradual changes in pupil diameter (the so-called “pupil dilation response”), here we focus on the occurrence of pupil dilation events. This approach is substantially more sensitive in the temporal domain and allowed us to tap early activity with the putative salience network. Participants (N = 10) passively listened to snippets (750 ms) of a familiar, personally relevant and, an acoustically matched, unfamiliar song, presented in random order. A group of control participants (N = 12), who were unfamiliar with all of the songs, was also tested. We reveal a rapid differentiation between snippets from familiar and unfamiliar songs: Pupil responses showed greater dilation rate to familiar music from 100–300 ms post-stimulus-onset, consistent with a faster activation of the autonomic salience network. Brain responses measured with EEG showed a later differentiation between familiar and unfamiliar music from 350 ms post onset. Remarkably, the cluster pattern identified in the EEG response is very similar to that commonly found in the classic old/new memory retrieval paradigms, suggesting that the recognition of brief, randomly presented, music snippets, draws on similar processes.

Eye tracking - The overlooked method to measure cognition in neurodegeneration?

Eye tracking (ET) studies are becoming increasingly popular due to rapid methodological and technological advances as well as the development of cost efficient and portable eye trackers. Although historically ET has been mostly employed in psychophysics or developmental cognition studies, there is also promising scope to use ET for movement disorders and measuring cognitive processes in neurodegeneration. Particularly, ET can be a powerful tool for cognitive and neuropsychological assessments of patients with pathologies affecting motor and verbal abilities, as tasks can be adapted without requiring motor (except eye movements) or verbal responses. In this review, we will examine the existing evidence of ET methods in neurodegenerative conditions and its potential clinical impact for cognitive assessment. We highlight that current evidence for ET is mostly focused on diagnostics of cognitive impairments in neurodegenerative disorders, where it is debatable whether it has any more sensitivity or specificity than existing cognitive assessments. By contrast, there is currently a lack of ET studies in more advanced disease stages, when patients' motor and verbal functions can be significantly affected, and standard cognitive assessments are challenging or often not possible. We conclude that ET is a promising method not only for cognitive diagnostics but more importantly, for potential cognitive disease tracking in progressive neurodegenerative conditions.

Opposing Timing Constraints Severely Limit the Use of Pupillometry to Investigate Visual Statistical Learning

Majority of visual statistical learning (VSL) research uses only offline measures, collected after the familiarization phase (i.e., learning) has occurred. Offline measures have revealed a lot about the extent of statistical learning (SL) but less is known about the learning mechanisms that support VSL. Studies have shown that prediction can be a potential learning mechanism for VSL, but it is difficult to examine the role of prediction in VSL using offline measures alone. Pupil diameter is a promising online measure to index prediction in VSL because it can be collected during learning, requires no overt action or task and can be used in a wide-range of populations (e.g., infants and adults). Furthermore, pupil diameter has already been used to investigate processes that are part of prediction such as prediction error and updating. While the properties of pupil diameter have the potentially to powerfully expand studies in VSL, through a series of three experiments, we find that the two are not compatible with each other. Our results revealed that pupil diameter, used to index prediction, is not related to offline measures of learning. We also found that pupil differences that appear to be a result of prediction, are actually a result of where we chose to baseline instead. Ultimately, we conclude that the fast-paced nature of VSL paradigms make it incompatible with the slow nature of pupil change. Therefore, our findings suggest pupillometry should not be used to investigate learning mechanisms in fast-paced VSL tasks.

Action boosts episodic memory encoding in humans via engagement of a noradrenergic system

We are constantly interacting with our environment whilst we encode memories. However, how actions influence memory formation remains poorly understood. Goal-directed movement engages the locus coeruleus (LC), the main source of noradrenaline in the brain. Noradrenaline is also known to enhance episodic encoding, suggesting that action could improve memory via LC engagement. Here we demonstrate, across seven experiments, that action (Go-response) enhances episodic encoding for stimuli unrelated to the action itself, compared to action inhibition (NoGo). Functional magnetic resonance imaging, and pupil diameter as a proxy measure for LC-noradrenaline transmission, indicate increased encoding-related LC activity during action. A final experiment, replicated in two independent samples, confirmed a novel prediction derived from these data that emotionally aversive stimuli, which recruit the noradrenergic system, modulate the mnemonic advantage conferred by Go-responses relative to neutral stimuli. We therefore provide converging evidence that action boosts episodic memory encoding via a noradrenergic mechanism.