1
|
Wynn JS, Schacter DL. Eye movements reinstate remembered locations during episodic simulation. Cognition 2024; 248:105807. [PMID: 38688077 DOI: 10.1016/j.cognition.2024.105807] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Revised: 03/04/2024] [Accepted: 04/25/2024] [Indexed: 05/02/2024]
Abstract
Imagining the future, like recalling the past, relies on the ability to retrieve and imagine a spatial context. Research suggests that eye movements support this process by reactivating spatial contextual details from memory, a process termed gaze reinstatement. While gaze reinstatement has been linked to successful memory retrieval, it remains unclear whether it supports the related process of future simulation. In the present study, we recorded both eye movements and audio while participants described familiar locations from memory and subsequently imagined future events occurring in those locations while either freely moving their eyes or maintaining central fixation. Restricting viewing during simulation significantly reduced self-reported vividness ratings, supporting a critical role for eye movements in simulation. When viewing was unrestricted, participants spontaneously reinstated gaze patterns specific to the simulated location, replicating findings of gaze reinstatement during memory retrieval. Finally, gaze-based location reinstatement was predictive of simulation success, indexed by the number of internal (episodic) details produced, with both measures peaking early and co-varying over time. Together, these findings suggest that the same oculomotor processes that support episodic memory retrieval - that is, gaze-based reinstatement of spatial context - also support episodic simulation.
Collapse
Affiliation(s)
- Jordana S Wynn
- Department of Psychology, University of Victoria, Victoria, Canada.
| | | |
Collapse
|
2
|
El Haj M. When you look at your past: Eye movement during autobiographical retrieval. Conscious Cogn 2024; 118:103652. [PMID: 38301389 DOI: 10.1016/j.concog.2024.103652] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Revised: 01/19/2024] [Accepted: 01/22/2024] [Indexed: 02/03/2024]
Abstract
Until recently, little was known about whether or how autobiographical memory (i.e., memory of personal information) activates eye movement. This issue is now being addressed by several studies demonstrating not only how autobiographical memory activates eye movement, but also how eye movement influences the characteristics of autobiographical retrieval. This paper summarizes this research and presents a hypothesis according to which fixations and saccades during autobiographical retrieval mirror the construction of the visual image of the retrieved event. This hypothesis suggests that eye movements during autobiographical retrieval mirror the attempts of the visual system to generate and manipulate mental representations of autobiographical retrieval. It offers a theoretical framework for a burgeoning area of research that provides a rigorous behavioral evaluation of the phenomenological experience of memory.
Collapse
|
3
|
Nikolaev AR, Meghanathan RN, van Leeuwen C. Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates. Atten Percept Psychophys 2024:10.3758/s13414-023-02836-9. [PMID: 38169029 DOI: 10.3758/s13414-023-02836-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/17/2023] [Indexed: 01/05/2024]
Abstract
When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.
Collapse
Affiliation(s)
- Andrey R Nikolaev
- Department of Psychology, Lund University, Box 213, 22100, Lund, Sweden.
- Brain & Cognition Research Unit, KU Leuven-University of Leuven, Leuven, Belgium.
| | | | - Cees van Leeuwen
- Brain & Cognition Research Unit, KU Leuven-University of Leuven, Leuven, Belgium
- Center for Cognitive Science, Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau, Kaiserslautern, Germany
| |
Collapse
|
4
|
Gurtner LM, Bischof WF, Mast FW. Gaze Restriction and Reactivation of Place-bound Content Drive Eye Movements During Mental Imagery. J Cogn 2023; 6:51. [PMID: 37663138 PMCID: PMC10473167 DOI: 10.5334/joc.316] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Accepted: 08/09/2023] [Indexed: 09/05/2023] Open
Abstract
When we imagine a picture, we move our eyes even though the picture is physically not present. These eye movements provide information about the ongoing process of mental imagery. Eye movements unfold over time, and previous research has shown that the temporal gaze dynamics of eye movements in mental imagery have unique properties, which are unrelated to those in perception. In mental imagery, refixations of previously fixated locations happen more often and in a more systematic manner than in perception. The origin of these unique properties remains unclear. We tested how the temporal structure of eye movements is influenced by the complexity of the mental image. Participants briefly saw and then maintained a pattern stimulus, consisting of one (easy condition) to four black segments (most difficult condition). When maintaining a simple pattern in imagery, participants restricted their gaze to a narrow area, and for more complex stimuli, eye movements were more spread out to distant areas. At the same time, fewer refixations were made in imagery when the stimuli were complex. The results show that refixations depend on the imagined content. While fixations of stimulus-related areas reflect the so-called 'looking at nothing' effect, gaze restriction emphasizes differences between mental imagery and perception.
Collapse
Affiliation(s)
| | - Walter F. Bischof
- Department of Psychology, University of British Columbia, Vancouver BC, Canada
| | - Fred W. Mast
- Department of Psychology, University of Bern, Bern, Switzerland
| |
Collapse
|
5
|
Lucas HD, Daugherty AM, McAuley E, Kramer AF, Cohen NJ. Dynamic interactions between memory and viewing behaviors: Insights from dyadic modeling of eye movements. J Exp Psychol Hum Percept Perform 2023; 49:786-801. [PMID: 37166935 PMCID: PMC10621599 DOI: 10.1037/xhp0001123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
Humans use eye movements to build visual memories. We investigated how the contributions of specific viewing behaviors to memory formation evolve over individual study epochs. We used dyadic modeling to explain performance on a spatial reconstruction task based on interactions among two gaze measures: (a) the entropy of the scanpath and (b) the frequency of item-to-item gaze transitions. To measure these interactions, our hypothesized model included causal pathways by which early-trial viewing behaviors impacted subsequent memory via downstream effects on later viewing. We found that lower scanpath entropy throughout the trial predicted better memory performance. By contrast, the effect of item-to-item transition frequency changed from negative to positive as the trial progressed. The model also revealed multiple pathways by which early-trial viewing dynamically altered late-trial viewing, thereby impacting memory indirectly. Finally, individual differences in scores on an independent measure of memory ability were found to predict viewing effectiveness, and viewing behaviors partially mediated the relation between memory ability and reconstruction accuracy. In a second experiment, the model showed a good fit for an independent dataset. These results highlight the dynamic nature of memory formation and suggest that the order in which eye movements occur can critically determine their effectiveness. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Collapse
Affiliation(s)
- Heather D. Lucas
- Department of Psychology, Louisiana State University, Baton Rouge, LA, 70803, USA
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, IL, 61801, USA
| | - Ana M. Daugherty
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, IL, 61801, USA
- Department of Psychology and Institute of Gerontology, Wayne State University, Detroit, MI, 48202, USA
| | - Edward McAuley
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, IL, 61801, USA
- Department of Kinesiology and Community Health, University of Illinois at Urbana-Champaign, Urbana, IL, 61801, USA
| | - Arthur F. Kramer
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, IL, 61801, USA
- Center for Cognitive & Brain Health, Northeastern University, Boston, MA, 02115, USA
| | - Neal J. Cohen
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, IL, 61801, USA
- Department of Psychology, University of Illinois at Urbana-Champaign, Champaign, IL, 61820, USA
| |
Collapse
|
6
|
Doidy F, Desaunay P, Rebillard C, Clochon P, Lambrechts A, Wantzen P, Guénolé F, Baleyte JM, Eustache F, Bowler DM, Lebreton K, Guillery-Girard B. How scene encoding affects memory discrimination: Analysing eye movements data using data driven methods. VISUAL COGNITION 2023. [DOI: 10.1080/13506285.2023.2188335] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/08/2023]
Affiliation(s)
- F. Doidy
- Normandie Université, UNICAEN, PSL Université Paris, EPHE, INSERM, U1077, CHU de Caen, GIP Cyceron, Neuropsychologie et Imagerie de la Mémoire Humaine, Caen, France
| | - P. Desaunay
- Normandie Université, UNICAEN, PSL Université Paris, EPHE, INSERM, U1077, CHU de Caen, GIP Cyceron, Neuropsychologie et Imagerie de la Mémoire Humaine, Caen, France
- Service de Psychiatrie de l’enfant et de l’adolescent, CHU de Caen, Caen, France
| | - C. Rebillard
- Normandie Université, UNICAEN, PSL Université Paris, EPHE, INSERM, U1077, CHU de Caen, GIP Cyceron, Neuropsychologie et Imagerie de la Mémoire Humaine, Caen, France
| | - P. Clochon
- Normandie Université, UNICAEN, PSL Université Paris, EPHE, INSERM, U1077, CHU de Caen, GIP Cyceron, Neuropsychologie et Imagerie de la Mémoire Humaine, Caen, France
| | - A. Lambrechts
- Autism Research Group, Department of Psychology, City, University of London, London, UK
| | - P. Wantzen
- Normandie Université, UNICAEN, PSL Université Paris, EPHE, INSERM, U1077, CHU de Caen, GIP Cyceron, Neuropsychologie et Imagerie de la Mémoire Humaine, Caen, France
| | - F. Guénolé
- Normandie Université, UNICAEN, PSL Université Paris, EPHE, INSERM, U1077, CHU de Caen, GIP Cyceron, Neuropsychologie et Imagerie de la Mémoire Humaine, Caen, France
- Service de Psychiatrie de l’enfant et de l’adolescent, CHU de Caen, Caen, France
| | - J. M. Baleyte
- Normandie Université, UNICAEN, PSL Université Paris, EPHE, INSERM, U1077, CHU de Caen, GIP Cyceron, Neuropsychologie et Imagerie de la Mémoire Humaine, Caen, France
- Service de Psychiatrie de l’enfant et de l’adolescent, Centre Hospitalier Interuniversitaire de Créteil, Créteil, France
| | - F. Eustache
- Normandie Université, UNICAEN, PSL Université Paris, EPHE, INSERM, U1077, CHU de Caen, GIP Cyceron, Neuropsychologie et Imagerie de la Mémoire Humaine, Caen, France
| | - D. M. Bowler
- Autism Research Group, Department of Psychology, City, University of London, London, UK
| | - K. Lebreton
- Normandie Université, UNICAEN, PSL Université Paris, EPHE, INSERM, U1077, CHU de Caen, GIP Cyceron, Neuropsychologie et Imagerie de la Mémoire Humaine, Caen, France
| | - B. Guillery-Girard
- Normandie Université, UNICAEN, PSL Université Paris, EPHE, INSERM, U1077, CHU de Caen, GIP Cyceron, Neuropsychologie et Imagerie de la Mémoire Humaine, Caen, France
| |
Collapse
|
7
|
Rehrig G, Hayes TR, Henderson JM, Ferreira F. Visual attention during seeing for speaking in healthy aging. Psychol Aging 2023; 38:49-66. [PMID: 36395016 PMCID: PMC10021028 DOI: 10.1037/pag0000718] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Abstract
As we age, we accumulate a wealth of information about the surrounding world. Evidence from visual search suggests that older adults retain intact knowledge for where objects tend to occur in everyday environments (semantic information) that allows them to successfully locate objects in scenes, but may overrely on semantic guidance. We investigated age differences in the allocation of attention to semantically informative and visually salient information in a task in which the eye movements of younger (N = 30, aged 18-24) and older (N = 30, aged 66-82) adults were tracked as they described real-world scenes. We measured the semantic information in scenes based on "meaning map" ratings from a norming sample of young and older adults, and image salience as graph-based visual saliency. Logistic mixed-effects modeling was used to determine whether, controlling for center bias, fixated scene locations differed in semantic informativeness and visual salience from locations that were not fixated, and whether these effects differed for young and older adults. Semantic informativeness predicted fixated locations well overall, as did image salience, although unique variance in the model was better explained by semantic informativeness than image salience. Older adults were less likely to fixate informative locations in scenes than young adults were, though the locations older adults' fixated were independently predicted well by informativeness. These results suggest young and older adults both use semantic information to guide attention in scenes and that older adults do not overrely on semantic information across the board. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Collapse
Affiliation(s)
| | | | - John M. Henderson
- Department of Psychology, University of California, Davis
- Center for Mind and Brain, University of California, Davis
| | | |
Collapse
|
8
|
Katz CN, Schjetnan AGP, Patel K, Barkley V, Hoffman KL, Kalia SK, Duncan KD, Valiante TA. A corollary discharge mediates saccade-related inhibition of single units in mnemonic structures of the human brain. Curr Biol 2022; 32:3082-3094.e4. [PMID: 35779529 DOI: 10.1016/j.cub.2022.06.015] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2021] [Revised: 04/04/2022] [Accepted: 06/08/2022] [Indexed: 11/18/2022]
Abstract
Despite the critical link between visual exploration and memory, little is known about how neuronal activity in the human mesial temporal lobe (MTL) is modulated by saccades. Here, we characterize saccade-associated neuronal modulations, unit-by-unit, and contrast them to image onset and to occipital lobe neurons. We reveal evidence for a corollary discharge (CD)-like modulatory signal that accompanies saccades, inhibiting/exciting a unique population of broad-/narrow-spiking units, respectively, before and during saccades and with directional selectivity. These findings comport well with the timing, directional nature, and inhibitory circuit implementation of a CD. Additionally, by linking neuronal activity to event-related potentials (ERPs), which are directionally modulated following saccades, we recontextualize the ERP associated with saccades as a proxy for both the strength of inhibition and saccade direction, providing a mechanistic underpinning for the more commonly recorded saccade-related ERP in the human brain.
Collapse
Affiliation(s)
- Chaim N Katz
- Krembil Brain Institute, Toronto Western Hospital (TWH), Toronto, ON M5T 1M8, Canada; Institute of Biomedical Engineering, University of Toronto, Toronto, ON M5S 3G9, Canada; CRANIA, University Health Network and University of Toronto, Toronto, ON M5G 2A2, Canada; Faculty of Medicine, University of Calgary, Calgary, AB T2N 4N1, Canada
| | - Andrea G P Schjetnan
- Krembil Brain Institute, Toronto Western Hospital (TWH), Toronto, ON M5T 1M8, Canada; CRANIA, University Health Network and University of Toronto, Toronto, ON M5G 2A2, Canada
| | - Kramay Patel
- Krembil Brain Institute, Toronto Western Hospital (TWH), Toronto, ON M5T 1M8, Canada; Institute of Biomedical Engineering, University of Toronto, Toronto, ON M5S 3G9, Canada; Faculty of Medicine, University of Toronto, Toronto, ON M5S 1A8, Canada; CRANIA, University Health Network and University of Toronto, Toronto, ON M5G 2A2, Canada
| | - Victoria Barkley
- Krembil Brain Institute, Toronto Western Hospital (TWH), Toronto, ON M5T 1M8, Canada; CRANIA, University Health Network and University of Toronto, Toronto, ON M5G 2A2, Canada
| | - Kari L Hoffman
- Department of Psychology, Vanderbilt University, Nashville, TN 37240, USA
| | - Suneil K Kalia
- Krembil Brain Institute, Toronto Western Hospital (TWH), Toronto, ON M5T 1M8, Canada; Division of Neurosurgery, Department of Surgery, University of Toronto, Toronto, ON M5S 1A1, Canada; CRANIA, University Health Network and University of Toronto, Toronto, ON M5G 2A2, Canada; The KITE Research Institute, University Health Network, Toronto, ON M5G 2A2, Canada
| | - Katherine D Duncan
- Department of Psychology, University of Toronto, Toronto, ON M5S 3G3, Canada
| | - Taufik A Valiante
- Krembil Brain Institute, Toronto Western Hospital (TWH), Toronto, ON M5T 1M8, Canada; Institute of Biomedical Engineering, University of Toronto, Toronto, ON M5S 3G9, Canada; Division of Neurosurgery, Department of Surgery, University of Toronto, Toronto, ON M5S 1A1, Canada; Institute of Medical Sciences, University of Toronto, Toronto, ON M5S 1A8, Canada; Electrical and Computer Engineering, University of Toronto, Toronto, ON M5S 3G4, Canada; CRANIA, University Health Network and University of Toronto, Toronto, ON M5G 2A2, Canada; The KITE Research Institute, University Health Network, Toronto, ON M5G 2A2, Canada; Max Planck-University of Toronto Center for Neural Science and Technology, Toronto, ON, Canada.
| |
Collapse
|
9
|
Ryan JD, Wynn JS, Shen K, Liu ZX. Aging changes the interactions between the oculomotor and memory systems. NEUROPSYCHOLOGY, DEVELOPMENT, AND COGNITION. SECTION B, AGING, NEUROPSYCHOLOGY AND COGNITION 2022; 29:418-442. [PMID: 34856890 DOI: 10.1080/13825585.2021.2007841] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/24/2021] [Accepted: 11/12/2021] [Indexed: 06/13/2023]
Abstract
The use of multi-modal approaches, particularly in conjunction with multivariate analytic techniques, can enrich models of cognition, brain function, and how they change with age. Recently, multivariate approaches have been applied to the study of eye movements in a manner akin to that of neural activity (i.e., pattern similarity). Here, we review the literature regarding multi-modal and/or multivariate approaches, with specific reference to the use of eyetracking to characterize age-related changes in memory. By applying multi-modal and multivariate approaches to the study of aging, research has shown that aging is characterized by moment-to-moment alterations in the amount and pattern of visual exploration, and by extension, alterations in the activity and function of the hippocampus and broader medial temporal lobe (MTL). These methodological advances suggest that age-related declines in the integrity of the memory system has consequences for oculomotor behavior in the moment, in a reciprocal fashion. Age-related changes in hippocampal and MTL structure and function may lead to an increase in, and change in the patterns of, visual exploration in an effort to upregulate the encoding of information. However, such visual exploration patterns may be non-optimal and actually reduce the amount and/or type of incoming information that is bound into a lasting memory representation. This research indicates that age-related cognitive impairments are considerably broader in scope than previously realized.
Collapse
Affiliation(s)
- Jennifer D Ryan
- Rotman Research Institute at Baycrest Health Sciences, Toronto, ON, Canada
- Departments of Psychology, Psychiatry, University of Toronto, Toronto, ON, Canada
| | - Jordana S Wynn
- Department of Psychology, Harvard University, Cambridge MA, USA
| | - Kelly Shen
- Rotman Research Institute at Baycrest Health Sciences, Toronto, ON, Canada
| | - Zhong-Xu Liu
- Department of Behavioral Sciences, University of Michigan-Dearborn, Dearborn MI, USA
| |
Collapse
|
10
|
Amer T, Wynn JS, Hasher L. Cluttered memory representations shape cognition in old age. Trends Cogn Sci 2022; 26:255-267. [DOI: 10.1016/j.tics.2021.12.002] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Revised: 12/06/2021] [Accepted: 12/08/2021] [Indexed: 01/11/2023]
|
11
|
Wynn JS, Buchsbaum BR, Ryan JD. Encoding and retrieval eye movements mediate age differences in pattern completion. Cognition 2021; 214:104746. [PMID: 34034008 DOI: 10.1016/j.cognition.2021.104746] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2020] [Revised: 04/19/2021] [Accepted: 04/21/2021] [Indexed: 01/10/2023]
Abstract
Older adults often mistake new information as 'old', yet the mechanisms underlying this response bias remain unclear. Typically, false alarms by older adults are thought to reflect pattern completion - the retrieval of a previously encoded stimulus in response to partial input. However, other work suggests that age-related retrieval errors can be accounted for by deficient encoding processes. In the present study, we used eye movement monitoring to quantify age-related changes in behavioral pattern completion as a function of eye movements during both encoding and partially cued retrieval. Consistent with an age-related encoding deficit, older adults executed more gaze fixations and more similar eye movements across repeated image presentations than younger adults, and such effects were predictive of subsequent recognition memory. Analysis of eye movements at retrieval further indicated that in response to partial lure cues, older adults reactivated the similar studied image, indexed by the similarity between encoding and retrieval gaze patterns, and did so more than younger adults. Critically, reactivation of encoded image content via eye movements was associated with lure false alarms in older adults, providing direct evidence for a pattern completion bias. Together, these findings suggest that age-related changes in both encoding and retrieval processes, indexed by eye movements, underlie older adults' increased vulnerability to memory errors.
Collapse
Affiliation(s)
- Jordana S Wynn
- Department of Psychology, University of Toronto, Toronto, ON, Canada; Rotman Research Institute, Baycrest Hospital, Toronto, ON, Canada.
| | - Bradley R Buchsbaum
- Department of Psychology, University of Toronto, Toronto, ON, Canada; Rotman Research Institute, Baycrest Hospital, Toronto, ON, Canada
| | - Jennifer D Ryan
- Department of Psychology, University of Toronto, Toronto, ON, Canada; Rotman Research Institute, Baycrest Hospital, Toronto, ON, Canada
| |
Collapse
|
12
|
Conti F, Irish M. Harnessing Visual Imagery and Oculomotor Behaviour to Understand Prospection. Trends Cogn Sci 2021; 25:272-283. [PMID: 33618981 DOI: 10.1016/j.tics.2021.01.009] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Revised: 01/20/2021] [Accepted: 01/20/2021] [Indexed: 12/20/2022]
Abstract
Much of the rich internal world constructed by humans is derived from, and experienced through, visual mental imagery. Despite growing appreciation of visual exploration in guiding episodic memory processes, extant theories of prospection have yet to accommodate the precise role of visual mental imagery in the service of future-oriented thinking. We propose that the construction of future events relies on the assimilation of perceptual details originally experienced, and subsequently reinstantiated, predominantly in the visual domain. Individual differences in the capacity to summon discrete aspects of visual imagery can therefore account for the diversity of content generated by humans during future simulation. Our integrative framework provides a novel testbed to query alterations in future thinking in health and disease.
Collapse
Affiliation(s)
- Federica Conti
- Institut des Neurosciences de la Timone, Aix-Marseille University, 27 Boulevard Jean Moulin, 13005 Marseille, France; The University of Sydney, Brain and Mind Centre and School of Psychology, 94 Mallett Street, Camperdown, NSW 2050, Australia.
| | - Muireann Irish
- The University of Sydney, Brain and Mind Centre and School of Psychology, 94 Mallett Street, Camperdown, NSW 2050, Australia.
| |
Collapse
|
13
|
Gurtner LM, Hartmann M, Mast FW. Eye movements during visual imagery and perception show spatial correspondence but have unique temporal signatures. Cognition 2021; 210:104597. [PMID: 33508576 DOI: 10.1016/j.cognition.2021.104597] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2020] [Revised: 01/07/2021] [Accepted: 01/08/2021] [Indexed: 11/20/2022]
Abstract
Eye fixation patterns during mental imagery are similar to those during perception of the same picture, suggesting that oculomotor mechanisms play a role in mental imagery (i.e., the "looking at nothing" effect). Previous research has focused on the spatial similarities of eye movements during perception and mental imagery. The primary aim of this study was to assess whether the spatial similarity translates to the temporal domain. We used recurrence quantification analysis (RQA) to assess the temporal structure of eye fixations in visual perception and mental imagery and we compared the temporal as well as the spatial characteristics in mental imagery with perception by means of Bayesian hierarchical regression models. We further investigated how person and picture-specific characteristics contribute to eye movement behavior in mental imagery. Working memory capacity and mental imagery abilities were assessed to either predict gaze dynamics in visual imagery or to moderate a possible correspondence between spatial or temporal gaze dynamics in perception and mental imagery. We were able to show the spatial similarity of fixations between visual perception and imagery and we provide first evidence for its moderation by working memory capacity. Interestingly, the temporal gaze dynamics in mental imagery were unrelated to those in perception and their variance between participants was not explained by variance in visuo-spatial working memory capacity or vividness of mental images. The semantic content of the imagined pictures was the only meaningful predictor of temporal gaze dynamics. The spatial correspondence reflects shared spatial structure of mental images and perceived pictures, while the unique temporal gaze behavior could be driven by generation, maintenance and protection processes specific to visual imagery. The unique temporal gaze dynamics offer a window to new insights into the genuine process of mental imagery independent of its similarity to perception.
Collapse
Affiliation(s)
- Lilla M Gurtner
- Department of Psychology, University of Bern, Fabrikstrasse 8, 3012 Bern, Switzerland.
| | - Matthias Hartmann
- Department of Psychology, University of Bern, Fabrikstrasse 8, 3012 Bern, Switzerland; Faculty of Psychology, UniDistance Suisse, Überlandstrasse 12, 3900 Brig, Switzerland
| | - Fred W Mast
- Department of Psychology, University of Bern, Fabrikstrasse 8, 3012 Bern, Switzerland
| |
Collapse
|
14
|
Mahadas S, Semkewyc C, Suresh S, Hung GK. Scan path during change-detection visual search. Comput Biol Med 2021; 131:104233. [PMID: 33561672 DOI: 10.1016/j.compbiomed.2021.104233] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2020] [Revised: 01/16/2021] [Accepted: 01/16/2021] [Indexed: 11/25/2022]
Abstract
When observing a particular image or object, one's perception depends upon prior expectations, memory, and cognitive abilities. It is hypothesized that cognitive processing in the form of top-down or bottom-up processing could be determined via analysis of the eye fixation scan path. To assess the variations in scan paths, 7 subjects underwent 5 change-detection trials. During each trial, they were presented with a specific set of images via a MATLAB program, in which the original image alternated with a modified image consisting of a single change. An open-source program called GazeRecorder was used to track the subject's eye movements and to record the eye fixations. The scan path was then analyzed through the use of a 4 by 4 grid pattern superimposed on the image to determine the subject's eye fixation distribution pattern in terms of Boxes Viewed and Concentration within a single area. It was determined that higher Concentration was positively correlated with faster Detection Speed (R = 0.84), while higher number of Boxes Viewed was negatively correlated with Detection Speed (R = -0.71). Among the subjects, the more optimal scan paths were found in those with a balance between Concentration and Boxes Viewed, as subjects with a more balanced approach had the greatest Accuracy (p = 0.02). This indicates an optimal scan path involves both top-down and bottom-up processing to more efficiently identify a change. Moreover, the methodology developed in this study could be used in the home or clinic for quantitative assessment of improvement following therapy in patients with neurological deficits.
Collapse
Affiliation(s)
- Srikrishnaraja Mahadas
- Department of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ, USA
| | - Courtney Semkewyc
- Department of Biomedical Engineering, Rutgers University, Piscataway, NJ, USA
| | - Shradha Suresh
- Department of Biomedical Engineering, Rutgers University, Piscataway, NJ, USA
| | - George K Hung
- Department of Biomedical Engineering, Rutgers University, Piscataway, NJ, USA.
| |
Collapse
|
15
|
Armson MJ, Diamond NB, Levesque L, Ryan JD, Levine B. Vividness of recollection is supported by eye movements in individuals with high, but not low trait autobiographical memory. Cognition 2021; 206:104487. [DOI: 10.1016/j.cognition.2020.104487] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2020] [Revised: 10/05/2020] [Accepted: 10/07/2020] [Indexed: 11/25/2022]
|
16
|
Ramey MM, Henderson JM, Yonelinas AP. The spatial distribution of attention predicts familiarity strength during encoding and retrieval. J Exp Psychol Gen 2020; 149:2046-2062. [PMID: 32250136 PMCID: PMC7541439 DOI: 10.1037/xge0000758] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The memories we form are determined by what we attend to, and conversely, what we attend to is influenced by our memory for past experiences. Although we know that shifts of attention via eye movements are related to memory during encoding and retrieval, the role of specific memory processes in this relationship is unclear. There is evidence that attention may be especially important for some forms of memory (i.e., conscious recollection), and less so for others (i.e., familiarity-based recognition and unconscious influences of memory), but results are conflicting with respect to both the memory processes and eye movement patterns involved. To address this, we used a confidence-based method of isolating eye movement indices of spatial attention that are related to different memory processes (i.e., recollection, familiarity strength, and unconscious memory) during encoding and retrieval of real-world scenes. We also developed a new method of measuring the dispersion of eye movements, which proved to be more sensitive to memory processing than previously used measures. Specifically, in 2 studies, we found that familiarity strength-that is, changes in subjective reports of memory confidence-increased with (a) more dispersed patterns of viewing during encoding, (b) less dispersed viewing during retrieval, and (c) greater overlap in regions viewed between encoding and retrieval (i.e., resampling). Recollection was also related to these eye movements in a similar manner, though the associations with recollection were less consistent across experiments. Furthermore, we found no evidence for effects related to unconscious influences of memory. These findings indicate that attentional processes during viewing may not preferentially relate to recollection, and that the spatial distribution of eye movements is directly related to familiarity-based memory during encoding and retrieval. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Collapse
Affiliation(s)
- Michelle M. Ramey
- Department of Psychology, University of California, Davis, CA, USA
- Center for Neuroscience, University of California, Davis, CA, USA
- Center for Mind and Brain, University of California, Davis, CA, USA
| | - John M. Henderson
- Department of Psychology, University of California, Davis, CA, USA
- Center for Mind and Brain, University of California, Davis, CA, USA
| | - Andrew P. Yonelinas
- Department of Psychology, University of California, Davis, CA, USA
- Center for Neuroscience, University of California, Davis, CA, USA
| |
Collapse
|
17
|
Kragel JE, Voss JL. Temporal context guides visual exploration during scene recognition. J Exp Psychol Gen 2020; 150:873-889. [PMID: 32969680 DOI: 10.1037/xge0000827] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/05/2023]
Abstract
Memories for episodes are temporally structured. Cognitive models derived from list-learning experiments attribute this structure to the retrieval of temporal context information that indicates when a memory occurred. These models predict key features of memory recall, such as the strong tendency to retrieve studied items in the order in which they were first encountered. Can such models explain ecological memory behaviors, such as eye movements during encoding and retrieval of complex visual stimuli? We tested predictions from retrieved-context models using three data sets involving recognition memory and free viewing of complex scenes. Subjects reinstated sequences of eye movements from one scene-viewing episode to the next. Moreover, sequence reinstatement decayed over time and was associated with successful memory. We observed memory-driven reinstatement even after accounting for intrinsic scene properties that produced consistent eye movements. These findings confirm predictions of retrieved-context models, suggesting retrieval of temporal context influences complex behaviors generated during naturalistic memory experiences. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
Collapse
|
18
|
Ramey MM, Yonelinas AP, Henderson JM. Why do we retrace our visual steps? Semantic and episodic memory in gaze reinstatement. ACTA ACUST UNITED AC 2020; 27:275-283. [PMID: 32540917 PMCID: PMC7301753 DOI: 10.1101/lm.051227.119] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2019] [Accepted: 05/14/2020] [Indexed: 12/05/2022]
Abstract
When we look at repeated scenes, we tend to visit similar regions each time—a phenomenon known as resampling. Resampling has long been attributed to episodic memory, but the relationship between resampling and episodic memory has recently been found to be less consistent than assumed. A possibility that has yet to be fully considered is that factors unrelated to episodic memory may generate resampling: for example, other factors such as semantic memory and visual salience that are consistently present each time an image is viewed and are independent of specific prior viewing instances. We addressed this possibility by tracking participants’ eyes during scene viewing to examine how semantic memory, indexed by the semantic informativeness of scene regions (i.e., meaning), is involved in resampling. We found that viewing more meaningful regions predicted resampling, as did episodic familiarity strength. Furthermore, we found that meaning interacted with familiarity strength to predict resampling. Specifically, the effect of meaning on resampling was attenuated in the presence of strong episodic memory, and vice versa. These results suggest that episodic and semantic memory are each involved in resampling behavior and are in competition rather than synergistically increasing resampling. More generally, this suggests that episodic and semantic memory may compete to guide attention.
Collapse
Affiliation(s)
- Michelle M Ramey
- Department of Psychology, University of California, Davis, California 95616, USA.,Center for Neuroscience, University of California, Davis, California 95618, USA.,Center for Mind and Brain, University of California, Davis, California 95618, USA
| | - Andrew P Yonelinas
- Department of Psychology, University of California, Davis, California 95616, USA.,Center for Neuroscience, University of California, Davis, California 95618, USA
| | - John M Henderson
- Department of Psychology, University of California, Davis, California 95616, USA.,Center for Mind and Brain, University of California, Davis, California 95618, USA
| |
Collapse
|
19
|
Bone MB, St-Laurent M, Dang C, McQuiggan DA, Ryan JD, Buchsbaum BR. Eye Movement Reinstatement and Neural Reactivation During Mental Imagery. Cereb Cortex 2020; 29:1075-1089. [PMID: 29415220 DOI: 10.1093/cercor/bhy014] [Citation(s) in RCA: 38] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2017] [Accepted: 01/09/2018] [Indexed: 11/14/2022] Open
Abstract
Half a century ago, Donald Hebb posited that mental imagery is a constructive process that emulates perception. Specifically, Hebb claimed that visual imagery results from the reactivation of neural activity associated with viewing images. He also argued that neural reactivation and imagery benefit from the re-enactment of eye movement patterns that first occurred at viewing (fixation reinstatement). To investigate these claims, we applied multivariate pattern analyses to functional MRI (fMRI) and eye tracking data collected while healthy human participants repeatedly viewed and visualized complex images. We observed that the specificity of neural reactivation correlated positively with vivid imagery and with memory for stimulus image details. Moreover, neural reactivation correlated positively with fixation reinstatement, meaning that image-specific eye movements accompanied image-specific patterns of brain activity during visualization. These findings support the conception of mental imagery as a simulation of perception, and provide evidence consistent with the supportive role of eye movement in neural reactivation.
Collapse
Affiliation(s)
- Michael B Bone
- Rotman Research Institute at Baycrest, Toronto, Ontario, Canada.,Department of Psychology, University of Toronto, Toronto, Ontario, Canada
| | | | - Christa Dang
- Rotman Research Institute at Baycrest, Toronto, Ontario, Canada
| | | | - Jennifer D Ryan
- Rotman Research Institute at Baycrest, Toronto, Ontario, Canada.,Department of Psychology, University of Toronto, Toronto, Ontario, Canada
| | - Bradley R Buchsbaum
- Rotman Research Institute at Baycrest, Toronto, Ontario, Canada.,Department of Psychology, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
20
|
Yoo SA, Rosenbaum RS, Tsotsos JK, Fallah M, Hoffman KL. Long-term memory and hippocampal function support predictive gaze control during goal-directed search. J Vis 2020; 20:10. [PMID: 32455429 PMCID: PMC7409592 DOI: 10.1167/jov.20.5.10] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Eye movements during visual search change with prior experience for search stimuli. Previous studies measured these gaze effects shortly after initial viewing, typically during free viewing; it remains open whether the effects are preserved across long delays and for goal-directed search, and which memory system guides gaze. In Experiment 1, we analyzed eye movements of healthy adults viewing novel and repeated scenes while searching for a scene-embedded target. The task was performed across different time points to examine the repetition effects in long-term memory, and memory types were grouped based on explicit recall of targets. In Experiment 2, an amnesic person with bilateral extended hippocampal damage and the age-matched control group performed the same task with shorter intervals to determine whether or not the repetition effects depend on hippocampal function. When healthy adults explicitly remembered repeated target-scene pairs, search time and fixation duration decreased, and gaze was directed closer to the target region, than when they forgot targets. These effects were seen even after a one-month delay from their initial viewing, suggesting the effects are associated with long-term, explicit memory. Saccadic amplitude was not strongly modulated by scene repetition or explicit recall of targets. The amnesic person did not show explicit recall or implicit repetition effects, whereas his control group showed similar patterns to those seen in Experiment 1. The results reveal several aspects of gaze control that are influenced by long-term memory. The dependence of gaze effects on medial temporal lobe integrity support a role for this region in predictive gaze control.
Collapse
|
21
|
Bone MB, Ahmad F, Buchsbaum BR. Feature-specific neural reactivation during episodic memory. Nat Commun 2020; 11:1945. [PMID: 32327642 PMCID: PMC7181630 DOI: 10.1038/s41467-020-15763-2] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2019] [Accepted: 03/12/2020] [Indexed: 12/04/2022] Open
Abstract
We present a multi-voxel analytical approach, feature-specific informational connectivity (FSIC), that leverages hierarchical representations from a neural network to decode neural reactivation in fMRI data collected while participants performed an episodic visual recall task. We show that neural reactivation associated with low-level (e.g. edges), high-level (e.g. facial features), and semantic (e.g. “terrier”) features occur throughout the dorsal and ventral visual streams and extend into the frontal cortex. Moreover, we show that reactivation of both low- and high-level features correlate with the vividness of the memory, whereas only reactivation of low-level features correlates with recognition accuracy when the lure and target images are semantically similar. In addition to demonstrating the utility of FSIC for mapping feature-specific reactivation, these findings resolve the contributions of low- and high-level features to the vividness of visual memories and challenge a strict interpretation the posterior-to-anterior visual hierarchy. Memory recollection involves reactivation of neural activity that occurred during the recalled experience. Here, the authors show that neural reactivation can be decomposed into visual-semantic features, is widely synchronized throughout the brain, and predicts memory vividness and accuracy.
Collapse
Affiliation(s)
- Michael B Bone
- Rotman Research Institute at Baycrest, Toronto, ON, M6A 2E1, Canada. .,Department of Psychology, University of Toronto, Toronto, ON, M5S 1A1, Canada.
| | - Fahad Ahmad
- Rotman Research Institute at Baycrest, Toronto, ON, M6A 2E1, Canada
| | - Bradley R Buchsbaum
- Rotman Research Institute at Baycrest, Toronto, ON, M6A 2E1, Canada.,Department of Psychology, University of Toronto, Toronto, ON, M5S 1A1, Canada
| |
Collapse
|
22
|
Wynn JS, Ryan JD, Buchsbaum BR. Eye movements support behavioral pattern completion. Proc Natl Acad Sci U S A 2020; 117:6246-6254. [PMID: 32123109 PMCID: PMC7084073 DOI: 10.1073/pnas.1917586117] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022] Open
Abstract
The ability to recall a detailed event from a simple reminder is supported by pattern completion, a cognitive operation performed by the hippocampus wherein existing mnemonic representations are retrieved from incomplete input. In behavioral studies, pattern completion is often inferred through the false endorsement of lure (i.e., similar) items as old. However, evidence that such a response is due to the specific retrieval of a similar, previously encoded item is severely lacking. We used eye movement (EM) monitoring during a partial-cue recognition memory task to index reinstatement of lure images behaviorally via the recapitulation of encoding-related EMs or gaze reinstatement. Participants reinstated encoding-related EMs following degraded retrieval cues and this reinstatement was negatively correlated with accuracy for lure images, suggesting that retrieval of existing representations (i.e., pattern completion) underlies lure false alarms. Our findings provide evidence linking gaze reinstatement and pattern completion and advance a functional role for EMs in memory retrieval.
Collapse
Affiliation(s)
- Jordana S Wynn
- Department of Psychology, University of Toronto, Toronto, ON M55 3G3, Canada;
- Rotman Research Institute, Baycrest Hospital, Toronto, ON M6A 2E1, Canada
| | - Jennifer D Ryan
- Department of Psychology, University of Toronto, Toronto, ON M55 3G3, Canada
- Rotman Research Institute, Baycrest Hospital, Toronto, ON M6A 2E1, Canada
| | - Bradley R Buchsbaum
- Department of Psychology, University of Toronto, Toronto, ON M55 3G3, Canada
- Rotman Research Institute, Baycrest Hospital, Toronto, ON M6A 2E1, Canada
| |
Collapse
|
23
|
Ryan JD, Shen K, Liu Z. The intersection between the oculomotor and hippocampal memory systems: empirical developments and clinical implications. Ann N Y Acad Sci 2020; 1464:115-141. [PMID: 31617589 PMCID: PMC7154681 DOI: 10.1111/nyas.14256] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2019] [Revised: 08/29/2019] [Accepted: 09/19/2019] [Indexed: 12/28/2022]
Abstract
Decades of cognitive neuroscience research has shown that where we look is intimately connected to what we remember. In this article, we review findings from human and nonhuman animals, using behavioral, neuropsychological, neuroimaging, and computational modeling methods, to show that the oculomotor and hippocampal memory systems interact in a reciprocal manner, on a moment-to-moment basis, mediated by a vast structural and functional network. Visual exploration serves to efficiently gather information from the environment for the purpose of creating new memories, updating existing memories, and reconstructing the rich, vivid details from memory. Conversely, memory increases the efficiency of visual exploration. We call for models of oculomotor control to consider the influence of the hippocampal memory system on the cognitive control of eye movements, and for models of hippocampal and broader medial temporal lobe function to consider the influence of the oculomotor system on the development and expression of memory. We describe eye movement-based applications for the detection of neurodegeneration and delivery of therapeutic interventions for mental health disorders for which the hippocampus is implicated and memory dysfunctions are at the forefront.
Collapse
Affiliation(s)
- Jennifer D. Ryan
- Rotman Research InstituteBaycrestTorontoOntarioCanada
- Department of PsychologyUniversity of TorontoTorontoOntarioCanada
- Department of PsychiatryUniversity of TorontoTorontoOntarioCanada
| | - Kelly Shen
- Rotman Research InstituteBaycrestTorontoOntarioCanada
| | - Zhong‐Xu Liu
- Department of Behavioral SciencesUniversity of Michigan‐DearbornDearbornMichigan
| |
Collapse
|
24
|
Williams CC, Castelhano MS. The Changing Landscape: High-Level Influences on Eye Movement Guidance in Scenes. Vision (Basel) 2019; 3:E33. [PMID: 31735834 PMCID: PMC6802790 DOI: 10.3390/vision3030033] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2019] [Revised: 06/20/2019] [Accepted: 06/24/2019] [Indexed: 11/16/2022] Open
Abstract
The use of eye movements to explore scene processing has exploded over the last decade. Eye movements provide distinct advantages when examining scene processing because they are both fast and spatially measurable. By using eye movements, researchers have investigated many questions about scene processing. Our review will focus on research performed in the last decade examining: (1) attention and eye movements; (2) where you look; (3) influence of task; (4) memory and scene representations; and (5) dynamic scenes and eye movements. Although typically addressed as separate issues, we argue that these distinctions are now holding back research progress. Instead, it is time to examine the intersections of these seemingly separate influences and examine the intersectionality of how these influences interact to more completely understand what eye movements can tell us about scene processing.
Collapse
Affiliation(s)
- Carrick C. Williams
- Department of Psychology, California State University San Marcos, San Marcos, CA 92069, USA
| | | |
Collapse
|
25
|
Wynn JS, Shen K, Ryan JD. Eye Movements Actively Reinstate Spatiotemporal Mnemonic Content. Vision (Basel) 2019; 3:E21. [PMID: 31735822 PMCID: PMC6802778 DOI: 10.3390/vision3020021] [Citation(s) in RCA: 48] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2019] [Revised: 05/09/2019] [Accepted: 05/10/2019] [Indexed: 12/23/2022] Open
Abstract
Eye movements support memory encoding by binding distinct elements of the visual world into coherent representations. However, the role of eye movements in memory retrieval is less clear. We propose that eye movements play a functional role in retrieval by reinstating the encoding context. By overtly shifting attention in a manner that broadly recapitulates the spatial locations and temporal order of encoded content, eye movements facilitate access to, and reactivation of, associated details. Such mnemonic gaze reinstatement may be obligatorily recruited when task demands exceed cognitive resources, as is often observed in older adults. We review research linking gaze reinstatement to retrieval, describe the neural integration between the oculomotor and memory systems, and discuss implications for models of oculomotor control, memory, and aging.
Collapse
Affiliation(s)
- Jordana S. Wynn
- Rotman Research Institute, Baycrest, 3560 Bathurst St., Toronto, ON M6A 2E1, Canada
- Department of Psychology, University of Toronto, 100 St George St., Toronto, ON M5S 3G3, Canada
| | - Kelly Shen
- Rotman Research Institute, Baycrest, 3560 Bathurst St., Toronto, ON M6A 2E1, Canada
| | - Jennifer D. Ryan
- Rotman Research Institute, Baycrest, 3560 Bathurst St., Toronto, ON M6A 2E1, Canada
- Department of Psychology, University of Toronto, 100 St George St., Toronto, ON M5S 3G3, Canada
- Department of Psychiatry, University of Toronto, 250 College St., Toronto, ON M5T 1R8, Canada
| |
Collapse
|
26
|
Abstract
We investigated visual working memory encoding across saccadic eye movements, focusing our analysis on refixation behavior. Over 10-s periods, participants performed a visual search for three, four, or five targets and remembered their orientations for a subsequent change-detection task. In 50% of the trials, one of the targets had its orientation changed. From the visual search period, we scored three types of refixations and applied measures for quantifying eye-fixation recurrence patterns. Repeated fixations on the same regions as well as repeated fixation patterns increased with memory load. Correct change detection was associated with more refixations on targets and less on distractors, with increased frequency of recurrence, and with longer intervals between refixations. The results are in accordance with the view that patterns of eye movement are an integral part of visual working memory representation.
Collapse
|
27
|
Montefusco-Siegmund R, Leonard TK, Hoffman KL. Hippocampal gamma-band Synchrony and pupillary responses index memory during visual search. Hippocampus 2017; 27:425-434. [PMID: 28032676 DOI: 10.1002/hipo.22702] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/20/2016] [Indexed: 01/20/2023]
Abstract
Memory for scenes is supported by the hippocampus, among other interconnected structures, but the neural mechanisms related to this process are not well understood. To assess the role of the hippocampus in memory-guided scene search, we recorded local field potentials and multiunit activity from the hippocampus of macaques as they performed goal-directed search tasks using natural scenes. We additionally measured pupil size during scene presentation, which in humans is modulated by recognition memory. We found that both pupil dilation and search efficiency accompanied scene repetition, thereby indicating memory for scenes. Neural correlates included a brief increase in hippocampal multiunit activity and a sustained synchronization of unit activity to gamma band oscillations (50-70 Hz). The repetition effects on hippocampal gamma synchronization occurred when pupils were most dilated, suggesting an interaction between aroused, attentive processing and hippocampal correlates of recognition memory. These results suggest that the hippocampus may support memory-guided visual search through enhanced local gamma synchrony. © 2016 Wiley Periodicals, Inc.
Collapse
Affiliation(s)
| | - Timothy K Leonard
- Department of Psychology, Centre for Vision Research, York University, Toronto, Ontario, Canada
| | - Kari L Hoffman
- Department of Psychology, Department of Biology, Centre for Vision Research, Toronto, Ontario, Canada
| |
Collapse
|
28
|
Sharp-Wave Ripples in Primates Are Enhanced near Remembered Visual Objects. Curr Biol 2016; 27:257-262. [PMID: 28041797 DOI: 10.1016/j.cub.2016.11.027] [Citation(s) in RCA: 40] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2016] [Revised: 10/08/2016] [Accepted: 11/10/2016] [Indexed: 12/25/2022]
Abstract
The hippocampus plays an important role in memory for events that are distinct in space and time. One of the strongest, most synchronous neural signals produced by the hippocampus is the sharp-wave ripple (SWR), observed in a variety of mammalian species during offline behaviors, such as slow-wave sleep [1-3] and quiescent waking and pauses in exploration [4-8], leading to long-standing and widespread theories of its contribution to plasticity and memory during these inactive or immobile states [9-14]. Indeed, during sleep and waking inactivity, hippocampal SWRs in rodents appear to support spatial long-term and working memory [4, 15-23], but so far, they have not been linked to memory in primates. More recently, SWRs have been observed during active, visual scene exploration in macaques [24], opening up the possibility that these active-state ripples in the primate hippocampus are linked to memory for objects embedded in scenes. By measuring hippocampal SWRs in macaques during search for scene-contextualized objects, we found that SWR rate increased with repeated presentations. Furthermore, gaze during SWRs was more likely to be near the target object on repeated than on novel presentations, even after accounting for overall differences in gaze location with scene repetition. This proximity bias with repetition occurred near the time of target object detection for remembered targets. The increase in ripple likelihood near remembered visual objects suggests a link between ripples and memory in primates; specifically, SWRs may reflect part of a mechanism supporting the guidance of search based on past experience.
Collapse
|
29
|
Pupillary responses and memory-guided visual search reveal age-related and Alzheimer's-related memory decline. Behav Brain Res 2016; 322:351-361. [PMID: 27616343 DOI: 10.1016/j.bbr.2016.09.014] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2016] [Revised: 09/02/2016] [Accepted: 09/06/2016] [Indexed: 11/22/2022]
Abstract
Episodic memory - composed of memory for unique spatiotemporal experiences - is known to decline with aging, and even more severely in Alzheimer 's disease (AD). Memory for trial-unique objects in spatial scenes depends on the integrity of the hippocampus and interconnected structures that are among the first areas affected in AD. We reasoned that memory for objects-in-scenes would be impaired with aging, and that further impairments would be observed in AD. We asked younger adults, healthy older adults, older adults at-risk for developing cognitive impairments, and older adults with probable early AD to find changing items ('targets') within images of natural scenes, measuring repeated-trial changes in search efficiency and pupil diameter. Compared to younger adults, older adults took longer to detect target objects in repeated scenes, they required more fixations and those fixations were more dispersed. Whereas individuals with AD showed some benefit of memory in this task, they had substantially longer detection times, and more numerous, dispersed fixations on repeated scenes compared to age-matched older adults. Correspondingly, pupillary responses to novel and repeated scenes were diminished with aging and further in AD, and the memory-related changes were weaker with aging and absent in AD. Our results suggest that several nonverbal measures from memory-guided visual search tasks can index aging and Alzheimer's disease status, including pupillary dynamics. The task measurements are sensitive to the integrity of brain structures that are associated with Alzheimer's-related neurodegeneration, the task is well tolerated across a range of abilities, and thus, it may prove useful in early diagnostics and longitudinal tracking of memory decline.
Collapse
|