1
|
Özdemir Ş, Şentürk YD, Ünver N, Demircan C, Olivers CNL, Egner T, Günseli E. Effects of Context Changes on Memory Reactivation. J Neurosci 2024; 44:e2096232024. [PMID: 39103222 PMCID: PMC11376331 DOI: 10.1523/jneurosci.2096-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Revised: 06/26/2024] [Accepted: 07/10/2024] [Indexed: 08/07/2024] Open
Abstract
While the influence of context on long-term memory (LTM) is well documented, its effects on the interaction between working memory (WM) and LTM remain less understood. In this study, we explored these interactions using a delayed match-to-sample task, where participants (6 males, 16 females) encountered the same target object across six consecutive trials, facilitating the transition from WM to LTM. During half of these target repetitions, the background color changed. We measured the WM storage of the target using the contralateral delay activity in electroencephalography. Our results reveal that task-irrelevant context changes trigger the reactivation of long-term memories in WM. This reactivation may be attributed to content-context binding in WM and hippocampal pattern separation.
Collapse
Affiliation(s)
- Şahcan Özdemir
- Department of Psychology, Sabancı University, Istanbul 34956, Turkey
| | | | - Nursima Ünver
- Department of Psychology, Sabancı University, Istanbul 34956, Turkey
| | - Can Demircan
- Department of Psychology, Sabancı University, Istanbul 34956, Turkey
| | - Christian N L Olivers
- Department of Experimental and Applied Psychology, Vrije Universiteit, Amsterdam 1081 BT, the Netherlands
| | - Tobias Egner
- Department of Psychology & Neuroscience, Duke University, Durham, North Carolina 27708
| | - Eren Günseli
- Department of Psychology, Sabancı University, Istanbul 34956, Turkey
| |
Collapse
|
2
|
Aivar MP, Li CL, Tong MH, Kit DM, Hayhoe MM. Knowing where to go: Spatial memory guides eye and body movements in a naturalistic visual search task. J Vis 2024; 24:1. [PMID: 39226069 PMCID: PMC11373708 DOI: 10.1167/jov.24.9.1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/04/2024] Open
Abstract
Most research on visual search has used simple tasks presented on a computer screen. However, in natural situations visual search almost always involves eye, head, and body movements in a three-dimensional (3D) environment. The different constraints imposed by these two types of search tasks might explain some of the discrepancies in our understanding concerning the use of memory resources and the role of contextual objects during search. To explore this issue, we analyzed a visual search task performed in an immersive virtual reality apartment. Participants searched for a series of geometric 3D objects while eye movements and head coordinates were recorded. Participants explored the apartment to locate target objects whose location and visibility were manipulated. For objects with reliable locations, we found that repeated searches led to a decrease in search time and number of fixations and to a reduction of errors. Searching for those objects that had been visible in previous trials but were only tested at the end of the experiment was also easier than finding objects for the first time, indicating incidental learning of context. More importantly, we found that body movements showed changes that reflected memory for target location: trajectories were shorter and movement velocities were higher, but only for those objects that had been searched for multiple times. We conclude that memory of 3D space and target location is a critical component of visual search and also modifies movement kinematics. In natural search, memory is used to optimize movement control and reduce energetic costs.
Collapse
Affiliation(s)
- M Pilar Aivar
- Facultad de Psicología, Universidad Autónoma de Madrid, Madrid, Spain
- https://www.psicologiauam.es/aivar/
| | - Chia-Ling Li
- Institute of Neuroscience, The University of Texas at Austin, Austin, TX, USA
- Present address: Apple Inc., Cupertino, California, USA
| | - Matthew H Tong
- Center for Perceptual Systems, The University of Texas at Austin, Austin, TX, USA
- Present address: IBM Research, Cambridge, Massachusetts, USA
| | - Dmitry M Kit
- Center for Perceptual Systems, The University of Texas at Austin, Austin, TX, USA
- Present address: F5, Boston, Massachusetts, USA
| | - Mary M Hayhoe
- Center for Perceptual Systems, The University of Texas at Austin, Austin, TX, USA
| |
Collapse
|
3
|
Teoh J, Saito JM, Yeo Y, Winter S, Fukuda K. Perceptual comparisons induce lasting and generalizing changes to face memory reports. Cogn Res Princ Implic 2024; 9:57. [PMID: 39218993 PMCID: PMC11366729 DOI: 10.1186/s41235-024-00584-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Accepted: 08/06/2024] [Indexed: 09/04/2024] Open
Abstract
Humans are often tasked to remember new faces so that they can recognize the faces later in time. Previous studies found that memory reports for basic visual features (e.g., colors and shapes) are susceptible to systematic distortions as a result of comparison with new visual input, especially when the input is perceived as similar to the memory. The current study tested whether this similarity-induced memory bias (SIMB) would also occur with more complex face stimuli. The results showed that faces that are just perceptually encoded into visual working memory as well as retrieved from visual long-term memory are also susceptible to SIMB. Furthermore, once induced, SIMB persisted over time across cues through which the face memory was accessed for memory report. These results demonstrate the generalizability of SIMB to more complex and practically relevant stimuli, and thus, suggest potential real-world implications.
Collapse
Affiliation(s)
- Jerrick Teoh
- Department of Psychology, University of Toronto Mississauga, Mississauga, Canada
| | - Joseph M Saito
- Department of Psychology, University of Toronto, Toronto, Canada
| | - Yvanna Yeo
- Department of Psychology, University of Toronto Mississauga, Mississauga, Canada
| | - Sophia Winter
- Department of Psychology, University of Toronto Mississauga, Mississauga, Canada
| | - Keisuke Fukuda
- Department of Psychology, University of Toronto Mississauga, Mississauga, Canada.
- Department of Psychology, University of Toronto, Toronto, Canada.
| |
Collapse
|
4
|
Warden AC, Wickens CD, Clegg BA, Rehberg D, Ortega FR. Information Access Effort: The Role of Head Movements for Information Presented at Increasing Eccentricity on Flat Panel and Head-Mounted Displays. HUMAN FACTORS 2024; 66:2057-2081. [PMID: 37943177 DOI: 10.1177/00187208231204567] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2023]
Abstract
OBJECTIVE This experiment examined performance costs when processing two sources of information positioned at increasing distances using a flat panel display and an augmented reality head-mounted display (AR-HMD). BACKGROUND The AR-HMD enables positioning virtual information at various distances in space. However, the proximity compatibility principle suggests that closer separation when two sources of information require mental integration assists performance, whereas increased separation between two sources hurts integration performance more than when a single source requires focused attention. Previous studies have provided inconsistent findings regarding costs associated with increased separation. Few of these experiments have examined separation for both focused and integration tasks, compared vertical and lateral separation, or measured head movements. METHOD Three experiments collectively examined these issues using a flat panel display and a virtual display presented with an HMD, where the separation of information varied laterally or vertically during a focused attention (digit reading) task and an information integration (mental subtraction) task. RESULTS There was no performance cost for either display when information was increasingly separated. However, head movements mitigated performance costs by preserving accuracy at larger separations without increasing response time. CONCLUSION Head movements appear to mitigate performance costs associated with presenting information increasingly far apart on flat panel displays and HMDs. Both eye scanning and head movements appear to be less effortful than expected. APPLICATION These findings have important implications for design guidelines regarding the placement of information presented on flat panel displays and, more specifically, HMDs, which can present information 360° around the user.
Collapse
|
5
|
de Sardenberg Schmid L, Hardiess G. Inter-individual variability (but intra-individual stability) of overt versus covert rehearsal strategies in a digital Corsi task. J Vis 2024; 24:2. [PMID: 39087936 PMCID: PMC11305427 DOI: 10.1167/jov.24.8.2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Accepted: 06/26/2024] [Indexed: 08/02/2024] Open
Abstract
The Corsi (block-tapping) paradigm is a classic and well-established visuospatial working memory task in humans involving internal computations (memorizing of item sequences, organizing and updating the memorandum, and recall processes), as well as both overt and covert shifts of attention to facilitate rehearsal, serving to maintain the Corsi sequences during the retention phase. Here, we introduce a novel digital version of a Corsi task in which i) the difficulty of the memorandum (using sequence lengths ranging from 3 to 8) was controlled, ii) the execution of overt and/or covert attention as well as the visuospatial working memory load during the retention phase was manipulated, and iii) shifts of attention were quantified in all experimental phases. With this, we present behavioral data that demonstrate, characterize, and classify the individual effects of overt and covert strategies used as a means of encoding and rehearsal. In a full within-subject design, we tested 28 participants who had to solve three different Corsi conditions. While in condition A neither of the two strategies were restricted, in condition B the overt and in condition C the overt as well as the covert strategies were suppressed. Analyzing Corsi span, (eye) exploration index, and pupil size (change), data clearly show a continuum between overt and covert strategies over all participants (indicating inter-individual variability). Further, all participants showed stable strategy choice (indicating intra-individual stability), meaning that the preferred strategy was maintained in all three conditions, phases, and sequence lengths of the experiment.
Collapse
Affiliation(s)
- Lílian de Sardenberg Schmid
- Cognitive Neuroscience, Department of Biology, Institute of Neurobiology, University of Tübingen, Tübingen, Germany
- Systems Neuroscience & Neuroengineering, MPI for Biological Cybernetics, Tübingen, Germany
| | - Gregor Hardiess
- Cognitive Neuroscience, Department of Biology, Institute of Neurobiology, University of Tübingen, Tübingen, Germany
| |
Collapse
|
6
|
Bays PM, Schneegans S, Ma WJ, Brady TF. Representation and computation in visual working memory. Nat Hum Behav 2024; 8:1016-1034. [PMID: 38849647 DOI: 10.1038/s41562-024-01871-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Accepted: 03/22/2024] [Indexed: 06/09/2024]
Abstract
The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have advanced our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event. New experimental tools have revealed that observers possess richer information about the uncertainty in their memories and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visuospatial WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research on human WM from a computational perspective, including the neural mechanisms that support it.
Collapse
Affiliation(s)
- Paul M Bays
- Department of Psychology, University of Cambridge, Cambridge, UK
| | | | - Wei Ji Ma
- Center for Neural Science and Department of Psychology, New York University, New York, NY, USA
| | - Timothy F Brady
- Department of Psychology, University of California, San Diego, La Jolla, CA, USA.
| |
Collapse
|
7
|
Kumle L, Võ MLH, Nobre AC, Draschkow D. Multifaceted consequences of visual distraction during natural behaviour. COMMUNICATIONS PSYCHOLOGY 2024; 2:49. [PMID: 38812582 PMCID: PMC11129948 DOI: 10.1038/s44271-024-00099-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Accepted: 05/15/2024] [Indexed: 05/31/2024]
Abstract
Visual distraction is a ubiquitous aspect of everyday life. Studying the consequences of distraction during temporally extended tasks, however, is not tractable with traditional methods. Here we developed a virtual reality approach that segments complex behaviour into cognitive subcomponents, including encoding, visual search, working memory usage, and decision-making. Participants copied a model display by selecting objects from a resource pool and placing them into a workspace. By manipulating the distractibility of objects in the resource pool, we discovered interfering effects of distraction across the different cognitive subcomponents. We successfully traced the consequences of distraction all the way from overall task performance to the decision-making processes that gate memory usage. Distraction slowed down behaviour and increased costly body movements. Critically, distraction increased encoding demands, slowed visual search, and decreased reliance on working memory. Our findings illustrate that the effects of visual distraction during natural behaviour can be rather focal but nevertheless have cascading consequences.
Collapse
Affiliation(s)
- Levi Kumle
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Melissa L.-H. Võ
- Department of Psychology, Goethe University Frankfurt, Frankfurt, Germany
| | - Anna C. Nobre
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Wu Tsai Institute and Department of Psychology, Yale University, New Haven, CT USA
| | - Dejan Draschkow
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| |
Collapse
|
8
|
Hoogerbrugge AJ, Strauch C, Böing S, Nijboer TCW, Van der Stigchel S. Just-in-Time Encoding Into Visual Working Memory Is Contingent Upon Constant Availability of External Information. J Cogn 2024; 7:39. [PMID: 38706717 PMCID: PMC11067970 DOI: 10.5334/joc.364] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 04/16/2024] [Indexed: 05/07/2024] Open
Abstract
Humans maintain an intricate balance between storing information in visual working memory (VWM) and just-in-time sampling of the external world, rooted in a trade-off between the cost of maintaining items in VWM versus retrieving information as it is needed. Previous studies have consistently shown that one prerequisite of just-in-time sampling is a high degree of availability of external information, and that introducing a delay before being able to access information led participants to rely less on the external world and more on VWM. However, these studies manipulated availability in such a manner that the cost of sampling was stable and predictable. It is yet unclear whether participants become less reliant on external information when it is more difficult to factor in the cost of sampling that information. In two experiments, participants copied an example layout from the left to the right side of the screen. In Experiment 1, intermittent occlusion of the example layout led participants to attempt to encode more items per inspection than when the layout was constantly available, but this did not consistently result in more correct placements. However, these findings could potentially be explained by inherent differences in how long the example layout could be viewed. Therefore in Experiment 2, the example layout only became available after a gaze-contingent delay, which could be constant or variable. Here, the introduction of any delay led to increased VWM load compared to no delay, although the degree of variability in the delay did not alter behaviour. These results reaffirm that the nature of when we engage VWM is dynamical, and suggest that any disruption to the continuous availability of external information is the main driver of increased VWM usage relative to whether availability is predictable or not.
Collapse
Affiliation(s)
- Alex J. Hoogerbrugge
- Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| | - Christoph Strauch
- Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| | - Sanne Böing
- Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| | - Tanja C. W. Nijboer
- Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| | | |
Collapse
|
9
|
Kvitelashvili S, Kessler Y. The voluntary utilization of visual working memory. Sci Rep 2024; 14:7987. [PMID: 38575646 PMCID: PMC10995117 DOI: 10.1038/s41598-024-58685-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Accepted: 04/02/2024] [Indexed: 04/06/2024] Open
Abstract
While a vast amount of research has focused on understanding the capacity limits of visual working memory (VWM), little is known about how VWM resources are employed in unforced behavior and how they correlate with individual capacity constraints. We present a novel, openly available, and easy-to-administer paradigm enabling participants to freely utilize their VWM capacity. Participants had to reconstruct an array of colored squares. In each trial, they were allowed to alternate between the memory array and the reconstruction screen as many times as they wished, each time choosing how many items to reconstruct. This approach allowed us to estimate the number of utilized items, as well as the accuracy of the reconstruction. In addition, VWM capacity was measured using a change detection task. In two experiments, we show that participants tend to under-utilize their VWM resources, performing well below their capacity limits. Surprisingly, while the extent to which participants utilized their VWM was highly reliable, it was uncorrelated with VWM capacity, suggesting that VWM utilization is limited due to strategic considerations rather than capacity limits.
Collapse
Affiliation(s)
- Shalva Kvitelashvili
- Department of Psychology and School of Brain Sciences and Cognition, Ben-Gurion University of the Negev, Beer Sheva, Israel
| | - Yoav Kessler
- Department of Psychology and School of Brain Sciences and Cognition, Ben-Gurion University of the Negev, Beer Sheva, Israel.
| |
Collapse
|
10
|
Cruz AS, Cruz S, Remondes M. Effects of optogenetic silencing the anterior cingulate cortex in a delayed non-match to trajectory task. OXFORD OPEN NEUROSCIENCE 2024; 3:kvae002. [PMID: 38595941 PMCID: PMC10939314 DOI: 10.1093/oons/kvae002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Revised: 01/16/2024] [Accepted: 01/19/2024] [Indexed: 04/11/2024]
Abstract
Working memory is a fundamental cognitive ability, allowing us to keep information in memory for the time needed to perform a given task. A complex neural circuit fulfills these functions, among which is the anterior cingulate cortex (CG). Functionally and anatomically connected to the medial prefrontal, retrosplenial, midcingulate and hippocampus, as well as motor cortices, CG has been implicated in retrieving appropriate information when needed to select and control appropriate behavior. The role of cingulate cortex in working memory-guided behaviors remains unclear due to the lack of studies reversibly interfering with its activity during specific epochs of working memory. We used eNpHR3.0 to silence cingulate neurons while animals perform a standard delayed non-match to trajectory task, and found that, while not causing an absolute impairment in working memory, silencing cingulate neurons during retrieval decreases the mean performance if compared to silencing during encoding. Such retrieval-associated changes are accompanied by longer delays observed when light is delivered to control animals, when compared to eNpHR3.0+ ones, consistent with an adaptive recruitment of additional cognitive resources.
Collapse
Affiliation(s)
- Ana S Cruz
- Instituto de Medicina Molecular João Lobo Antunes, Faculdade de Medicina Universidade de Lisboa, Lisbon 1649-028, Portugal
| | - Sara Cruz
- Instituto de Medicina Molecular João Lobo Antunes, Faculdade de Medicina Universidade de Lisboa, Lisbon 1649-028, Portugal
| | - Miguel Remondes
- Instituto de Medicina Molecular João Lobo Antunes, Faculdade de Medicina Universidade de Lisboa, Lisbon 1649-028, Portugal
- Faculdade de Medicina Veterinária Universidade Lusófona, Lisbon 1749-024, Portugal
| |
Collapse
|
11
|
Yu X, Rahim RA, Geng JJ. Task-adaptive changes to the target template in response to distractor context: Separability versus similarity. J Exp Psychol Gen 2024; 153:564-572. [PMID: 37917441 PMCID: PMC10843062 DOI: 10.1037/xge0001507] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2023]
Abstract
Theories of attention hypothesize the existence of an attentional template that contains target features in working or long-term memory. It is frequently assumed that the template contains a veridical copy of the target, but recent studies suggest that this is not true when the distractors are linearly separable from the target. In such cases, target representations shift "off-veridical" in response to the distractor context, presumably because doing so is adaptive and increases the representational distinctiveness of targets from distractors. However, some have argued that the shifts may be entirely explained by perceptual biases created by simultaneous color contrast. Here we address this debate and test the more general hypothesis that the target template is adaptively shaped by elements of the distractor context needed to distinguish targets from distractors. We used a two-dimensional target and separately manipulated the linear separability of one dimension (color) and the visual similarity of the other (orientation). We found that target shifting along the linearly separable color dimension was dependent on the similarity of targets-to-distractors along the other dimension. The target representations were consistent with a postexperiment strategy questionnaire in which participants reported using color more when orientation was hard to use, and orientation more when it was easier to use. We conclude that the target template is task-adaptive and exploit features in the distractor context that most predictably distinguish targets from distractors to increase visual search efficiency. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
Collapse
Affiliation(s)
- Xinger Yu
- Center for Mind and Brain, University of California, Davis
| | - Raisa A. Rahim
- Center for Mind and Brain, University of California, Davis
| | - Joy J. Geng
- Center for Mind and Brain, University of California, Davis
- Department of Psychology, University of California, Davis
| |
Collapse
|
12
|
Jeong SK. Perceived image size modulates visual memory. Psychon Bull Rev 2023; 30:2282-2288. [PMID: 37268748 DOI: 10.3758/s13423-023-02313-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/18/2023] [Indexed: 06/04/2023]
Abstract
Previous studies have demonstrated that visual memory is improved when stimuli are processed by larger cortical regions. For example, a physically large stimulus that recruits larger areas of the retinotopic cortex is better remembered. However, the spatial extent of neural responses in the visual cortex is not only modulated by the retinal size of a stimulus, but also by the perceived size of the stimulus. In this online study, we modulated the perceived size of the visual stimuli using the Ebbinghaus illusion and asked participants to remember the stimuli. The results showed that perceptually larger images were remembered better than perceptually smaller but physically same-sized images. Our finding supports the idea that visual memory is modulated by top-down feedback from higher visual regions to the early visual cortex.
Collapse
Affiliation(s)
- Su Keun Jeong
- Department of Psychology, Chungbuk National University, Chungdae-ro 1, Seowon- Gu, Cheongju, Chungbuk, Korea, 28644.
| |
Collapse
|
13
|
Klotzsche F, Gaebler M, Villringer A, Sommer W, Nikulin V, Ohl S. Visual short-term memory-related EEG components in a virtual reality setup. Psychophysiology 2023; 60:e14378. [PMID: 37393581 DOI: 10.1111/psyp.14378] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Revised: 05/12/2023] [Accepted: 06/09/2023] [Indexed: 07/04/2023]
Abstract
Virtual reality (VR) offers a powerful tool for investigating cognitive processes, as it allows researchers to gauge behaviors and mental states in complex, yet highly controlled, scenarios. The use of VR head-mounted displays in combination with physiological measures such as EEG presents new challenges and raises the question whether established findings also generalize to a VR setup. Here, we used a VR headset to assess the spatial constraints underlying two well-established EEG correlates of visual short-term memory: the amplitude of the contralateral delay activity (CDA) and the lateralization of induced alpha power during memory retention. We tested observers' visual memory in a change detection task with bilateral stimulus arrays of either two or four items while varying the horizontal eccentricity of the memory arrays (4, 9, or 14 degrees of visual angle). The CDA amplitude differed between high and low memory load at the two smaller eccentricities, but not at the largest eccentricity. Neither memory load nor eccentricity significantly influenced the observed alpha lateralization. We further fitted time-resolved spatial filters to decode memory load from the event-related potential as well as from its time-frequency decomposition. Classification performance during the retention interval was above-chance level for both approaches and did not vary significantly across eccentricities. We conclude that commercial VR hardware can be utilized to study the CDA and lateralized alpha power, and we provide caveats for future studies targeting these EEG markers of visual memory in a VR setup.
Collapse
Affiliation(s)
- Felix Klotzsche
- Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- Faculty of Philosophy, Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Michael Gaebler
- Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- Faculty of Philosophy, Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Arno Villringer
- Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- Faculty of Philosophy, Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Werner Sommer
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Vadim Nikulin
- Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Sven Ohl
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| |
Collapse
|
14
|
Martarelli CS, Chiquet S, Ertl M. Keeping track of reality: embedding visual memory in natural behaviour. Memory 2023; 31:1295-1305. [PMID: 37727126 DOI: 10.1080/09658211.2023.2260148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Accepted: 07/21/2023] [Indexed: 09/21/2023]
Abstract
Since immersive virtual reality (IVR) emerged as a research method in the 1980s, the focus has been on the similarities between IVR and actual reality. In this vein, it has been suggested that IVR methodology might fill the gap between laboratory studies and real life. IVR allows for high internal validity (i.e., a high degree of experimental control and experimental replicability), as well as high external validity by letting participants engage with the environment in an almost natural manner. Despite internal validity being crucial to experimental designs, external validity also matters in terms of the generalizability of results. In this paper, we first highlight and summarise the similarities and differences between IVR, desktop situations (both non-immersive VR and computer experiments), and reality. In the second step, we propose that IVR is a promising tool for visual memory research in terms of investigating the representation of visual information embedded in natural behaviour. We encourage researchers to carry out experiments on both two-dimensional computer screens and in immersive virtual environments to investigate visual memory and validate and replicate the findings. IVR is valuable because of its potential to improve theoretical understanding and increase the psychological relevance of the findings.
Collapse
Affiliation(s)
| | - Sandra Chiquet
- Faculty of Psychology, UniDistance Suisse, Brig, Switzerland
| | - Matthias Ertl
- Department of Psychology, University of Bern, Bern, Switzerland
| |
Collapse
|
15
|
Draschkow D, Anderson NC, David E, Gauge N, Kingstone A, Kumle L, Laurent X, Nobre AC, Shiels S, Võ MLH. Using XR (Extended Reality) for Behavioral, Clinical, and Learning Sciences Requires Updates in Infrastructure and Funding. POLICY INSIGHTS FROM THE BEHAVIORAL AND BRAIN SCIENCES 2023; 10:317-323. [PMID: 37900910 PMCID: PMC10602770 DOI: 10.1177/23727322231196305] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/31/2023]
Abstract
Extended reality (XR, including augmented and virtual reality) creates a powerful intersection between information technology and cognitive, clinical, and education sciences. XR technology has long captured the public imagination, and its development is the focus of major technology companies. This article demonstrates the potential of XR to (1) deliver behavioral insights, (2) transform clinical treatments, and (3) improve learning and education. However, without appropriate policy, funding, and infrastructural investment, many research institutions will struggle to keep pace with the advances and opportunities of XR. To realize the full potential of XR for basic and translational research, funding should incentivize (1) appropriate training, (2) open software solutions, and (3) collaborations between complementary academic and industry partners. Bolstering the XR research infrastructure with the right investments and incentives is vital for delivering on the potential for transformative discoveries, innovations, and applications.
Collapse
Affiliation(s)
- Dejan Draschkow
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Nicola C. Anderson
- Department of Psychology, University of British Columbia, Vancouver, Canada
| | - Erwan David
- Department of Psychology, Scene Grammar Lab, Goethe University Frankfurt, Frankfurt am Main, Germany
| | - Nathan Gauge
- OxSTaR Oxford Simulation Teaching and Research, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Alan Kingstone
- Department of Psychology, University of British Columbia, Vancouver, Canada
| | - Levi Kumle
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Xavier Laurent
- Centre for Teaching and Learning, University of Oxford, Oxford, UK
| | - Anna C. Nobre
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wu Tsai Institute, Yale University, New Haven, USA
| | - Sally Shiels
- OxSTaR Oxford Simulation Teaching and Research, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Melissa L.-H. Võ
- Department of Psychology, Scene Grammar Lab, Goethe University Frankfurt, Frankfurt am Main, Germany
| |
Collapse
|
16
|
Fajardo S, Kozowyk PRB, Langejans GHJ. Measuring ancient technological complexity and its cognitive implications using Petri nets. Sci Rep 2023; 13:14961. [PMID: 37737280 PMCID: PMC10516984 DOI: 10.1038/s41598-023-42078-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Accepted: 09/05/2023] [Indexed: 09/23/2023] Open
Abstract
We implement a method from computer sciences to address a challenge in Paleolithic archaeology: how to infer cognition differences from material culture. Archaeological material culture is linked to cognition, and more complex ancient technologies are assumed to have required complex cognition. We present an application of Petri net analysis to compare Neanderthal tar production technologies and tie the results to cognitive requirements. We applied three complexity metrics, each relying on their own unique definitions of complexity, to the modeled production processes. Based on the results, we propose that Neanderthal technical cognition may have been analogous to that of contemporary modern humans. This method also enables us to distinguish the high-order cognitive functions combining traits like planning, inhibitory control, and learning that were likely required by different ancient technological processes. The Petri net approach can contribute to our understanding of technology and cognitive evolution as it can be used on different materials and technologies, across time and species.
Collapse
Affiliation(s)
- Sebastian Fajardo
- Department of Materials Science and Engineering, Delft University of Technology, 2628 CD, Delft, Zuid-Holland, The Netherlands.
| | - Paul R B Kozowyk
- Department of Materials Science and Engineering, Delft University of Technology, 2628 CD, Delft, Zuid-Holland, The Netherlands
| | - Geeske H J Langejans
- Department of Materials Science and Engineering, Delft University of Technology, 2628 CD, Delft, Zuid-Holland, The Netherlands
- Palaeo-Research Institute, University of Johannesburg, Johannesburg, 2092, Gauteng, South Africa
| |
Collapse
|
17
|
Kozowyk PRB, Fajardo S, Langejans GHJ. Scaling Palaeolithic tar production processes exponentially increases behavioural complexity. Sci Rep 2023; 13:14709. [PMID: 37679497 PMCID: PMC10485137 DOI: 10.1038/s41598-023-41963-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Accepted: 09/04/2023] [Indexed: 09/09/2023] Open
Abstract
Technological processes, reconstructed from the archaeological record, are used to study the evolution of behaviour and cognition of Neanderthals and early modern humans. In comparisons, technologies that are more complex infer more complex behaviour and cognition. The manufacture of birch bark tar adhesives is regarded as particularly telling and often features in debates about Neanderthal cognition. One method of tar production, the 'condensation technique', demonstrates a pathway for Neanderthals to have discovered birch bark tar. However, to improve on the relatively low yield, and to turn tar into a perennial innovation, this method likely needed to be scaled up. Yet, it is currently unknown how scaling Palaeolithic technological processes influences their complexity. We used Petri net models and the Extended Cyclomatic Metric to measure system complexity of birch tar production with a single and three concurrent condensation assemblies. Our results show that changing the number of concurrent tar production assemblies substantially increases the measured complexity. This has potential implications on the behavioural and cognitive capacities required by Neanderthals, such as an increase in cooperation or inhibition control.
Collapse
Affiliation(s)
- Paul R B Kozowyk
- Department of Materials Science and Engineering, Delft University of Technology, 2628 CD, Delft, The Netherlands.
| | - Sebastian Fajardo
- Department of Materials Science and Engineering, Delft University of Technology, 2628 CD, Delft, The Netherlands
| | - Geeske H J Langejans
- Department of Materials Science and Engineering, Delft University of Technology, 2628 CD, Delft, The Netherlands
- Palaeo-Research Institute, University of Johannesburg, Johannesburg, Gauteng, 2092, South Africa
| |
Collapse
|
18
|
Kallmayer A, Võ MLH, Draschkow D. Viewpoint dependence and scene context effects generalize to depth rotated three-dimensional objects. J Vis 2023; 23:9. [PMID: 37707802 PMCID: PMC10506680 DOI: 10.1167/jov.23.10.9] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Accepted: 08/17/2023] [Indexed: 09/15/2023] Open
Abstract
Viewpoint effects on object recognition interact with object-scene consistency effects. While recognition of objects seen from "noncanonical" viewpoints (e.g., a cup from below) is typically impeded compared to processing of objects seen from canonical viewpoints (e.g., the string-side of a guitar), this effect is reduced by meaningful scene context information. In the present study we investigated if these findings established by using photographic images, generalize to strongly noncanonical orientations of three-dimensional (3D) models of objects. Using 3D models allowed us to probe a broad range of viewpoints and empirically establish viewpoints with very strong noncanonical and canonical orientations. In Experiment 1, we presented 3D models of objects from six different viewpoints (0°, 60°, 120°, 180° 240°, 300°) in color (1a) and grayscaled (1b) in a sequential matching task. Viewpoint had a significant effect on accuracy and response times. Based on the viewpoint effect in Experiments 1a and 1b, we could empirically determine the most canonical and noncanonical viewpoints from our set of viewpoints to use in Experiment 2. In Experiment 2, participants again performed a sequential matching task, however now the objects were paired with scene backgrounds which could be either consistent (e.g., a cup in the kitchen) or inconsistent (e.g., a guitar in the bathroom) to the object. Viewpoint interacted significantly with scene consistency in that object recognition was less affected by viewpoint when consistent scene information was provided, compared to inconsistent information. Our results show that scene context supports object recognition even when using extremely noncanonical orientations of depth rotated 3D objects. This supports the important role object-scene processing plays for object constancy especially under conditions of high uncertainty.
Collapse
Affiliation(s)
- Aylin Kallmayer
- Department of Psychology, Goethe University Frankfurt, Frankfurt am Main, Germany
| | - Melissa L-H Võ
- Department of Psychology, Goethe University Frankfurt, Frankfurt am Main, Germany
| | - Dejan Draschkow
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, UK
| |
Collapse
|
19
|
Chawoush B, Draschkow D, van Ede F. Capacity and selection in immersive visual working memory following naturalistic object disappearance. J Vis 2023; 23:9. [PMID: 37548958 PMCID: PMC10411649 DOI: 10.1167/jov.23.8.9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2023] [Accepted: 07/06/2023] [Indexed: 08/08/2023] Open
Abstract
Visual working memory-holding past visual information in mind for upcoming behavior-is commonly studied following the abrupt removal of visual objects from static two-dimensional (2D) displays. In everyday life, visual objects do not typically vanish from the environment in front of us. Rather, visual objects tend to enter working memory following self or object motion: disappearing from view gradually and changing the spatial relation between memoranda and observer. Here, we used virtual reality (VR) to investigate whether two classic findings from visual working memory research-a capacity of around three objects and the reliance on space for object selection-generalize to more naturalistic modes of object disappearance. Our static reference condition mimicked traditional laboratory tasks whereby visual objects were held static in front of the participant and removed from view abruptly. In our critical flow condition, the same visual objects flowed by participants, disappearing from view gradually and behind the observer. We considered visual working memory performance and capacity, as well as space-based mnemonic selection, indexed by directional biases in gaze. Despite vastly distinct modes of object disappearance and altered spatial relations between memoranda and observer, we found comparable capacity and comparable gaze signatures of space-based mnemonic selection. This finding reveals how classic findings from visual working memory research generalize to immersive situations with more naturalistic modes of object disappearance and with dynamic spatial relations between memoranda and observer.
Collapse
Affiliation(s)
- Babak Chawoush
- Institute for Brain and Behavior Amsterdam, Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Dejan Draschkow
- Department of Experimental Psychology, University of Oxford, United Kingdom
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, United Kingdom
| | - Freek van Ede
- Institute for Brain and Behavior Amsterdam, Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
20
|
Hoogerbrugge AJ, Strauch C, Nijboer TCW, Van der Stigchel S. Don't hide the instruction manual: A dynamic trade-off between using internal and external templates during visual search. J Vis 2023; 23:14. [PMID: 37486300 PMCID: PMC10382786 DOI: 10.1167/jov.23.7.14] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/25/2023] Open
Abstract
Visual search is typically studied by requiring participants to memorize a template initially, for which they subsequently search in a crowded display. Search in daily life, however, often involves templates that remain accessible externally, and may therefore be (re)attended for just-in-time encoding or to refresh internal template representations. Here, we show that participants indeed use external templates during search when given the chance. This behavior was observed during both simple and complex search, scaled with task difficulty, and was associated with improved performance. Furthermore, we show that participants used external sampling not only to offload memory, but also as a means of verifying whether the template was remembered correctly at the end of trials. We conclude that the external world may not only provide the challenge (e.g., distractors), but may dynamically ease search. These results argue for extensions of state-of-the-art models of search, because external sampling seems to be used frequently, in at least two ways and is actually beneficial for task performance. Our findings support a model of visual working memory that emphasizes a resource-efficient trade-off between storing and (re)attending external information.
Collapse
Affiliation(s)
- Alex J Hoogerbrugge
- Experimental Psychology, Helmholtz Institute, Utrecht University, The Netherlands
| | - Christoph Strauch
- Experimental Psychology, Helmholtz Institute, Utrecht University, The Netherlands
| | - Tanja C W Nijboer
- Experimental Psychology, Helmholtz Institute, Utrecht University, The Netherlands
- Center of Excellence for Rehabilitation Medicine, UMC Utrecht Brain Center, University Medical Center Utrecht, The Netherlands
- De Hoogstraat Rehabilitation, Department of Rehabilitation, Physical Therapy Science & Sports, UMC Utrecht Brain Center, University Medical Center Utrecht, The Netherlands
| | | |
Collapse
|
21
|
Hamblin-Frohman Z, Becker SI. Attentional selection is a sufficient cause for visual working memory interference. J Vis 2023; 23:15. [PMID: 37486298 PMCID: PMC10382781 DOI: 10.1167/jov.23.7.15] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/25/2023] Open
Abstract
Visual attention and visual working memory (VWM) are intertwined processes that allow navigation of the visual world. These systems can compete for highly limited cognitive resources, creating interference effects when both operate in tandem. Performing an attentional task while maintaining a VWM load often leads to a loss of memory information. These losses are seen even with very simple visual search tasks. Previous research has argued that this may be due to the attentional selection process, of choosing the target item out of surrounding nontarget items. Over two experiments, the current study disentangles the roles of search and selection in visual search and their influence on a retained VWM load. Experiment 1 revealed that, when search stimuli were relatively simple, target-absent searches (which did not require attentional selection) did not provoke memory interference, whereas target-present search did. In Experiment 2, the number of potential targets was varied in the search displays. In one condition, participants were required to select any one of the items displayed, requiring an attentional selection but no need to search for a specific item. Importantly, this condition led to memory interference to the same extent as a condition where a single target was presented among nontargets. Together, these results show that the process of attentional selection is a sufficient cause for interference with a concurrently maintained VWM load.
Collapse
Affiliation(s)
| | - Stefanie I Becker
- School of Psychology, The University of Queensland, Brisbane, Australia
| |
Collapse
|
22
|
Reeves JS, Proffitt T, Almeida-Warren K, Luncz LV. Modeling Oldowan tool transport from a primate perspective. J Hum Evol 2023; 181:103399. [PMID: 37356333 DOI: 10.1016/j.jhevol.2023.103399] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Revised: 05/12/2023] [Accepted: 05/12/2023] [Indexed: 06/27/2023]
Abstract
Living nonhuman primates have long served as a referential framework for understanding various aspects of hominin biological and cultural evolution. Comparing the cognitive, social, and ecological contexts of nonhuman primate and hominin tool use has allowed researchers to identify key adaptations relevant to the evolution of hominin behavior. Although the Oldowan is often considered to be a major evolutionary milestone, it has been argued that the Oldowan is rather an extension of behaviors already present in the ape lineage. This is based on the fact that while apes move tools through repeated, unplanned, short-distance transport bouts, they produce material patterning often associated with long-distance transport, planning, and foresight in the Oldowan. Nevertheless, remain fundamental differences in how Oldowan core and flake technology and nonhuman primate tools are used. The goal of the Oldowan hominins is to produce sharp-edged flakes, whereas nonhuman primates use stone tools primarily as percussors. Here, we present an agent-based model that investigates the explanatory power of the ape tool transport model in light of these differences. The model simulates the formation of the Oldowan record under the conditions of an accumulated short-distance transport pattern, as seen in extant chimpanzees. Our results show that while ape tool transport can account for some of the variation observed in the archaeological record, factors related to use-life duration severely limit how far an Oldowan core can be moved through repeated short-distance transport bouts. Thus, the ape tool transport has limitations in its ability to explain patterns in the Oldowan. These results provide a basis for discussing adaptive processes that would have facilitated the development of the Oldowan.
Collapse
Affiliation(s)
- Jonathan S Reeves
- Technological Primates Research Group, Max Planck Institute for Evolutionary Anthropology, Deutscher Platz 6, Leipzig, 04103, Germany; Center for the Advanced Study of Human Paleobiology, Department of Anthropology, The George Washington University, 800 2nd Street, NW, 20052, USA.
| | - Tomos Proffitt
- Technological Primates Research Group, Max Planck Institute for Evolutionary Anthropology, Deutscher Platz 6, Leipzig, 04103, Germany
| | - Katarina Almeida-Warren
- Primate Models for Behavioural Evolution Lab, Institute of Human Sciences, University of Oxford, 64 Banbury Road, Oxford, OX2 6PN, UK; Interdisciplinary Center for Archaeology and Evolution of Human Behaviour (ICArEHB), Universidade do Algarve, Faro, Portugal
| | - Lydia V Luncz
- Technological Primates Research Group, Max Planck Institute for Evolutionary Anthropology, Deutscher Platz 6, Leipzig, 04103, Germany; Center for the Advanced Study of Human Paleobiology, Department of Anthropology, The George Washington University, 800 2nd Street, NW, 20052, USA
| |
Collapse
|
23
|
Koevoet D, Naber M, Strauch C, Somai RS, Van der Stigchel S. Differential aspects of attention predict the depth of visual working memory encoding: Evidence from pupillometry. J Vis 2023; 23:9. [PMID: 37318440 PMCID: PMC10278550 DOI: 10.1167/jov.23.6.9] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2022] [Accepted: 05/15/2023] [Indexed: 06/16/2023] Open
Abstract
What determines how much one encodes into visual working memory? Traditionally, encoding depth is considered to be indexed by spatiotemporal properties of gaze, such as gaze position and dwell time. Although these properties inform about where and how long one looks, they do not necessarily inform about the current arousal state or how strongly attention is deployed to facilitate encoding. Here, we found that two types of pupillary dynamics predict how much information is encoded during a copy task. The task involved encoding a spatial pattern of multiple items for later reproduction. Results showed that smaller baseline pupil sizes preceding and stronger pupil orienting responses during encoding predicted that more information was encoded into visual working memory. Additionally, we show that pupil size reflects not only how much but also how precisely material is encoded. We argue that a smaller pupil size preceding encoding is related to increased exploitation, whereas larger pupil constrictions signal stronger attentional (re)orienting to the to-be-encoded pattern. Our findings support the notion that the depth of visual working memory encoding is the integrative outcome of differential aspects of attention: how alert one is, how much attention one deploys, and how long it is deployed. Together, these factors determine how much information is encoded into visual working memory.
Collapse
Affiliation(s)
- Damian Koevoet
- Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| | - Marnix Naber
- Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| | - Christoph Strauch
- Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| | - Rosyl S Somai
- Psychology, Faculty of Natural Sciences, University of Stirling, Stirling, UK
| | | |
Collapse
|
24
|
Böing S, Ten Brink AF, Hoogerbrugge AJ, Oudman E, Postma A, Nijboer TCW, Van der Stigchel S. Eye Movements as Proxy for Visual Working Memory Usage: Increased Reliance on the External World in Korsakoff Syndrome. J Clin Med 2023; 12:jcm12113630. [PMID: 37297825 DOI: 10.3390/jcm12113630] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Revised: 05/12/2023] [Accepted: 05/14/2023] [Indexed: 06/12/2023] Open
Abstract
In the assessment of visual working memory, estimating the maximum capacity is currently the gold standard. However, traditional tasks disregard that information generally remains available in the external world. Only when to-be-used information is not readily accessible, memory is taxed. Otherwise, people sample information from the environment as a form of cognitive offloading. To investigate how memory deficits impact the trade-off between sampling externally or storing internally, we compared gaze behaviour of individuals with Korsakoff amnesia (n = 24, age range 47-74 years) and healthy controls (n = 27, age range 40-81 years) on a copy task that provoked different strategies by having information freely accessible (facilitating sampling) or introducing a gaze-contingent waiting time (provoking storing). Indeed, patients sampled more often and longer, compared to controls. When sampling became time-consuming, controls reduced sampling and memorised more. Patients also showed reduced and longer sampling in this condition, suggesting an attempt at memorisation. Importantly, however, patients sampled disproportionately more often than controls, whilst accuracy dropped. This finding suggests that amnesia patients sample frequently and do not fully compensate for increased sampling costs by memorising more at once. In other words, Korsakoff amnesia resulted in a heavy reliance on the world as 'external memory'.
Collapse
Affiliation(s)
- Sanne Böing
- Experimental Psychology, Helmholtz Institute, Utrecht University, 3584 CS Utrecht, The Netherlands
| | - Antonia F Ten Brink
- Experimental Psychology, Helmholtz Institute, Utrecht University, 3584 CS Utrecht, The Netherlands
| | - Alex J Hoogerbrugge
- Experimental Psychology, Helmholtz Institute, Utrecht University, 3584 CS Utrecht, The Netherlands
| | - Erik Oudman
- Korsakoff Center of Expertise Slingedael, 3086 EZ Rotterdam, The Netherlands
| | - Albert Postma
- Experimental Psychology, Helmholtz Institute, Utrecht University, 3584 CS Utrecht, The Netherlands
- Korsakoff Center of Expertise Slingedael, 3086 EZ Rotterdam, The Netherlands
| | - Tanja C W Nijboer
- Experimental Psychology, Helmholtz Institute, Utrecht University, 3584 CS Utrecht, The Netherlands
- Center of Excellence for Rehabilitation Medicine, University Medical Center Utrecht and De Hoogstraat Rehabilitation, 3583 TM Utrecht, The Netherlands
| | - Stefan Van der Stigchel
- Experimental Psychology, Helmholtz Institute, Utrecht University, 3584 CS Utrecht, The Netherlands
| |
Collapse
|
25
|
Sahakian A, Gayet S, Paffen CLE, Van der Stigchel S. Mountains of memory in a sea of uncertainty: Sampling the external world despite useful information in visual working memory. Cognition 2023; 234:105381. [PMID: 36724621 DOI: 10.1016/j.cognition.2023.105381] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Revised: 12/14/2022] [Accepted: 01/17/2023] [Indexed: 02/02/2023]
Abstract
A large part of research on visual working memory (VWM) has traditionally focused on estimating its maximum capacity. Yet, humans rarely need to load up their VWM maximally during natural behavior, since visual information often remains accessible in the external world. Recent work, using paradigms that take into account the accessibility of information in the outside world, has indeed shown that observers utilize only one or two items in VWM before sampling from the external world again. One straightforward interpretation of this finding is that, in daily behavior, much fewer items are memorized than the typically reported capacity limits. Here, we first investigate whether this lower reliance on VWM when information is externally accessible might instead reflect resampling before VWM is actually depleted. To this aim we devised an online task, in which participants copied a model (six items in a 4x4 grid; always accessible) in an adjacent empty 4x4 grid. A key aspect of our paradigm is that we (unpredictably) interrupted participants just before inspection of the model with a 2-alternative-forced-choice (2-AFC) question, probing their VWM content. Critically, we observed above-chance performance on probes appearing just before model inspection. This finding shows that the external world was resampled, despite VWM still containing relevant information. We then asked whether increasing the cost of sampling causes participants to load up more information in VWM or, alternatively, to squeeze out more information from VWM (at the cost of making more errors). To manipulate the cost of resampling, we made it more difficult (specifically, more time-consuming) to access the model. We show that with increased cost of accessing the model (which lead to fewer, but longer model inspections), participants could place more items correctly immediately after sampling, and they kept attempting to place items for longer after their first error. These findings demonstrate that participants both encoded more information in VWM and made attempts to squeeze out more information from VWM when sampling became more costly. We argue that human observers constantly evaluate how certain they are of their VWM contents, and only use that VWM content of which their certainty exceeds a context-dependent "action threshold". This threshold, in turn, depends on the trade-off between the cost of resampling and the benefits of making an action. We argue that considering the interplay between the available VWM contents and a context-dependent action threshold, is key for reconciling the traditional VWM literature with VWM use in our day-to-day behavior.
Collapse
Affiliation(s)
- Andre Sahakian
- Department of Experimental Psychology & Helmholtz Institute, Utrecht University, Heidelberglaan 1, 3584 CS, Utrecht, The Netherlands.
| | - Surya Gayet
- Department of Experimental Psychology & Helmholtz Institute, Utrecht University, Heidelberglaan 1, 3584 CS, Utrecht, The Netherlands
| | - Chris L E Paffen
- Department of Experimental Psychology & Helmholtz Institute, Utrecht University, Heidelberglaan 1, 3584 CS, Utrecht, The Netherlands
| | - Stefan Van der Stigchel
- Department of Experimental Psychology & Helmholtz Institute, Utrecht University, Heidelberglaan 1, 3584 CS, Utrecht, The Netherlands
| |
Collapse
|
26
|
Murphy E, North E, Nawaz S, Omigie D. The influence of music liking on episodic memory for rich spatiotemporal contexts. Memory 2023; 31:589-604. [PMID: 37083746 DOI: 10.1080/09658211.2022.2154367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/22/2023]
Abstract
It is thought that the presence of music influences episodic memory encoding. However, no studies have isolated the influence of music liking - the hedonic value listeners attribute to a musical stimulus - from that of the core affect induced by the presence of that music. In an online survey, participants rated musical excerpts in terms of how much they liked them, as well as in terms of felt valence, felt arousal and familiarity. These ratings were then used to inform the stimuli presented in an online episodic memory task which, across different scenarios, involved dragging cued objects to cued locations and then recalling details of what was moved, where they were moved to and the order of movements made. Our results showed an influence of liking and music-reward sensitivity on memory for what was moved, as well as a detrimental effect of arousing musical stimuli on memory for un-cued scenario details. Taken together, our study showcases the importance of episodic memory paradigms that involve rich spatiotemporal contexts and provides insights into how different aspects of episodic memory may be influenced by the presence of music.
Collapse
Affiliation(s)
- Ellen Murphy
- Department of Psychology, Goldsmiths University of London, London, UK
| | - E North
- Department of Psychology, Goldsmiths University of London, London, UK
| | - S Nawaz
- Department of Psychology, Goldsmiths University of London, London, UK
| | - D Omigie
- Department of Psychology, Goldsmiths University of London, London, UK
| |
Collapse
|
27
|
Nobre AC, van Ede F. Attention in flux. Neuron 2023; 111:971-986. [PMID: 37023719 DOI: 10.1016/j.neuron.2023.02.032] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Revised: 02/20/2023] [Accepted: 02/22/2023] [Indexed: 04/08/2023]
Abstract
Selective attention comprises essential infrastructural functions supporting cognition-anticipating, prioritizing, selecting, routing, integrating, and preparing signals to guide adaptive behavior. Most studies have examined its consequences, systems, and mechanisms in a static way, but attention is at the confluence of multiple sources of flux. The world advances, we operate within it, our minds change, and all resulting signals progress through multiple pathways within the dynamic networks of our brains. Our aim in this review is to raise awareness of and interest in three important facets of how timing impacts our understanding of attention. These include the challenges posed to attention by the timing of neural processing and psychological functions, the opportunities conferred to attention by various temporal structures in the environment, and how tracking the time courses of neural and behavioral modulations with continuous measures yields surprising insights into the workings and principles of attention.
Collapse
Affiliation(s)
- Anna C Nobre
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, UK; Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford OX3 7JX, UK.
| | - Freek van Ede
- Institute for Brain and Behavior Amsterdam, Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, Amsterdam 1081BT, the Netherlands.
| |
Collapse
|
28
|
Priming of probabilistic attentional templates. Psychon Bull Rev 2023; 30:22-39. [PMID: 35831678 DOI: 10.3758/s13423-022-02125-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/09/2022] [Indexed: 11/08/2022]
Abstract
Attentional priming has a dominating influence on vision, speeding visual search, releasing items from crowding, reducing masking effects, and during free-choice, primed targets are chosen over unprimed ones. Many accounts postulate that templates stored in working memory control what we attend to and mediate the priming. But what is the nature of these templates (or representations)? Analyses of real-world visual scenes suggest that tuning templates to exact color or luminance values would be impractical since those can vary greatly because of changes in environmental circumstances and perceptual interpretation. Tuning templates to a range of the most probable values would be more efficient. Recent evidence does indeed suggest that the visual system represents such probability, gradually encoding statistical variation in the environment through repeated exposure to input statistics. This is consistent with evidence from neurophysiology and theoretical neuroscience as well as computational evidence of probabilistic representations in visual perception. I argue that such probabilistic representations are the unit of attentional priming and that priming of, say, a repeated single-color value simply involves priming of a distribution with no variance. This "priming of probability" view can be modelled within a Bayesian framework where priming provides contextual priors. Priming can therefore be thought of as learning of the underlying probability density function of the target or distractor sets in a given continuous task.
Collapse
|
29
|
Schuetz I, Karimpur H, Fiehler K. vexptoolbox: A software toolbox for human behavior studies using the Vizard virtual reality platform. Behav Res Methods 2023; 55:570-582. [PMID: 35322350 PMCID: PMC10027796 DOI: 10.3758/s13428-022-01831-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/09/2022] [Indexed: 11/08/2022]
Abstract
Virtual reality (VR) is a powerful tool for researchers due to its potential to study dynamic human behavior in highly naturalistic environments while retaining full control over the presented stimuli. Due to advancements in consumer hardware, VR devices are now very affordable and have also started to include technologies such as eye tracking, further extending potential research applications. Rendering engines such as Unity, Unreal, or Vizard now enable researchers to easily create complex VR environments. However, implementing the experimental design can still pose a challenge, and these packages do not provide out-of-the-box support for trial-based behavioral experiments. Here, we present a Python toolbox, designed to facilitate common tasks when developing experiments using the Vizard VR platform. It includes functionality for common tasks like creating, randomizing, and presenting trial-based experimental designs or saving results to standardized file formats. Moreover, the toolbox greatly simplifies continuous recording of eye and body movements using any hardware supported in Vizard. We further implement and describe a simple goal-directed reaching task in VR and show sample data recorded from five volunteers. The toolbox, example code, and data are all available on GitHub under an open-source license. We hope that our toolbox can simplify VR experiment development, reduce code duplication, and aid reproducibility and open-science efforts.
Collapse
Affiliation(s)
- Immo Schuetz
- Experimental Psychology, Justus Liebig University, Otto-Behaghel-Str. 10 F, 35394, Giessen, Germany.
- Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen, Giessen, Germany.
| | - Harun Karimpur
- Experimental Psychology, Justus Liebig University, Otto-Behaghel-Str. 10 F, 35394, Giessen, Germany
- Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen, Giessen, Germany
| | - Katja Fiehler
- Experimental Psychology, Justus Liebig University, Otto-Behaghel-Str. 10 F, 35394, Giessen, Germany
- Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen, Giessen, Germany
| |
Collapse
|
30
|
Abstract
Flexible behavior requires guidance not only by sensations that are available immediately but also by relevant mental contents carried forward through working memory. Therefore, selective-attention functions that modulate the contents of working memory to guide behavior (inside-out) are just as important as those operating on sensory signals to generate internal contents (outside-in). We review the burgeoning literature on selective attention in the inside-out direction and underscore its functional, flexible, and future-focused nature. We discuss in turn the purpose (why), targets (what), sources (when), and mechanisms (how) of selective attention inside working memory, using visual working memory as a model. We show how the study of internal selective attention brings new insights concerning the core cognitive processes of attention and working memory and how considering selective attention and working memory together paves the way for a rich and integrated understanding of how mind serves behavior.
Collapse
Affiliation(s)
- Freek van Ede
- Institute for Brain and Behavior Amsterdam, and Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands;
| | - Anna C Nobre
- Departments of Experimental Psychology and Psychiatry, Oxford Centre for Human Brain Activity, and Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, United Kingdom;
| |
Collapse
|
31
|
Botch TL, Garcia BD, Choi YB, Feffer N, Robertson CE. Active visual search in naturalistic environments reflects individual differences in classic visual search performance. Sci Rep 2023; 13:631. [PMID: 36635491 PMCID: PMC9837148 DOI: 10.1038/s41598-023-27896-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Accepted: 01/10/2023] [Indexed: 01/13/2023] Open
Abstract
Visual search is a ubiquitous activity in real-world environments. Yet, traditionally, visual search is investigated in tightly controlled paradigms, where head-restricted participants locate a minimalistic target in a cluttered array that is presented on a computer screen. Do traditional visual search tasks predict performance in naturalistic settings, where participants actively explore complex, real-world scenes? Here, we leverage advances in virtual reality technology to test the degree to which classic and naturalistic search are limited by a common factor, set size, and the degree to which individual differences in classic search behavior predict naturalistic search behavior in a large sample of individuals (N = 75). In a naturalistic search task, participants looked for an object within their environment via a combination of head-turns and eye-movements using a head-mounted display. Then, in a classic search task, participants searched for a target within a simple array of colored letters using only eye-movements. In each task, we found that participants' search performance was impacted by increases in set size-the number of items in the visual display. Critically, we observed that participants' efficiency in classic search tasks-the degree to which set size slowed performance-indeed predicted efficiency in real-world scenes. These results demonstrate that classic, computer-based visual search tasks are excellent models of active, real-world search behavior.
Collapse
Affiliation(s)
- Thomas L Botch
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, 03755, USA.
| | - Brenda D Garcia
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, 03755, USA
| | - Yeo Bi Choi
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, 03755, USA
| | - Nicholas Feffer
- Department of Computer Science, Dartmouth College, Hanover, NH, 03755, USA
- Department of Computer Science, Stanford University, Stanford, CA, 94305, USA
| | - Caroline E Robertson
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, 03755, USA
| |
Collapse
|
32
|
Reggente N. VR for Cognition and Memory. Curr Top Behav Neurosci 2023; 65:189-232. [PMID: 37440126 DOI: 10.1007/7854_2023_425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/14/2023]
Abstract
This chapter will provide a review of research into human cognition through the lens of VR-based paradigms for studying memory. Emphasis is placed on why VR increases the ecological validity of memory research and the implications of such enhancements.
Collapse
Affiliation(s)
- Nicco Reggente
- Institute for Advanced Consciousness Studies, Santa Monica, CA, USA.
| |
Collapse
|
33
|
Updating, Fast and Slow: Items, but Not Item-Context Bindings, are Quickly Updated Into Working Memory as Part of Response Selection. J Cogn 2023; 6:11. [PMID: 36721798 PMCID: PMC9854283 DOI: 10.5334/joc.257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Accepted: 12/20/2022] [Indexed: 01/18/2023] Open
Abstract
It is commonly held that attending to items facilitates their encoding into working memory (WM). This implies that the content of WM is updated with new input as a consequence of directing attention to it. On the other hand, abundant research shows that WM updating is rather slow and effortful, suggesting that shielding WM representation against incoming input, rather than its updating, is the default. To resolve this discrepancy, we suggest that while updating item-to-context associations is costly, updating a single item is fast and is automatically carried out as part of directing attention to items, for example as part of response selection. Participants performed a choice-RT task, in which stimuli appeared within frames, and needed to update their WM with the most recent red item that appeared in each frame. The need for updating was manipulated, so that some trials required updating and others did not. Experiment 1 (N = 25) showed that updating was slower than not updating with a set-size of two items, that required item-context binding, but faster when the set-size only involved one item. Experiment 2 (N = 28) replicated this finding. Experiment 3 (N = 20) showed that the slower no-update RTs are due to the removal of erroneously updated information. In contrast to previous findings, these results suggest that updating can be effortless and obligatory.
Collapse
|
34
|
Abstract
In this reflective piece on visual working memory, I depart from the laboriously honed skills of writing a review. Instead of integrating approaches, synthesizing evidence, and building a cohesive perspective, I scratch my head and share niggles and puzzlements. I expose where my scholarship and understanding are stumped by findings and standard views in the literature.
Collapse
|
35
|
Helbing J, Draschkow D, L-H Võ M. Auxiliary Scene-Context Information Provided by Anchor Objects Guides Attention and Locomotion in Natural Search Behavior. Psychol Sci 2022; 33:1463-1476. [PMID: 35942922 DOI: 10.1177/09567976221091838] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
Successful adaptive behavior requires efficient attentional and locomotive systems. Previous research has thoroughly investigated how we achieve this efficiency during natural behavior by exploiting prior knowledge related to targets of our actions (e.g., attending to metallic targets when looking for a pot) and to the environmental context (e.g., looking for the pot in the kitchen). Less is known about whether and how individual nontarget components of the environment support natural behavior. In our immersive virtual reality task, 24 adult participants searched for objects in naturalistic scenes in which we manipulated the presence and arrangement of large, static objects that anchor predictions about targets (e.g., the sink provides a prediction for the location of the soap). Our results show that gaze and body movements in this naturalistic setting are strongly guided by these anchors. These findings demonstrate that objects auxiliary to the target are incorporated into the representations guiding attention and locomotion.
Collapse
Affiliation(s)
- Jason Helbing
- Scene Grammar Lab, Department of Psychology, Goethe University Frankfurt
| | - Dejan Draschkow
- Brain and Cognition Laboratory, Department of Experimental Psychology, University of Oxford.,Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford
| | - Melissa L-H Võ
- Scene Grammar Lab, Department of Psychology, Goethe University Frankfurt
| |
Collapse
|
36
|
Draschkow D, Nobre AC, van Ede F. Multiple spatial frames for immersive working memory. Nat Hum Behav 2022; 6:536-544. [PMID: 35058640 PMCID: PMC7612679 DOI: 10.1038/s41562-021-01245-y] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2020] [Accepted: 10/25/2021] [Indexed: 11/09/2022]
Abstract
As we move around, relevant information that disappears from sight can still be held in working memory to serve upcoming behaviour. How we maintain and select visual information as we move through the environment remains poorly understood because most laboratory tasks of working memory rely on removing visual material while participants remain still. We used virtual reality to study visual working memory following self-movement in immersive environments. Directional biases in gaze revealed the recruitment of more than one spatial frame for maintaining and selecting memoranda following self-movement. The findings bring the important realization that multiple spatial frames support working memory in natural behaviour. The results also illustrate how virtual reality can be a critical experimental tool to characterize this core memory system.
Collapse
Affiliation(s)
- Dejan Draschkow
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, UK.
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Anna C Nobre
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Freek van Ede
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, UK
- Institute for Brain and Behavior Amsterdam, Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
37
|
Peacock CE, Zhang T, David-John B, Murdison TS, Boring MJ, Benko H, Jonker TR. Gaze dynamics are sensitive to target orienting for working memory encoding in virtual reality. J Vis 2022; 22:2. [PMID: 34982104 PMCID: PMC8742516 DOI: 10.1167/jov.22.1.2] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Numerous studies have demonstrated that visuospatial attention is a requirement for successful working memory encoding. It is unknown, however, whether this established relationship manifests in consistent gaze dynamics as people orient their visuospatial attention toward an encoding target when searching for information in naturalistic environments. To test this hypothesis, participants' eye movements were recorded while they searched for and encoded objects in a virtual apartment (Experiment 1). We decomposed gaze into 61 features that capture gaze dynamics and a trained sliding window logistic regression model that has potential for use in real-time systems to predict when participants found target objects for working memory encoding. A model trained on group data successfully predicted when people oriented to a target for encoding for the trained task (Experiment 1) and for a novel task (Experiment 2), where a new set of participants found objects and encoded an associated nonword in a cluttered virtual kitchen. Six of these features were predictive of target orienting for encoding, even during the novel task, including decreased distances between subsequent fixation/saccade events, increased fixation probabilities, and slower saccade decelerations before encoding. This suggests that as people orient toward a target to encode new information at the end of search, they decrease task-irrelevant, exploratory sampling behaviors. This behavior was common across the two studies. Together, this research demonstrates how gaze dynamics can be used to capture target orienting for working memory encoding and has implications for real-world use in technology and special populations.
Collapse
Affiliation(s)
| | - Ting Zhang
- Reality Labs Research; Redmond, WA, USA.,
| | | | | | | | | | | |
Collapse
|
38
|
Udale R, Tran MT, Manohar S, Husain M. Dynamic in-flight shifts of working memory resources across saccades. J Exp Psychol Hum Percept Perform 2022; 48:21-36. [PMID: 35073141 PMCID: PMC8785606 DOI: 10.1037/xhp0000960] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2020] [Revised: 08/06/2021] [Accepted: 08/13/2021] [Indexed: 11/08/2022]
Abstract
Little is known about how memory resources are allocated in natural vision across sequential eye movements and fixations, as people actively extract information from the visual environment. Here, we used gaze-contingent eye tracking to examine how such resources are dynamically reallocated from old to new information entering working memory. As participants looked sequentially at items, we interrupted the process at different times by extinguishing the display as a saccade was initiated. After a brief interval, participants were probed on one of the items that had been presented. Paradoxically, across all experiments, the final (unfixated) saccade target was recalled more precisely when more items had previously been fixated, that is, with longer rather than shorter saccade sequences. This result is difficult to explain on current models of working memory because recall error, even for the final item, is typically higher as memory load increases. The findings could however be accounted for by a model that describes how resources are dynamically reallocated on a moment-by-moment basis. During each saccade, the target is encoded by consuming a proportion of currently available resources from a limited working memory, as well as by reallocating resources away from previously encoded items. These findings reveal how working memory resources are shifted across memoranda in active vision. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Collapse
Affiliation(s)
- Rob Udale
- Department of Psychology, University of Sheffield
| | - Moc Tram Tran
- Department of Experimental Psychology, University of Oxford
| | - Sanjay Manohar
- Department of Experimental Psychology, University of Oxford
| | - Masud Husain
- Department of Experimental Psychology, University of Oxford
| |
Collapse
|
39
|
Mastropasqua A, Vural G, Taylor PCJ. Elements of exogenous attentional cueing preserved during optokinetic motion of the visual scene. Eur J Neurosci 2021; 55:746-761. [PMID: 34964525 DOI: 10.1111/ejn.15582] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2021] [Revised: 12/14/2021] [Accepted: 12/18/2021] [Indexed: 11/27/2022]
Abstract
Navigating through our environment raises challenges for perception by generating salient background visual motion, and eliciting prominent eye movements to stabilise the retinal image. It remains unclear if exogenous spatial attentional orienting is possible during background motion and the eye movements it causes, and whether this compromises the underlying neural processing. To test this, we combined exogenous orienting, visual scene motion, and EEG. 26 participants viewed a background of moving black and grey bars (optokinetic stimulation). We tested for effects of non-spatially predictive peripheral cueing on visual motion discrimination of a target dot, presented either at the same (valid) or opposite (invalid) location as the preceding cue. Valid cueing decreased reaction times not only when participants kept their gaze fixed on a central point (fixation blocks), but even when there was no fixation point, so that participants performed intensive, repetitive tracking eye movements (eye movements blocks). Overall, manual response reaction times were slower during eye movements. Cueing also produced reliable effects on neural activity on either block, including within the first 120 milliseconds of neural processing of the target. The key pattern with larger ERP amplitudes on invalid versus valid trials showed that the neural substrate of exogenous cueing was highly similar during eye movements or fixation. Exogenous peripheral cueing and its neural correlates are robust against distraction from the moving visual scene, important for perceptual cognition during navigation.
Collapse
Affiliation(s)
- Angela Mastropasqua
- Danish Research Centre for Magnetic Resonance, Centre for Functional and Diagnostic Imaging and Research, Copenhagen University Hospital - Amager and Hvidovre, Copenhagen, Denmark
| | - Gizem Vural
- Department of Forensic Psychiatry, Psychiatric Hospital of the LMU Munich, Germany
| | - Paul C J Taylor
- Department of Neurology, University Hospital, LMU Munich, Germany.,German Center for Vertigo and Balance Disorders, University Hospital, LMU Munich, Germany.,Department of Psychology, LMU Munich, Germany.,Faculty of Philosophy and Philosophy of Science, LMU Munich, Germany.,Munich Center for Neuroscience, LMU Munich, Germany
| |
Collapse
|
40
|
Abstract
A new study suggests that visual working memory usage is interestingly low during a more naturalistic virtual reality paradigm, compared to capacity estimates from traditional lab studies. This raises new questions about the use of working memory in everyday tasks.
Collapse
Affiliation(s)
- Jamal Williams
- Department of Psychology, University of California, San Diego, CA 92093, USA.
| | - Viola S Störmer
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755, USA.
| |
Collapse
|
41
|
Franchak JM, McGee B, Blanch G. Adapting the coordination of eyes and head to differences in task and environment during fully-mobile visual exploration. PLoS One 2021; 16:e0256463. [PMID: 34415981 PMCID: PMC8378697 DOI: 10.1371/journal.pone.0256463] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2021] [Accepted: 08/06/2021] [Indexed: 11/19/2022] Open
Abstract
How are eyes and head adapted to meet the demands of visual exploration in different tasks and environments? In two studies, we measured the horizontal movements of the eyes (using mobile eye tracking in Studies 1 and 2) and the head (using inertial sensors in Study 2) while participants completed a walking task and a search and retrieval task in a large, outdoor environment. We found that the spread of visual exploration was greater while searching compared with walking, and this was primarily driven by increased movement of the head as opposed to the eyes. The contributions of the head to gaze shifts of different eccentricities was greater when searching compared to when walking. Findings are discussed with respect to understanding visual exploration as a motor action with multiple degrees of freedom.
Collapse
Affiliation(s)
- John M. Franchak
- Department of Psychology, University of California, Riverside, Riverside, California, United States of America
| | - Brianna McGee
- Department of Psychology, University of California, Riverside, Riverside, California, United States of America
| | - Gabrielle Blanch
- Department of Psychology, University of California, Riverside, Riverside, California, United States of America
| |
Collapse
|
42
|
Stout D. The Cognitive Science of Technology. Trends Cogn Sci 2021; 25:964-977. [PMID: 34362661 DOI: 10.1016/j.tics.2021.07.005] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2021] [Revised: 07/09/2021] [Accepted: 07/13/2021] [Indexed: 01/23/2023]
Abstract
Technology is central to human life but hard to define and study. This review synthesizes advances in fields from anthropology to evolutionary biology and neuroscience to propose an interdisciplinary cognitive science of technology. The foundation of this effort is an evolutionarily motivated definition of technology that highlights three key features: material production, social collaboration, and cultural reproduction. This broad scope respects the complexity of the subject but poses a challenge for theoretical unification. Addressing this challenge requires a comparative approach to reduce the diversity of real-world technological cognition to a smaller number of recurring processes and relationships. To this end, a synthetic perceptual-motor hypothesis (PMH) for the evolutionary-developmental-cultural construction of technological cognition is advanced as an initial target for investigation.
Collapse
Affiliation(s)
- Dietrich Stout
- Department of Anthropology, Emory University, 1557 Dickey Drive, Atlanta, GA 30322, USA.
| |
Collapse
|
43
|
Abstract
Cognitive processes-from basic sensory analysis to language understanding-are typically contextualized. While the importance of considering context for understanding cognition has long been recognized in psychology and philosophy, it has not yet had much impact on cognitive neuroscience research, where cognition is often studied in decontextualized paradigms. Here, we present examples of recent studies showing that context changes the neural basis of diverse cognitive processes, including perception, attention, memory, and language. Within the domains of perception and language, we review neuroimaging results showing that context interacts with stimulus processing, changes activity in classical perception and language regions, and recruits additional brain regions that contribute crucially to naturalistic perception and language. We discuss how contextualized cognitive neuroscience will allow for discovering new principles of the mind and brain.
Collapse
Affiliation(s)
- Roel M Willems
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands.,Centre for Language Studies, Radboud University, Nijmegen, the Netherlands.,Max Planck Institute for Psycholinguistics, Nijmegen, the Netherlands
| | - Marius V Peelen
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
| |
Collapse
|
44
|
Kristjánsson Á, Draschkow D. Keeping it real: Looking beyond capacity limits in visual cognition. Atten Percept Psychophys 2021; 83:1375-1390. [PMID: 33791942 PMCID: PMC8084831 DOI: 10.3758/s13414-021-02256-7] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/23/2020] [Indexed: 11/23/2022]
Abstract
Research within visual cognition has made tremendous strides in uncovering the basic operating characteristics of the visual system by reducing the complexity of natural vision to artificial but well-controlled experimental tasks and stimuli. This reductionist approach has for example been used to assess the basic limitations of visual attention, visual working memory (VWM) capacity, and the fidelity of visual long-term memory (VLTM). The assessment of these limits is usually made in a pure sense, irrespective of goals, actions, and priors. While it is important to map out the bottlenecks our visual system faces, we focus here on selected examples of how such limitations can be overcome. Recent findings suggest that during more natural tasks, capacity may be higher than reductionist research suggests and that separable systems subserve different actions, such as reaching and looking, which might provide important insights about how pure attentional or memory limitations could be circumvented. We also review evidence suggesting that the closer we get to naturalistic behavior, the more we encounter implicit learning mechanisms that operate "for free" and "on the fly." These mechanisms provide a surprisingly rich visual experience, which can support capacity-limited systems. We speculate whether natural tasks may yield different estimates of the limitations of VWM, VLTM, and attention, and propose that capacity measurements should also pass the real-world test within naturalistic frameworks. Our review highlights various approaches for this and suggests that our understanding of visual cognition will benefit from incorporating the complexities of real-world cognition in experimental approaches.
Collapse
Affiliation(s)
- Árni Kristjánsson
- School of Health Sciences, University of Iceland, Reykjavík, Iceland.
- School of Psychology, National Research University Higher School of Economics, Moscow, Russia.
| | - Dejan Draschkow
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, UK.
| |
Collapse
|