1
|
Rasanan AHH, Evans NJ, Fontanesi L, Manning C, Huang-Pollock C, Matzke D, Heathcote A, Rieskamp J, Speekenbrink M, Frank MJ, Palminteri S, Lucas CG, Busemeyer JR, Ratcliff R, Rad JA. Beyond discrete-choice options. Trends Cogn Sci 2024; 28:857-870. [PMID: 39138030 DOI: 10.1016/j.tics.2024.07.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Revised: 07/12/2024] [Accepted: 07/14/2024] [Indexed: 08/15/2024]
Abstract
While decision theories have evolved over the past five decades, their focus has largely been on choices among a limited number of discrete options, even though many real-world situations have a continuous-option space. Recently, theories have attempted to address decisions with continuous-option spaces, and several computational models have been proposed within the sequential sampling framework to explain how we make a decision in continuous-option space. This article aims to review the main attempts to understand decisions on continuous-option spaces, give an overview of applications of these types of decisions, and present puzzles to be addressed by future developments.
Collapse
Affiliation(s)
| | - Nathan J Evans
- School of Psychology, The University of Queensland, St Lucia, QLD 4072, Australia; Department of Psychology, Ludwig Maximilian University of Munich, Munich, Germany
| | - Laura Fontanesi
- Department of Psychology, University of Basel, Missionsstrasse 62A, 4055, Basel, Switzerland
| | | | | | - Dora Matzke
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
| | - Andrew Heathcote
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands; School of Psychological Sciences, University of Newcastle, Newcastle, Australia
| | - Jörg Rieskamp
- Department of Psychology, University of Basel, Missionsstrasse 62A, 4055, Basel, Switzerland
| | | | - Michael J Frank
- Department of Cognitive, Linguistic, and Psychological Sciences and Carney Institute for Brain Science, Brown University, Providence, RI, USA
| | - Stefano Palminteri
- Laboratoire de Neurosciences Cognitives Computationnelles, Institut National de la Santé et Recherche Médicale, Paris, France; Département d'Etudes Cognitives, Ecole Normale Supérieure, PSL Research University, Paris, France
| | | | - Jerome R Busemeyer
- Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN 47405, USA
| | - Roger Ratcliff
- The Ohio State University, 1835 Neil Avenue, Columbus, OH, 43210, USA
| | - Jamal Amani Rad
- Choice Modelling Centre and Institute for Transport Studies, University of Leeds, Leeds LS2 9JT, UK.
| |
Collapse
|
2
|
Morra S, Patella P, Muscella L. Modelling Working Memory Capacity: Is the Magical Number Four, Seven, or Does it Depend on What You Are Counting? J Cogn 2024; 7:60. [PMID: 39035073 PMCID: PMC11259112 DOI: 10.5334/joc.387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Accepted: 07/03/2024] [Indexed: 07/23/2024] Open
Abstract
Limited attentional capacity is essential to working memory. How its limit should be assessed is a debated issue. Five experiments compare Cowan's 4-units and Pascual-Leone's 7-units models of limited working memory capacity, with presentation time and attention to operative schemes as potential explanations of this discrepancy. Experiments 1a-1c used the Compound Stimuli Visual Information (CSVI) task, with long versus brief presentation. Capacity was estimated with the Bose-Einstein model, assuming a different number of attending acts in each condition. Participants' k estimates in both conditions were highly correlated and the means were not different, indicating that the same capacity is assessed in both conditions. Experiments 2 and 3 used the 5000-msec CSVI, and the Visual Array Task (VAT) in two conditions (5000- vs. 120-msec presentation). Capacity in the VAT was estimated with Morey's Bayesian method. Participants' k estimates in both VAT conditions were correlated, but the mean was higher with long presentation, suggesting that the long condition benefits from recoding or chunking. The k estimate in the CSVI correlated with the short VAT and (to a lesser degree in Exp.2) with the long VAT. The mean estimate of k in the CSVI was one unit more than in the short VAT. We conclude that the CSVI and the short VAT tap the same capacity, one unit of which in the short VAT is allocated to an operative scheme; we discuss how Cowan's and Pascual-Leone's views on limited capacity can be reconciled.
Collapse
Affiliation(s)
| | - Paola Patella
- Universitàdi Genova, Italy
- I.I.T. –Center for Neuroscience and Cognitive Systems, Italy
| | | |
Collapse
|
3
|
Bays PM, Schneegans S, Ma WJ, Brady TF. Representation and computation in visual working memory. Nat Hum Behav 2024; 8:1016-1034. [PMID: 38849647 DOI: 10.1038/s41562-024-01871-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Accepted: 03/22/2024] [Indexed: 06/09/2024]
Abstract
The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have advanced our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event. New experimental tools have revealed that observers possess richer information about the uncertainty in their memories and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visuospatial WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research on human WM from a computational perspective, including the neural mechanisms that support it.
Collapse
Affiliation(s)
- Paul M Bays
- Department of Psychology, University of Cambridge, Cambridge, UK
| | | | - Wei Ji Ma
- Center for Neural Science and Department of Psychology, New York University, New York, NY, USA
| | - Timothy F Brady
- Department of Psychology, University of California, San Diego, La Jolla, CA, USA.
| |
Collapse
|
4
|
Macias C, Persaud K. From silos to synergy: Integrating approaches to investigate the role of prior knowledge and expectations on episodic memory. Psychon Bull Rev 2024:10.3758/s13423-024-02505-4. [PMID: 38691223 DOI: 10.3758/s13423-024-02505-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/19/2024] [Indexed: 05/03/2024]
Abstract
Significant progress in the investigation of how prior knowledge influences episodic memory has been made using three sometimes isolated (but not mutually exclusive) approaches: strictly adult behavioral investigations, computational models, and investigations into the development of the system. Here we point out that these approaches are complementary, each approach informs and is informed by the other. Thus, a natural next step for research is to combine all three approaches to further our understanding of the role of prior knowledge in episodic memory. Here we use studies of memory for expectation-congruent and incongruent information from each of these often disparate approaches to illustrate how combining approaches can be used to test and revise theories from the other. This domain is particularly advantageous because it highlights important features of more general memory processes, further differentiates models of memory, and can shed light on developmental change in the memory system. We then present a case study to illustrate the progress that can be made from integrating all three approaches and highlight the need for more endeavors in this vein. As a first step, we also propose a new computational model of memory that takes into account behavioral and developmental factors that can influence prior knowledge and episodic memory interactions. This integrated approach has great potential for offering novel insights into the relationship between prior knowledge and episodic memory, and cognition more broadly.
Collapse
Affiliation(s)
- Carla Macias
- Psychology Department, Rutgers University - Newark, Smith Hall, 101 Warren Street, Newark, NJ, 07102, USA.
| | - Kimele Persaud
- Psychology Department, Rutgers University - Newark, Smith Hall, 101 Warren Street, Newark, NJ, 07102, USA
| |
Collapse
|
5
|
Westebbe L, Liang Y, Blaser E. The Accuracy and Precision of Memory for Natural Scenes: A Walk in the Park. Open Mind (Camb) 2024; 8:131-147. [PMID: 38435706 PMCID: PMC10898787 DOI: 10.1162/opmi_a_00122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Accepted: 01/17/2024] [Indexed: 03/05/2024] Open
Abstract
It is challenging to quantify the accuracy and precision of scene memory because it is unclear what 'space' scenes occupy (how can we quantify error when misremembering a natural scene?). To address this, we exploited the ecologically valid, metric space in which scenes occur and are represented: routes. In a delayed estimation task, participants briefly saw a target scene drawn from a video of an outdoor 'route loop', then used a continuous report wheel of the route to pinpoint the scene. Accuracy was high and unbiased, indicating there was no net boundary extension/contraction. Interestingly, precision was higher for routes that were more self-similar (as characterized by the half-life, in meters, of a route's Multiscale Structural Similarity index), consistent with previous work finding a 'similarity advantage' where memory precision is regulated according to task demands. Overall, scenes were remembered to within a few meters of their actual location.
Collapse
Affiliation(s)
- Leo Westebbe
- Department of Psychology, University of Massachusetts Boston, Boston, MA, USA
| | - Yibiao Liang
- Department of Psychology, University of Massachusetts Boston, Boston, MA, USA
| | - Erik Blaser
- Department of Psychology, University of Massachusetts Boston, Boston, MA, USA
| |
Collapse
|
6
|
Bates CJ, Alvarez GA, Gershman SJ. Scaling models of visual working memory to natural images. COMMUNICATIONS PSYCHOLOGY 2024; 2:3. [PMID: 39242907 DOI: 10.1038/s44271-023-00048-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Accepted: 12/05/2023] [Indexed: 09/09/2024]
Abstract
Over the last few decades, psychologists have developed precise quantitative models of human recall performance in visual working memory (VWM) tasks. However, these models are tailored to a particular class of artificial stimulus displays and simple feature reports from participants (e.g., the color or orientation of a simple object). Our work has two aims. The first is to build models that explain people's memory errors in continuous report tasks with natural images. Here, we use image generation algorithms to generate continuously varying response alternatives that differ from the stimulus image in natural and complex ways, in order to capture the richness of people's stored representations. The second aim is to determine whether models that do a good job of explaining memory errors with natural images also explain errors in the more heavily studied domain of artificial displays with simple items. We find that: (i) features taken from state-of-the-art deep encoders predict trial-level difficulty in natural images better than several reasonable baselines; and (ii) the same visual encoders can reproduce set-size effects and response bias curves in the artificial stimulus domains of orientation and color. Moving forward, our approach offers a scalable way to build a more generalized understanding of VWM representations by combining recent advances in both AI and cognitive modeling.
Collapse
Affiliation(s)
- Christopher J Bates
- Department of Psychology, Harvard University, William James Hall, 33 Kirkland Street, Cambridge, MA, 02138, USA.
| | - George A Alvarez
- Department of Psychology, Harvard University, William James Hall, 33 Kirkland Street, Cambridge, MA, 02138, USA
| | - Samuel J Gershman
- Department of Psychology, Harvard University, William James Hall, 33 Kirkland Street, Cambridge, MA, 02138, USA
| |
Collapse
|
7
|
German JS, Jacobs RA. Implications of capacity-limited, generative models for human vision. Behav Brain Sci 2023; 46:e391. [PMID: 38054373 DOI: 10.1017/s0140525x23001772] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/07/2023]
Abstract
Although discriminative deep neural networks are currently dominant in cognitive modeling, we suggest that capacity-limited, generative models are a promising avenue for future work. Generative models tend to learn both local and global features of stimuli and, when properly constrained, can learn componential representations and response biases found in people's behaviors.
Collapse
Affiliation(s)
- Joseph Scott German
- Department of Cognitive Science, University of California, San Diego, La Jolla, CA, USA
| | - Robert A Jacobs
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY, USA https://www2.bcs.rochester.edu/sites/jacobslab/people.html
| |
Collapse
|
8
|
Hamblin-Frohman Z, Low JX, Becker SI. Attentional prioritisation and facilitation for similar stimuli in visual working memory. PSYCHOLOGICAL RESEARCH 2023; 87:2031-2038. [PMID: 36633707 PMCID: PMC10457231 DOI: 10.1007/s00426-023-01790-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2022] [Accepted: 01/02/2023] [Indexed: 01/13/2023]
Abstract
Visual working memory (VWM) allows for the brief retention of approximately three to four items. Interestingly, when these items are similar to each other in a feature domain, memory recall performance is elevated compared to when they are dissimilar. This similarity benefit is currently not accounted for by models of VWM. Previous research has suggested that this similarity benefit may arise from selective attentional prioritisation in the maintenance phase. However, the similarity effect has not been contrasted under circumstances where dissimilar item types can adequately compete for memory resources. In Experiment 1, similarity benefits were seen for all-similar over all-dissimilar displays. This was also seen in mixed displays, change detection performance was higher when one of the two similar items changed, compared to when the dissimilar item changed. Surprisingly, the similarity effect was stronger in these mixed displays then when comparing the all-similar and all-dissimilar. Experiment 2 investigated this further by examining how attention was allocated in the memory encoding phase via eye movements. Results revealed that attention prioritised similar over dissimilar items in the mixed displays. Similar items were more likely to receive the first fixation and were fixated more often than dissimilar items. Furthermore, dwell times were elongated for dissimilar items, suggesting that encoding was less efficient. These results suggest that there is an attentional strategy towards prioritising similar items over dissimilar items, and that this strategy's influence can be observed in the memory encoding phase.
Collapse
Affiliation(s)
- Zachary Hamblin-Frohman
- School of Psychology, The University of Queensland, 1/18 Archibald Street, West End, QLD, 4101, Australia.
| | - Jia Xuan Low
- School of Psychology, The University of Queensland, 1/18 Archibald Street, West End, QLD, 4101, Australia
| | - Stefanie I Becker
- School of Psychology, The University of Queensland, 1/18 Archibald Street, West End, QLD, 4101, Australia
| |
Collapse
|
9
|
Aujla H. d[Formula: see text]: Sensitivity at the optimal criterion location. Behav Res Methods 2023; 55:2532-2558. [PMID: 36050574 DOI: 10.3758/s13428-022-01913-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/16/2022] [Indexed: 11/08/2022]
Abstract
Signal detection theory (SDT) was developed to provide a measure of the discriminability of a signal against background noise, independently of response bias. However, equal discriminability over a range of bias is only achieved by the traditional signal detection measure d[Formula: see text] under a narrow set of conditions - i.e., binormal noise and signal distributions of equal variance and base rates. In response to observed departures from these conditions, more robust alternative measures of d[Formula: see text] have been developed, including da and, more recently, d[Formula: see text]. Each of these alternatives addresses some, but not all, of the difficulties that arise when the assumptions of SDT are violated. Moreover, none of these measures directly follow from a central idea of discriminability by an observer that adopts a minimize error count (MEC) strategy. I propose a new d[Formula: see text] alternative, d[Formula: see text], that is robust to violations of the standard signal detection assumptions, remains consistent with varying bias, and is grounded in the principle of discriminability following a MEC strategy. Simulations illustrate how d[Formula: see text] is similar to the recently developed d[Formula: see text] when the observer optimizes their criterion placement to minimize the number of errors but, unlike d[Formula: see text], remains consistent irrespective of the observer's criterion placement Moreover, unlike da, d[Formula: see text] reflects changes in discriminability related to base rates of signal vs. noise presentations. The use of d[Formula: see text] also has implications for the interpretation of bias metrics, such as β and c, which are examined at the optimal criterion under a variety of conditions.
Collapse
Affiliation(s)
- Harinder Aujla
- Department of Psychology, University of Winnipeg, Winnipeg, R3B 2E9, Canada.
| |
Collapse
|
10
|
Schaffner J, Bao SD, Tobler PN, Hare TA, Polania R. Sensory perception relies on fitness-maximizing codes. Nat Hum Behav 2023:10.1038/s41562-023-01584-y. [PMID: 37106080 PMCID: PMC10365992 DOI: 10.1038/s41562-023-01584-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Accepted: 03/09/2023] [Indexed: 04/29/2023]
Abstract
Sensory information encoded by humans and other organisms is generally presumed to be as accurate as their biological limitations allow. However, perhaps counterintuitively, accurate sensory representations may not necessarily maximize the organism's chances of survival. To test this hypothesis, we developed a unified normative framework for fitness-maximizing encoding by combining theoretical insights from neuroscience, computer science, and economics. Behavioural experiments in humans revealed that sensory encoding strategies are flexibly adapted to promote fitness maximization, a result confirmed by deep neural networks with information capacity constraints trained to solve the same task as humans. Moreover, human functional MRI data revealed that novel behavioural goals that rely on object perception induce efficient stimulus representations in early sensory structures. These results suggest that fitness-maximizing rules imposed by the environment are applied at early stages of sensory processing in humans and machines.
Collapse
Affiliation(s)
- Jonathan Schaffner
- Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zurich, Switzerland
- Neuroscience Center Zurich, Zurich, Switzerland
| | - Sherry Dongqi Bao
- Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zurich, Switzerland
- Neuroscience Center Zurich, Zurich, Switzerland
| | - Philippe N Tobler
- Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zurich, Switzerland
- Neuroscience Center Zurich, Zurich, Switzerland
| | - Todd A Hare
- Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zurich, Switzerland.
- Neuroscience Center Zurich, Zurich, Switzerland.
| | - Rafael Polania
- Neuroscience Center Zurich, Zurich, Switzerland.
- Decision Neuroscience Lab, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland.
| |
Collapse
|
11
|
No effect of monetary reward in a visual working memory task. PLoS One 2023; 18:e0280257. [PMID: 36649241 PMCID: PMC9844926 DOI: 10.1371/journal.pone.0280257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Accepted: 12/24/2022] [Indexed: 01/18/2023] Open
Abstract
Previous work has shown that humans distribute their visual working memory (VWM) resources flexibly across items: the higher the importance of an item, the better it is remembered. A related, but much less studied question is whether people also have control over the total amount of VWM resource allocated to a task. Here, we approach this question by testing whether increasing monetary incentives results in better overall VWM performance. In three experiments, subjects performed a delayed-estimation task on the Amazon Turk platform. In the first two experiments, four groups of subjects received a bonus payment based on their performance, with the maximum bonus ranging from $0 to $10 between groups. We found no effect of the amount of bonus on intrinsic motivation or on VWM performance in either experiment. In the third experiment, reward was manipulated on a trial-by-trial basis using a within-subjects design. Again, no evidence was found that VWM performance depended on the magnitude of potential reward. These results suggest that encoding quality in visual working memory is insensitive to monetary reward, which has implications for resource-rational theories of VWM.
Collapse
|
12
|
Hahn M, Futrell R, Levy R, Gibson E. A resource-rational model of human processing of recursive linguistic structure. Proc Natl Acad Sci U S A 2022; 119:e2122602119. [PMID: 36260742 PMCID: PMC9618130 DOI: 10.1073/pnas.2122602119] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2021] [Accepted: 08/31/2022] [Indexed: 01/03/2023] Open
Abstract
A major goal of psycholinguistic theory is to account for the cognitive constraints limiting the speed and ease of language comprehension and production. Wide-ranging evidence demonstrates a key role for linguistic expectations: A word's predictability, as measured by the information-theoretic quantity of surprisal, is a major determinant of processing difficulty. But surprisal, under standard theories, fails to predict the difficulty profile of an important class of linguistic patterns: the nested hierarchical structures made possible by recursion in human language. These nested structures are better accounted for by psycholinguistic theories of constrained working memory capacity. However, progress on theory unifying expectation-based and memory-based accounts has been limited. Here we present a unified theory of a rational trade-off between precision of memory representations with ease of prediction, a scaled-up computational implementation using contemporary machine learning methods, and experimental evidence in support of the theory's distinctive predictions. We show that the theory makes nuanced and distinctive predictions for difficulty patterns in nested recursive structures predicted by neither expectation-based nor memory-based theories alone. These predictions are confirmed 1) in two language comprehension experiments in English, and 2) in sentence completions in English, Spanish, and German. More generally, our framework offers computationally explicit theory and methods for understanding how memory constraints and prediction interact in human language comprehension and production.
Collapse
Affiliation(s)
- Michael Hahn
- Department of Linguistics, Stanford University, Stanford, CA 94305
- Collaborative Research Center 1102, Saarland University; Saarland 66123, Germany
| | - Richard Futrell
- Department of Language Science, University of California, Irvine, CA 92697
| | - Roger Levy
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139
| | - Edward Gibson
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139
| |
Collapse
|
13
|
Schach S, Lindner A, Braun DA. Bounded rational decision-making models suggest capacity-limited concurrent motor planning in human posterior parietal and frontal cortex. PLoS Comput Biol 2022; 18:e1010585. [PMID: 36227842 PMCID: PMC9560147 DOI: 10.1371/journal.pcbi.1010585] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Accepted: 09/18/2022] [Indexed: 11/05/2022] Open
Abstract
While traditional theories of sensorimotor processing have often assumed a serial decision-making pipeline, more recent approaches have suggested that multiple actions may be planned concurrently and vie for execution. Evidence for the latter almost exclusively stems from electrophysiological studies in posterior parietal and premotor cortex of monkeys. Here we study concurrent prospective motor planning in humans by recording functional magnetic resonance imaging (fMRI) during a delayed response task engaging movement sequences towards multiple potential targets. We find that also in human posterior parietal and premotor cortex delay activity modulates both with sequence complexity and the number of potential targets. We tested the hypothesis that this modulation is best explained by concurrent prospective planning as opposed to the mere maintenance of potential targets in memory. We devise a bounded rationality model with information constraints that optimally assigns information resources for planning and memory for this task and determine predicted information profiles according to the two hypotheses. When regressing delay activity on these model predictions, we find that the concurrent prospective planning strategy provides a significantly better explanation of the fMRI-signal modulations. Moreover, we find that concurrent prospective planning is more costly and thus limited for most subjects, as expressed by the best fitting information capacities. We conclude that bounded rational decision-making models allow relating both behavior and neural representations to utilitarian task descriptions based on bounded optimal information-processing assumptions. When the future is uncertain, it can be beneficial to concurrently plan several action possibilities in advance. Electrophysiological research found evidence in monkeys that brain regions in posterior parietal and promotor cortex are indeed capable of planning several actions in parallel. We now used fMRI to study brain activity in these brain regions in humans. For our analyses we applied bounded rationality models that optimally assign information resources to fMRI activity in a complex motor planning task. We find that theoretical information costs of concurrent prospective planning explained fMRI activity profiles significantly better than assuming alternative memory-based strategies. Moreover, exploiting the model allowed us to quantify the individual capacity limit for concurrent planning and to relate these individual limits to both subjects’ behavior and to their neural representations of planning.
Collapse
Affiliation(s)
- Sonja Schach
- Institute of Neural Information Processing, University of Ulm, Ulm, Germany
- * E-mail:
| | - Axel Lindner
- Tübingen Center for Mental Health, Department of Psychiatry and Psychotherapy, University of Tübingen, Tübingen, Germany
- Centre of Neurology, Division of Neuropsychology, Hertie-Institute for Clinical Brain Research, University of Tübingen, Tübingen, Germany
| | | |
Collapse
|
14
|
Sims CR, Lerch RA, Tarduno JA, Jacobs RA. Conceptual knowledge shapes visual working memory for complex visual information. Sci Rep 2022; 12:8088. [PMID: 35577845 PMCID: PMC9110428 DOI: 10.1038/s41598-022-12137-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Accepted: 05/05/2022] [Indexed: 11/28/2022] Open
Abstract
Human visual working memory (VWM) is a memory store people use to maintain the visual features of objects and scenes. Although it is obvious that bottom-up information influences VWM, the extent to which top-down conceptual information influences VWM is largely unknown. We report an experiment in which groups of participants were trained in one of two different categories of geologic faults (left/right lateral, or normal/reverse faults), or received no category training. Following training, participants performed a visual change detection task in which category knowledge was irrelevant to the task. Participants were more likely to detect a change in geologic scenes when the changes crossed a trained categorical distinction (e.g., the left/right lateral fault boundary), compared to within-category changes. In addition, participants trained to distinguish left/right lateral faults were more likely to detect changes when the scenes were mirror images along the left/right dimension. Similarly, participants trained to distinguish normal/reverse faults were more likely to detect changes when scenes were mirror images along the normal/reverse dimension. Our results provide direct empirical evidence that conceptual knowledge influences VWM performance for complex visual information. An implication of our results is that cognitive scientists may need to reconceptualize VWM so that it is closer to "conceptual short-term memory".
Collapse
Affiliation(s)
- Chris R Sims
- Department of Cognitive Science, Rensselaer Polytechnic Institute, Troy, NY, 12180, USA
| | - Rachel A Lerch
- Department of Psychology, University of Texas at Austin, Austin, TX, 78712, USA
| | - John A Tarduno
- Department of Earth and Environmental Sciences, University of Rochester, Rochester, NY, 14627, USA
| | - Robert A Jacobs
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY, 14627, USA.
| |
Collapse
|
15
|
Zhang K, Qian J. The role of ensemble average differs in working memory for depth and planar information. J Vis 2022; 22:4. [PMID: 35522260 PMCID: PMC9078066 DOI: 10.1167/jov.22.6.4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2021] [Accepted: 04/06/2022] [Indexed: 11/24/2022] Open
Abstract
The representation of individual planar locations and features stored in working memory can be affected by the average representation. However, less is known about how the average representation affects the short-term storage of depth information. To evaluate the possible different roles of the ensemble average in working memory for planar and depth information, we used mathematical models to fit the data collected from one study on working memory for depth and 12 studies on working memory for planar information. The pattern of recalled depth was well captured by models assuming that there was a probability of reporting the average depth instead of the individual depth, compressing the recalled front-back distance of the stimulus ensemble compared to the perceived distance. However, when modeling the recalled planar information, we found that participants tended to report individual nontarget features when the target was not memorized, and the assumption of reporting average information improves the data fitting only in very few studies. These results provide evidence for our hypothesis that average depth information can be used as a substitution for individual depth information stored in working memory, but for planar visual features, the substitution of target with the average works under a constraint that the average of to-be-remembered features is readily accessible.
Collapse
Affiliation(s)
- Ke Zhang
- Sun Yat-Sen University, Department of Psychology, Guangzhou, China
- Shaoxing University, Center for Brain, Mind, and Education, Shaoxing, China
| | - Jiehui Qian
- Sun Yat-Sen University, Department of Psychology, Guangzhou, China
| |
Collapse
|
16
|
Abstract
A hallmark of adaptation in humans and other animals is our ability to control how we think and behave across different settings. Research has characterized the various forms cognitive control can take-including enhancement of goal-relevant information, suppression of goal-irrelevant information, and overall inhibition of potential responses-and has identified computations and neural circuits that underpin this multitude of control types. Studies have also identified a wide range of situations that elicit adjustments in control allocation (e.g., those eliciting signals indicating an error or increased processing conflict), but the rules governing when a given situation will give rise to a given control adjustment remain poorly understood. Significant progress has recently been made on this front by casting the allocation of control as a decision-making problem. This approach has developed unifying and normative models that prescribe when and how a change in incentives and task demands will result in changes in a given form of control. Despite their successes, these models, and the experiments that have been developed to test them, have yet to face their greatest challenge: deciding how to select among the multiplicity of configurations that control can take at any given time. Here, we will lay out the complexities of the inverse problem inherent to cognitive control allocation, and their close parallels to inverse problems within motor control (e.g., choosing between redundant limb movements). We discuss existing solutions to motor control's inverse problems drawn from optimal control theory, which have proposed that effort costs act to regularize actions and transform motor planning into a well-posed problem. These same principles may help shed light on how our brains optimize over complex control configuration, while providing a new normative perspective on the origins of mental effort.
Collapse
|
17
|
Grujic N, Brus J, Burdakov D, Polania R. Rational inattention in mice. SCIENCE ADVANCES 2022; 8:eabj8935. [PMID: 35245128 PMCID: PMC8896787 DOI: 10.1126/sciadv.abj8935] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2023]
Abstract
Behavior exhibited by humans and other organisms is generally inconsistent and biased and, thus, is often labeled irrational. However, the origins of this seemingly suboptimal behavior remain elusive. We developed a behavioral task and normative framework to reveal how organisms should allocate their limited processing resources such that sensory precision and its related metabolic investment are balanced to guarantee maximal utility. We found that mice act as rational inattentive agents by adaptively allocating their sensory resources in a way that maximizes reward consumption in previously unexperienced stimulus-reward association environments. Unexpectedly, perception of commonly occurring stimuli was relatively imprecise; however, this apparent statistical fallacy implies "awareness" and efficient adaptation to their neurocognitive limitations. Arousal systems carry reward distribution information of sensory signals, and distributional reinforcement learning mechanisms regulate sensory precision via top-down normalization. These findings reveal how organisms efficiently perceive and adapt to previously unexperienced environmental contexts within the constraints imposed by neurobiology.
Collapse
Affiliation(s)
- Nikola Grujic
- Institute for Neuroscience, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland
- Neuroscience Center Zürich, Zurich, Switzerland
| | - Jeroen Brus
- Neuroscience Center Zürich, Zurich, Switzerland
- Decision Neuroscience Lab, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland
| | - Denis Burdakov
- Institute for Neuroscience, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland
- Neuroscience Center Zürich, Zurich, Switzerland
- Corresponding author. (R.P.); (D.B.)
| | - Rafael Polania
- Neuroscience Center Zürich, Zurich, Switzerland
- Decision Neuroscience Lab, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland
- Corresponding author. (R.P.); (D.B.)
| |
Collapse
|
18
|
Effects of item distinctiveness on the retrieval of objects and object-location bindings from visual working memory. Atten Percept Psychophys 2022; 84:2236-2254. [PMID: 35199322 DOI: 10.3758/s13414-022-02451-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/25/2022] [Indexed: 11/08/2022]
Abstract
Visual working memory (VWM) is prone to interference from stored items competing for its limited capacity. Distinctiveness or similarity of the items is acknowledged to affect this competition, such that poor item distinctiveness causes a failure to discriminate between items sharing common features. In three experiments, we studied how the distinctiveness of studied real-world objects (i.e., whether the objects belong to the same or different basic categories) affects the retrieval of objects themselves (simple recognition) and object-location conjunctions (information about which object was where in a display, cued recall). In Experiments 1 and 2, we found that distinctiveness did not affect memories for objects or for locations, but low-distinctive objects were more frequently reported at "swapped" locations that originally contained other objects, showing object-location memory swaps. In Experiments 3 we found that observers swapped the location of a tested object with another object from the same category more frequently than with any of the objects from another category. This suggests that more similar studied objects cause more retrieval competition in object-location judgments than in simple recognition. Additionally, we discuss a possible role of categorical labeling of locations that can support object-location retrieval when the studied objects are highly distinct.
Collapse
|
19
|
Martin A, Becker SI. A relational account of visual short-term memory (VSTM). Cortex 2021; 144:151-167. [PMID: 34666299 DOI: 10.1016/j.cortex.2021.08.013] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2020] [Revised: 10/31/2020] [Accepted: 08/18/2021] [Indexed: 10/20/2022]
Abstract
Visual short-term memory (VSTM) is an important resource that allows temporarily storing visual information. Current theories posit that elementary features (e.g., red, green) are encoded and stored independently of each other in VSTM. However, they have difficulty explaining the similarity effect, that similar items can be remembered better than dissimilar items. In Experiment 1, we tested (N = 20) whether the similarity effect may be due to storing items in a context-dependent manner in VSTM (e.g., as the reddest/yellowest item). In line with a relational account of VSTM, we found that the similarity effect is not due to feature similarity, but to an enhanced sensitivity for detecting changes when the relative colour of a to-be-memorised item changes (e.g., from reddest to not-reddest item; than when an item underwent the same change but retained its relative colour; e.g., still reddest). Experiment 2 (N = 20) showed that VSTM load, as indexed by the CDA amplitude in the EEG, was smaller when the colours were ordered so that they all had the same relationship than when the same colours were out-of-order, requiring encoding different relative colours. With this, we report two new effects in VSTM - a relational detection advantage that describes an enhanced sensitivity to relative changes in change detection, and a relational CDA effect, which reflects that VSTM load, as indexed by the CDA, scales with the number of (different) relative features between the memory items. These findings support a relational account of VSTM and question the view that VSTM stores features such as colours independently of each other.
Collapse
Affiliation(s)
- Aimee Martin
- The University of Queensland, School of Psychology, QLD, Brisbane, Australia.
| | - Stefanie I Becker
- The University of Queensland, School of Psychology, QLD, Brisbane, Australia.
| |
Collapse
|
20
|
Semantic influence on visual working memory of object identity and location. Cognition 2021; 217:104891. [PMID: 34481197 DOI: 10.1016/j.cognition.2021.104891] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2020] [Revised: 08/23/2021] [Accepted: 08/24/2021] [Indexed: 11/22/2022]
Abstract
Does semantic information-in particular, regularities in category membership across objects-influence visual working memory (VWM) processing? We predict that the answer is "yes". Four experiments evaluating this prediction are reported. Experimental stimuli were images of real-world objects arranged in either one or two spatial clusters. On coherent trials, all objects belonging to a cluster also belonged to the same category. On incoherent trials, at least one cluster contained objects from different categories. Experiments using a change-detection paradigm (Experiments 1-3) and an experiment in which participants recalled the locations of objects in a scene (Experiment 4) yielded the same result: participants showed better memory performance on coherent trials than on incoherent trials. Taken as a whole, these experiments provide the best (perhaps only) data to date demonstrating that statistical regularities in semantic category membership improve VWM performance. Because a conventional perspective in cognitive science regards VWM as being sensitive solely to bottom-up visual properties of objects (e.g., shape, color, orientation), our results indicate that cognitive science may need to modify its conceptualization of VWM so that it is closer to "conceptual short-term memory", a short-term memory store representing current stimuli and their associated concepts (Potter, 1993, 2012).
Collapse
|
21
|
Lo AW, Zhang R. The evolutionary origin of Bayesian heuristics and finite memory. iScience 2021; 24:102853. [PMID: 34381977 PMCID: PMC8340130 DOI: 10.1016/j.isci.2021.102853] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2021] [Revised: 06/23/2021] [Accepted: 07/09/2021] [Indexed: 11/26/2022] Open
Abstract
Bayes' rule is a fundamental principle that has been applied across multiple disciplines. However, few studies have addressed its origin as a cognitive strategy or the underlying basis for generalization from a small sample. Using a simple binary choice model subject to natural selection, we derive Bayesian inference as an adaptive behavior under certain stochastic environments. Such behavior emerges purely through the forces of evolution, despite the fact that our population consists of mindless individuals without any ability to reason, act strategically, or accurately encode or infer environmental states probabilistically. In addition, three specific environments favor the emergence of finite memory—those that are Markov, nonstationary, and environments where sampling contains too little or too much information about local conditions. These results provide an explanation for several known phenomena in human cognition, including deviations from the optimal Bayesian strategy and finite memory beyond resource constraints. Evolution leads to behaviors that resemble Bayesian inference Finite memory arises in nonstationary environments beyond resource constraints Intelligence is what advances fitness from the evolutionary lens This may provide insights for the quest toward artificial general intelligence
Collapse
Affiliation(s)
- Andrew W Lo
- MIT Laboratory for Financial Engineering, Cambridge, MA 02142, USA.,MIT Sloan School of Management, Cambridge, MA 02142, USA.,MIT Computer Science and Artificial Intelligence Laboratory, Cambridge, MA 02142, USA.,Santa Fe Institute, Santa Fe, NM 87501, USA
| | - Ruixun Zhang
- MIT Laboratory for Financial Engineering, Cambridge, MA 02142, USA
| |
Collapse
|
22
|
Gershman SJ, Lai L. The Reward-Complexity Trade-off in Schizophrenia. COMPUTATIONAL PSYCHIATRY (CAMBRIDGE, MASS.) 2021; 5:38-53. [PMID: 38773995 PMCID: PMC11104411 DOI: 10.5334/cpsy.71] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/26/2021] [Accepted: 05/05/2021] [Indexed: 11/20/2022]
Abstract
Action selection requires a policy that maps states of the world to a distribution over actions. The amount of memory needed to specify the policy (the policy complexity) increases with the state-dependence of the policy. If there is a capacity limit for policy complexity, then there will also be a trade-off between reward and complexity, since some reward will need to be sacrificed in order to satisfy the capacity constraint. This paper empirically characterizes the trade-off between reward and complexity for both schizophrenia patients and healthy controls. Schizophrenia patients adopt lower complexity policies on average, and these policies are more strongly biased away from the optimal reward-complexity trade-off curve compared to healthy controls. However, healthy controls are also biased away from the optimal trade-off curve, and both groups appear to lie on the same empirical trade-off curve. We explain these findings using a cost-sensitive actor-critic model. Our empirical and theoretical results shed new light on cognitive effort abnormalities in schizophrenia.
Collapse
Affiliation(s)
- Samuel J. Gershman
- Department of Psychology and Center for Brain Science, Harvard University, US
- Center for Brains, Minds and Machines, MIT, US
| | - Lucy Lai
- Program in Neuroscience, Harvard University, US
| |
Collapse
|
23
|
Bates CJ, Jacobs RA. Optimal attentional allocation in the presence of capacity constraints in uncued and cued visual search. J Vis 2021; 21:3. [PMID: 33944906 PMCID: PMC8107488 DOI: 10.1167/jov.21.5.3] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2020] [Accepted: 03/09/2021] [Indexed: 11/24/2022] Open
Abstract
The vision sciences literature contains a large diversity of experimental and theoretical approaches to the study of visual attention. We argue that this diversity arises, at least in part, from the field's inability to unify differing theoretical perspectives. In particular, the field has been hindered by a lack of a principled formal framework for simultaneously thinking about both optimal attentional processing and capacity-limited attentional processing, where capacity is limited in a general, task-independent manner. Here, we supply such a framework based on rate-distortion theory (RDT) and optimal lossy compression. Our approach defines Bayes-optimal performance when an upper limit on information processing rate is imposed. In this article, we compare Bayesian and RDT accounts in both uncued and cued visual search tasks. We start by highlighting a typical shortcoming of unlimited-capacity Bayesian models that is not shared by RDT models, namely, that they often overestimate task performance when information-processing demands are increased. Next, we reexamine data from two cued-search experiments that have previously been modeled as the result of unlimited-capacity Bayesian inference and demonstrate that they can just as easily be explained as the result of optimal lossy compression. To model cued visual search, we introduce the concept of a "conditional communication channel." This simple extension generalizes the lossy-compression framework such that it can, in principle, predict optimal attentional-shift behavior in any kind of perceptual task, even when inputs to the model are raw sensory data such as image pixels. To demonstrate this idea's viability, we compare our idealized model of cued search, which operates on a simplified abstraction of the stimulus, to a deep neural network version that performs approximately optimal lossy compression on the real (pixel-level) experimental stimuli.
Collapse
Affiliation(s)
| | - Robert A Jacobs
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY, USA
| |
Collapse
|
24
|
Norris D, Kalm K. Chunking and data compression in verbal short-term memory. Cognition 2021; 208:104534. [PMID: 33360054 DOI: 10.1016/j.cognition.2020.104534] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2020] [Revised: 11/27/2020] [Accepted: 12/02/2020] [Indexed: 11/24/2022]
Abstract
Short-term verbal memory is improved when words can be chunked into larger units. Miller (1956) suggested that the capacity of verbal short-term memory is determined by the number of chunks that can be stored in memory, rather than by the number of items or the amount of information. But how does the improvement due to chunking come about, and is memory really determined by the number of chunks? One possibility is that chunking is a form of data compression. It allows more information to be stored in the available capacity. An alternative is that chunking operates primarily by redintegration. Chunks exist only in long-term memory, and enable the corresponding items in short-term memory to be reconstructed more reliably from a degraded trace. We review the data favoring each of these views and discuss the implications of treating chunking as data compression. Contrary to Miller, we suggest that memory capacity is primarily determined both by the amount of information that can be stored but also by the underlying representational vocabulary of the memory system. Given the limitations on the representations that can be stored in verbal short-term memory, chunking can sometimes allow the information capacity of short-term memory to be exploited more efficiently. (202 words).
Collapse
Affiliation(s)
- Dennis Norris
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK.
| | - Kristjan Kalm
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK
| |
Collapse
|
25
|
Buss AT, Magnotta VA, Penny W, Schöner G, Huppert TJ, Spencer JP. How do neural processes give rise to cognition? Simultaneously predicting brain and behavior with a dynamic model of visual working memory. Psychol Rev 2021; 128:362-395. [PMID: 33570976 PMCID: PMC11327926 DOI: 10.1037/rev0000264] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
There is consensus that activation within distributed functional brain networks underlies human thought. The impact of this consensus is limited, however, by a gap that exists between data-driven correlational analyses that specify where functional brain activity is localized using functional magnetic resonance imaging (fMRI), and neural process accounts that specify how neural activity unfolds through time to give rise to behavior. Here, we show how an integrative cognitive neuroscience approach may bridge this gap. In an exemplary study of visual working memory, we use multilevel Bayesian statistics to demonstrate that a neural dynamic model simultaneously explains behavioral data and predicts localized patterns of brain activity, outperforming standard analytic approaches to fMRI. The model explains performance on both correct trials and incorrect trials where errors in change detection emerge from neural fluctuations amplified by neural interaction. Critically, predictions of the model run counter to cognitive theories of the origin of errors in change detection. Results reveal neural patterns predicted by the model within regions of the dorsal attention network that have been the focus of much debate. The model-based analysis suggests that key areas in the dorsal attention network such as the intraparietal sulcus play a central role in change detection rather than working memory maintenance, counter to previous interpretations of fMRI studies. More generally, the integrative cognitive neuroscience approach used here establishes a framework for directly testing theories of cognitive and brain function using the combined power of behavioral and fMRI data. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
Collapse
Affiliation(s)
- Aaron T Buss
- Department of Psychology, University of Tennessee, Knoxville
| | | | - Will Penny
- School of Psychology, University of East Anglia
| | | | | | | |
Collapse
|
26
|
Elliott MA, Nothelfer C, Xiong C, Szafir DA. A Design Space of Vision Science Methods for Visualization Research. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2021; 27:1117-1127. [PMID: 33090954 DOI: 10.1109/tvcg.2020.3029413] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
A growing number of efforts aim to understand what people see when using a visualization. These efforts provide scientific grounding to complement design intuitions, leading to more effective visualization practice. However, published visualization research currently reflects a limited set of available methods for understanding how people process visualized data. Alternative methods from vision science offer a rich suite of tools for understanding visualizations, but no curated collection of these methods exists in either perception or visualization research. We introduce a design space of experimental methods for empirically investigating the perceptual processes involved with viewing data visualizations to ultimately inform visualization design guidelines. This paper provides a shared lexicon for facilitating experimental visualization research. We discuss popular experimental paradigms, adjustment types, response types, and dependent measures used in vision science research, rooting each in visualization examples. We then discuss the advantages and limitations of each technique. Researchers can use this design space to create innovative studies and progress scientific understanding of design choices and evaluations in visualization. We highlight a history of collaborative success between visualization and vision science research and advocate for a deeper relationship between the two fields that can elaborate on and extend the methodological design space for understanding visualization and vision.
Collapse
|
27
|
The role of working memory in long-term learning: Implications for childhood development. PSYCHOLOGY OF LEARNING AND MOTIVATION 2021. [DOI: 10.1016/bs.plm.2021.02.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
28
|
Lai L, Gershman SJ. Policy compression: An information bottleneck in action selection. PSYCHOLOGY OF LEARNING AND MOTIVATION 2021. [DOI: 10.1016/bs.plm.2021.02.004] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
29
|
Thönes S, Arnau S, Wascher E, Schneider D. Boosting working memory with accelerated clocks. Neuroimage 2020; 226:117601. [PMID: 33249214 DOI: 10.1016/j.neuroimage.2020.117601] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2020] [Accepted: 11/23/2020] [Indexed: 10/22/2022] Open
Abstract
Our perception of time varies with the degree of cognitive engagement in tasks. The perceived passage of time accelerates while working on demanding tasks, whereas time appears to drag during boring situations. Our experiment aimed at investigating whether this relationship is mutual: Can manipulated announcements of elapsed time systematically affect the attentional resources applied to a cognitive task? We measured behavioral performance and the EEG in a whole report working memory paradigm with six items of different colors that each had to be reported after a short delay period. The 32 participants were informed about the current time after each 20 trials, while the clock was running at either 100% (normal), 120% (fast), or 80% (slow) of normal clock speed depending on the experimental block. The mean number of correctly reported colors per trial was significantly increased in the fast as compared to the slow and normal clock conditions. In the EEG, we focused on neural oscillations during working memory encoding and storage. As an electrophysiological correlate of task engagement, frontal theta power during the storage interval was increased in the fast clock condition. Also, the power of frontal theta oscillations predicted the number of correctly reported colors on a single-trial basis. This shows that a covert manipulation of clock speed can lead to an improvement in cognitive performance, presumably mediated by a higher allocation of attentional resources resulting from an adaptation of the subjective passage of time during an experiment.
Collapse
Affiliation(s)
- Sven Thönes
- Experimental Psychology, Johannes Gutenberg-University Mainz, Mainz, Germany; Leibniz Research Centre for Working Environment and Human Factors, Dortmund, Germany
| | - Stefan Arnau
- Leibniz Research Centre for Working Environment and Human Factors, Dortmund, Germany
| | - Edmund Wascher
- Leibniz Research Centre for Working Environment and Human Factors, Dortmund, Germany
| | - Daniel Schneider
- Leibniz Research Centre for Working Environment and Human Factors, Dortmund, Germany.
| |
Collapse
|
30
|
Gershman SJ. Origin of perseveration in the trade-off between reward and complexity. Cognition 2020; 204:104394. [DOI: 10.1016/j.cognition.2020.104394] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2020] [Revised: 06/29/2020] [Accepted: 06/30/2020] [Indexed: 11/25/2022]
|
31
|
Abstract
A key challenge for the visual system entails the extraction of constant properties of objects from sensory information that varies moment by moment due to changes in viewing conditions. Although successful performance in constancy tasks requires cooperation between perception and working memory, the function of the memory system has been under-represented in recent material perception literature. Here, we addressed the limits of material constancy by elucidating if and how working memory is involved in constancy tasks by using a variety of material stimuli, such as metals, glass, and translucent objects. We conducted experiments with a simultaneous and a successive matching-to-sample paradigm in which participants matched the perceived material properties of objects with or without a temporal delay under varying illumination contexts. The current study combined a detailed analysis of matching errors, data on the strategy use obtained via a self-report questionnaire, and the statistical image analysis of diagnostic image cues used for material discrimination. We found a comparable material constancy between simultaneous and successive matching conditions, and it was suggested that, in both matching conditions, participants used similar information processing strategies for the discrimination of materials. The study provides converging evidence on the critical role of working memory in material constancy, where working memory serves as a shared processing bottleneck that constrains both simultaneous and successive material constancy.
Collapse
Affiliation(s)
- Hiroyuki Tsuda
- Keio Advanced Research Center, Keio University, Tokyo, Japan
| | - Munendo Fujimichi
- Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan
| | | | - Jun Saiki
- Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan
| |
Collapse
|
32
|
Nagy DG, Török B, Orbán G. Optimal forgetting: Semantic compression of episodic memories. PLoS Comput Biol 2020; 16:e1008367. [PMID: 33057380 PMCID: PMC7591090 DOI: 10.1371/journal.pcbi.1008367] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2020] [Revised: 10/27/2020] [Accepted: 09/03/2020] [Indexed: 11/26/2022] Open
Abstract
It has extensively been documented that human memory exhibits a wide range of systematic distortions, which have been associated with resource constraints. Resource constraints on memory can be formalised in the normative framework of lossy compression, however traditional lossy compression algorithms result in qualitatively different distortions to those found in experiments with humans. We argue that the form of distortions is characteristic of relying on a generative model adapted to the environment for compression. We show that this semantic compression framework can provide a unifying explanation of a wide variety of memory phenomena. We harness recent advances in learning deep generative models, that yield powerful tools to approximate generative models of complex data. We use three datasets, chess games, natural text, and hand-drawn sketches, to demonstrate the effects of semantic compression on memory performance. Our model accounts for memory distortions related to domain expertise, gist-based distortions, contextual effects, and delayed recall.
Collapse
Affiliation(s)
- David G. Nagy
- Computational Systems Neuroscience Lab, Wigner Research Centre for Physics, Budapest, Hungary
- Institute of Physics, Eötvös Loránd University, Budapest, Hungary
| | - Balázs Török
- Computational Systems Neuroscience Lab, Wigner Research Centre for Physics, Budapest, Hungary
- Department of Cognitive Science, Budapest University of Technology and Economics, Budapest, Hungary
| | - Gergő Orbán
- Computational Systems Neuroscience Lab, Wigner Research Centre for Physics, Budapest, Hungary
| |
Collapse
|
33
|
Cheyette SJ, Piantadosi ST. A unified account of numerosity perception. Nat Hum Behav 2020; 4:1265-1272. [PMID: 32929205 DOI: 10.1038/s41562-020-00946-0] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2019] [Accepted: 08/07/2020] [Indexed: 01/29/2023]
Abstract
People can identify the number of objects in sets of four or fewer items with near-perfect accuracy but exhibit linearly increasing error for larger sets. Some researchers have taken this discontinuity as evidence of two distinct representational systems. Here, we present a mathematical derivation showing that this behaviour is an optimal representation of cardinalities under a limited informational capacity, indicating that this behaviour can emerge from a single system. Our derivation predicts how the amount of information accessible to viewers should influence the perception of quantity for both large and small sets. In a series of four preregistered experiments (N = 100 each), we varied the amount of information accessible to participants in number estimation. We find tight alignment between the model and human performance for both small and large quantities, implicating efficient representation as the common origin behind key phenomena of human and animal numerical cognition.
Collapse
|
34
|
Yeon J, Rahnev D. The suboptimality of perceptual decision making with multiple alternatives. Nat Commun 2020; 11:3857. [PMID: 32737317 PMCID: PMC7395091 DOI: 10.1038/s41467-020-17661-z] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2019] [Accepted: 07/08/2020] [Indexed: 11/23/2022] Open
Abstract
It is becoming widely appreciated that human perceptual decision making is suboptimal but the nature and origins of this suboptimality remain poorly understood. Most past research has employed tasks with two stimulus categories, but such designs cannot fully capture the limitations inherent in naturalistic perceptual decisions where choices are rarely between only two alternatives. We conduct four experiments with tasks involving multiple alternatives and use computational modeling to determine the decision-level representation on which the perceptual decisions are based. The results from all four experiments point to the existence of robust suboptimality such that most of the information in the sensory representation is lost during the transformation to a decision-level representation. These results reveal severe limits in the quality of decision-level representations for multiple alternatives and have strong implications about perceptual decision making in naturalistic settings.
Collapse
Affiliation(s)
- Jiwon Yeon
- School of Psychology, Georgia Institute of Technology, Atlanta, GA, USA.
| | - Dobromir Rahnev
- School of Psychology, Georgia Institute of Technology, Atlanta, GA, USA.
| |
Collapse
|
35
|
Affiliation(s)
- Tom Mercer
- Department of Psychology, Centre for Psychological Research, University of Wolverhampton, Wolverhampton, UK
| | - Emma Barker
- Department of Psychology, Centre for Psychological Research, University of Wolverhampton, Wolverhampton, UK
| |
Collapse
|
36
|
Abstract
Resource rationality holds great promise as a unifying principle across theories in neuroscience, cognitive science, and economics. The target article clearly lays out this potential for unification. However, resource-rational models are more diverse and less easily unified than might appear from the target article. Here, we explore some of that diversity.
Collapse
|
37
|
The importance of constraints on constraints. Behav Brain Sci 2020; 43:e3. [PMID: 32159481 DOI: 10.1017/s0140525x19001572] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
The "resource-rational" approach is ambitious and worthwhile. A shortcoming of the proposed approach is that it fails to constrain what counts as a constraint. As a result, constraints used in different cognitive domains often have nothing in common. We describe an alternative framework that satisfies many of the desiderata of the resource-rational approach, but in a more disciplined manner.
Collapse
|
38
|
Abstract
Previous studies have shown that when there are statistical regularities in the items stored in visual working memory, the responses are biased toward the ensemble average. This statistical-regularity-induced bias could happen in two ways: (1) a target bias, where the individual memory representations are pulled toward the ensemble average; or (2) a strategic guess, for items that are not memorized, other information in the ensemble (e.g., another item) is reported as a substitute. Here, these two mechanisms are distinguished on the basis of a three-part model (target responses + swap responses + random guesses; e.g., Bays, Catalao, & Husain, 2009, Journal of Vision, 9, 7). The strategic guess is operationalized as swap responses, whereas the target bias is reflected by a bias parameter in the target responses. This model was applied on 8 data sets (22 observers each). In this model, contributions of target biases and strategic guesses can be clearly distinguished from each other because they lead to distinctive patterns in the distribution of responses. In the present results, strategic guesses always contributed substantially to the statistical-regularity-induced biases, whereas target biases were limited to specific conditions. All in all, the Bayesian inference in visual working memory is much more limited than what is previously advocated.
Collapse
|
39
|
Jacobs RA, Xu C. Can multisensory training aid visual learning? A computational investigation. J Vis 2019; 19:1. [PMID: 31480074 DOI: 10.1167/19.11.1] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Although real-world environments are often multisensory, visual scientists typically study visual learning in unisensory environments containing visual signals only. Here, we use deep or artificial neural networks to address the question, Can multisensory training aid visual learning? We examine a network's internal representations of objects based on visual signals in two conditions: (a) when the network is initially trained with both visual and haptic signals, and (b) when it is initially trained with visual signals only. Our results demonstrate that a network trained in a visual-haptic environment (in which visual, but not haptic, signals are orientation-dependent) tends to learn visual representations containing useful abstractions, such as the categorical structure of objects, and also learns representations that are less sensitive to imaging parameters, such as viewpoint or orientation, that are irrelevant for object recognition or classification tasks. We conclude that researchers studying perceptual learning in vision-only contexts may be overestimating the difficulties associated with important perceptual learning problems. Although multisensory perception has its own challenges, perceptual learning can become easier when it is considered in a multisensory setting.
Collapse
Affiliation(s)
- Robert A Jacobs
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY, USA
| | - Chenliang Xu
- Department of Computer Science, University of Rochester, Rochester, NY, USA
| |
Collapse
|
40
|
Peters B, Rahm B, Kaiser J, Bledowski C. Differential trajectories of memory quality and guessing across sequential reports from working memory. J Vis 2019; 19:3. [PMID: 31287857 DOI: 10.1167/19.7.3] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Working memory enables the storage of few items for a short period of time. Previous research has shown that items in working memory cannot be accessed equally well, indicating that they are held in at least two different states with different capacity limitations. However, it is unclear whether differences between states are due to limitations of the number of items that can be stored, or the quality with which items are stored. We employed a sequential whole-report procedure where participants reported the remembered orientation of each of two or four encoded Gabor patches. In addition, they rated their memory confidence prior to each report. Participants performed 600 trials per condition, allowing us to obtain reliable subjective ratings and estimates of precision, guessing, and misreport using a mixture model, separately for each sequential report. Different measures of memory quality consistently showed discontinuous trajectories across reports with a steep drop from the first to the second remembered item but only slight decreases thereafter. In contrast, both reported and modeled guessing changed continuously across reports. Our results support the notion of two states in working memory and show that they are distinguished by memory quality rather than quantity.
Collapse
Affiliation(s)
- Benjamin Peters
- Institute of Medical Psychology, Goethe University, Frankfurt am Main, Germany.,Zuckerman Institute, Columbia University, New York, USA
| | - Benjamin Rahm
- Medical Psychology and Medical Sociology, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Jochen Kaiser
- Institute of Medical Psychology, Goethe University, Frankfurt am Main, Germany
| | - Christoph Bledowski
- Institute of Medical Psychology, Goethe University, Frankfurt am Main, Germany
| |
Collapse
|
41
|
Bates CJ, Lerch RA, Sims CR, Jacobs RA. Adaptive allocation of human visual working memory capacity during statistical and categorical learning. J Vis 2019; 19:11. [PMID: 30802280 DOI: 10.1167/19.2.11] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Human brains are finite, and thus have bounded capacity. An efficient strategy for a capacity-limited agent is to continuously adapt by dynamically reallocating capacity in a task-dependent manner. Here we study this strategy in the context of visual working memory (VWM). People use their VWM stores to remember visual information over seconds or minutes. However, their memory performances are often error-prone, presumably due to VWM capacity limits. We hypothesize that people attempt to be flexible and robust by strategically reallocating their limited VWM capacity based on two factors: (a) the statistical regularities (e.g., stimulus feature means and variances) of the to-be-remembered items, and (b) the requirements of the task that they are attempting to perform. The latter specifies, for example, which types of errors are costly versus irrelevant for task performance. These hypotheses are formalized within a normative computational modeling framework based on rate-distortion theory, an extension of conventional Bayesian approaches that uses information theory to study rate-limited (or capacity-limited) processes. Using images of plants that are naturalistic and precisely controlled, we carried out two sets of experiments. Experiment 1 found that when a stimulus dimension (the widths of plants' leaves) was assigned a distribution, subjects adapted their VWM performances based on this distribution. Experiment 2 found that when one stimulus dimension (e.g., leaf width) was relevant for distinguishing plant categories but another dimension (leaf angle) was irrelevant, subjects' responses in a memory task became relatively more sensitive to the relevant stimulus dimension. Together, these results illustrate the task-dependent robustness of VWM, thereby highlighting the dependence of memory on learning.
Collapse
Affiliation(s)
- Christopher J Bates
- Department of Brain & Cognitive Sciences, University of Rochester, Rochester, NY, USA
| | - Rachel A Lerch
- Cognitive Science Department, Rensselaer Polytechnic Institute, Troy, NY, USA
| | - Chris R Sims
- Cognitive Science Department, Rensselaer Polytechnic Institute, Troy, NY, USA
| | - Robert A Jacobs
- Department of Brain & Cognitive Sciences, University of Rochester, Rochester, NY, USA
| |
Collapse
|
42
|
Nassar MR, Helmers JC, Frank MJ. Chunking as a rational strategy for lossy data compression in visual working memory. Psychol Rev 2019; 125:486-511. [PMID: 29952621 DOI: 10.1037/rev0000101] [Citation(s) in RCA: 49] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The nature of capacity limits for visual working memory has been the subject of an intense debate that has relied on models that assume items are encoded independently. Here we propose that instead, similar features are jointly encoded through a "chunking" process to optimize performance on visual working memory tasks. We show that such chunking can: (a) facilitate performance improvements for abstract capacity-limited systems, (b) be optimized through reinforcement, (c) be implemented by center-surround dynamics, and (d) increase effective storage capacity at the expense of recall precision. Human performance on a variant of a canonical working memory task demonstrated performance advantages, precision detriments, interitem dependencies, and trial-to-trial behavioral adjustments diagnostic of performance optimization through center-surround chunking. Models incorporating center-surround chunking provided a better quantitative description of human performance in our study as well as in a meta-analytic dataset, and apparent differences in working memory capacity across individuals were attributable to individual differences in the implementation of chunking. Our results reveal a normative rationale for center-surround connectivity in working memory circuitry, call for reevaluation of memory performance differences that have previously been attributed to differences in capacity, and support a more nuanced view of visual working memory capacity limitations: strategic tradeoff between storage capacity and memory precision through chunking contribute to flexible capacity limitations that include both discrete and continuous aspects. (PsycINFO Database Record
Collapse
Affiliation(s)
- Matthew R Nassar
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown Institute for Brain Science, Brown University
| | - Julie C Helmers
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown Institute for Brain Science, Brown University
| | - Michael J Frank
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown Institute for Brain Science, Brown University
| |
Collapse
|
43
|
Brady TF, Störmer VS, Shafer-Skelton A, Williams JR, Chapman AF, Schill HM. Scaling up visual attention and visual working memory to the real world. PSYCHOLOGY OF LEARNING AND MOTIVATION 2019. [DOI: 10.1016/bs.plm.2019.03.001] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/16/2023]
|
44
|
Che X, Ding X, Ling X, Wang H, Gu Y, Li S. Does maintaining bindings in visual working memory require more attention than maintaining features? Memory 2018; 27:729-738. [PMID: 30580657 DOI: 10.1080/09658211.2018.1561894] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Whether more attention is required for maintaining bindings than maintaining features in visual working memory (VWM) remains an open question. If maintaining bindings in VWM does not require more attention than maintaining features, is it related to the stability of binding representations? In this study, we explored whether maintaining bindings requires more attention than maintaining features for similar and dissimilar objects by inserting a feature report task into the maintenance phase of VWM in Experiments 1 and 2. We also investigated whether the effect of similarity on the attentional requirement for maintaining bindings and features is due to the stability of VWM representations by inserting a suffix during the maintenance phase of VWM in Experiment 3. The results showed that when object-based attention was consumed, bindings were more impaired than features for dissimilar objects but not for similar objects. We also found that the bindings of similar objects were less interfered by the suffix than those of dissimilar objects. Our findings suggest that maintaining bindings does not require more attention than maintaining features when the binding representations are stable in VWM and similarity improves the stability of binding representations.
Collapse
Affiliation(s)
- Xiaowei Che
- a Department of Psychology , Shandong Normal University , Jinan , People's Republic of China
| | - Xiaowei Ding
- b Department of Psychology, Guangdong Provincial Key Laboratory of Social Cognitive Neuroscience and Mental Health, & Guangdong Provincial Key Laboratory of Brain Function and Disease , Sun Yat-Sen University , Guangzhou , People's Republic of China
| | - Xiaoli Ling
- a Department of Psychology , Shandong Normal University , Jinan , People's Republic of China
| | - Hailing Wang
- a Department of Psychology , Shandong Normal University , Jinan , People's Republic of China
| | - Yuanyuan Gu
- c Research Centre for Brain Function and Psychological Science , Shenzhen University , Shenzhen , People's Republic of China
| | - Shouxin Li
- a Department of Psychology , Shandong Normal University , Jinan , People's Republic of China
| |
Collapse
|
45
|
Comparing the Visual Representations and Performance of Humans and Deep Neural Networks. CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE 2018. [DOI: 10.1177/0963721418801342] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Although deep neural networks (DNNs) are state-of-the-art artificial intelligence systems, it is unclear what insights, if any, they provide about human intelligence. We address this issue in the domain of visual perception. After briefly describing DNNs, we provide an overview of recent results comparing human visual representations and performance with those of DNNs. In many cases, DNNs acquire visual representations and processing strategies that are very different from those used by people. We conjecture that there are at least two factors preventing them from serving as better psychological models. First, DNNs are currently trained with impoverished data, such as data lacking important visual cues to three-dimensional structure, data lacking multisensory statistical regularities, and data in which stimuli are unconnected to an observer’s actions and goals. Second, DNNs typically lack adaptations to capacity limits, such as attentional mechanisms, visual working memory, and compressed mental representations biased toward preserving task-relevant abstractions.
Collapse
|
46
|
Tsuda H, Saiki J. Constancy of visual working memory of glossiness under real-world illuminations. J Vis 2018; 18:14. [PMID: 30167672 DOI: 10.1167/18.8.14] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Glossiness is a surface property of material that is useful for recognizing objects and spaces. For glossiness to be effective across situations, our visual system must be unaffected by viewing contexts, such as lighting conditions. Although glossiness perception has constancy across changes in illumination, whether visual working memory also realizes glossiness constancy is not known. To address this issue, participants were presented with photo-realistic computer-generated images of spherical objects and asked to match the appearance of reference and test stimuli in relation to two dimensions of glossiness (contrast and sharpness). By comparing performance in terms of the match between perception and memory, we found that both features were well recalled, even when illumination contexts differed between the study and test periods. In addition, no correlation was found between recall errors related to contrast and sharpness, suggesting that these features are independently represented, not only in perception, as previously reported, but also in working memory. Taken together, these findings demonstrate the constancy of glossiness in visual working memory under conditions of real-world illumination.
Collapse
Affiliation(s)
- Hiroyuki Tsuda
- Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan
| | - Jun Saiki
- Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan
| |
Collapse
|
47
|
Tomić I, Bays PM. Internal but not external noise frees working memory resources. PLoS Comput Biol 2018; 14:e1006488. [PMID: 30321172 PMCID: PMC6201966 DOI: 10.1371/journal.pcbi.1006488] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2017] [Revised: 10/25/2018] [Accepted: 09/05/2018] [Indexed: 11/19/2022] Open
Abstract
The precision with which visual information can be recalled from working memory declines as the number of items in memory increases. This finding has been explained in terms of the distribution of a limited representational resource between items. Here we investigated how the sensory strength of memoranda affects resource allocation. We manipulated signal strength of an orientation stimulus in two ways: we varied the internal (sensory) noise by adjusting stimulus contrast, and varied the external (stimulus) noise by altering the within-stimulus variability. Both manipulations had similar effects on the precision with which the orientation could be recalled, but differed in their impact on memory for other stimuli. These results indicate that increasing internal noise released resources that could be used to store other stimuli more precisely; increasing external noise had no such effect. We show that these observations can be captured by a simple neural model of working memory encoding, in which spiking activity takes on the role of the limited resource. Investigations of visual short-term memory typically involve memorising clearly visible objects with elementary features, such as monochromatic disks or oriented bars. Results of such studies indicate that memory is allocated like a limited resource, i.e. shared out between objects. However, in daily life we are often confronted with visual features that are difficult to make out, like when an object is in shadow, or poorly-defined, like the color of a variegated leaf. Here we asked whether these kinds of features occupy as much memory resource as simple highly-visible objects. Our results demonstrate that reducing the sensory strength of a stimulus makes the quality of recall worse, but also takes up less resource so other objects can be remembered more precisely. Increasing the variability within a stimulus worsens recall, but has no effect on how other objects are remembered. These findings can be explained by considering how visual information is stored in populations of neurons: only the manipulation of sensory strength changes the amount of spiking activity dedicated to a stimulus.
Collapse
Affiliation(s)
- Ivan Tomić
- Faculty of Humanities and Social Sciences, Department of Psychology, University of Zagreb, Zagreb, Croatia
- Department of Psychology, University of Cambridge, Cambridge, United Kingdom
- * E-mail:
| | - Paul M. Bays
- Department of Psychology, University of Cambridge, Cambridge, United Kingdom
| |
Collapse
|
48
|
Yatziv T, Kessler Y. A two-level hierarchical framework of visual short-term memory. J Vis 2018; 18:2. [PMID: 30193344 DOI: 10.1167/18.9.2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Over the last couple of decades, a vast amount of research has been dedicated to understanding the nature and the architecture of visual short-term memory (VSTM), the mechanism by which currently relevant visual information is maintained. According to discrete-capacity models, VSTM is constrained by a limited number of discrete representations held simultaneously. In contrast, shared-resource models regard VSTM as limited in resources, which can be distributed flexibly between varying numbers of representations; and a new interference model posits that capacity is limited by interference among items. In this article, we begin by reviewing benchmark findings regarding the debate over VSTM limitations, focusing on whether VSTM storage is all-or-none and on whether object complexity affects capacity. After that, we put forward a hybrid framework of VSTM architecture, arguing that this system is composed of a two-level hierarchy of memory stores, each containing a different set of representations: (1) perceptual memory, a resourcelike level containing analog automatically formed representations of visual stimuli in varying degrees of activation, and (2) visual working memory, in which a subset of three to four items from perceptual memory are bound to conceptual representations and to their locations, thus conveying discrete (digital/symbolic) information which appears quantized. While perceptual memory has a large capacity and is relatively nonselective, visual working memory is restricted in the number of items that can be maintained simultaneously, and its content is regulated by a gating mechanism.
Collapse
Affiliation(s)
- Tal Yatziv
- Department of Psychology and Zlotowski Center for Neuroscience, Ben-Gurion University of the Negev, Beer-Sheva, Israel
| | - Yoav Kessler
- Department of Psychology and Zlotowski Center for Neuroscience, Ben-Gurion University of the Negev, Beer-Sheva, Israel
| |
Collapse
|
49
|
Chen M, Gaither K, John NW, McCann B. An Information-Theoretic Approach to the Cost-benefit Analysis of Visualization in Virtual Environments. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2018; 25:32-42. [PMID: 30136971 DOI: 10.1109/tvcg.2018.2865025] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Visualization and virtual environments (VEs) have been two interconnected parallel strands in visual computing for decades. Some VEs have been purposely developed for visualization applications, while many visualization applications are exemplary showcases in general-purpose VEs. Because of the development and operation costs of VEs, the majority of visualization applications in practice have yet to benefit from the capacity of VEs. In this paper, we examine this status quo from an information-theoretic perspective. Our objectives are to conduct cost-benefit analysis on typical VE systems (including augmented and mixed reality, theater-based systems, and large powerwalls), to explain why some visualization applications benefit more from VEs than others, and to sketch out pathways for the future development of visualization applications in VEs. We support our theoretical propositions and analysis using theories and discoveries in the literature of cognitive sciences and the practical evidence reported in the literatures of visualization and VEs.
Collapse
|
50
|
van den Berg R, Ma WJ. A resource-rational theory of set size effects in human visual working memory. eLife 2018; 7:e34963. [PMID: 30084356 PMCID: PMC6110611 DOI: 10.7554/elife.34963] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2018] [Accepted: 07/28/2018] [Indexed: 01/30/2023] Open
Abstract
Encoding precision in visual working memory decreases with the number of encoded items. Here, we propose a normative theory for such set size effects: the brain minimizes a weighted sum of an error-based behavioral cost and a neural encoding cost. We construct a model from this theory and find that it predicts set size effects. Notably, these effects are mediated by probing probability, which aligns with previous empirical findings. The model accounts well for effects of both set size and probing probability on encoding precision in nine delayed-estimation experiments. Moreover, we find support for the prediction that the total amount of invested resource can vary non-monotonically with set size. Finally, we show that it is sometimes optimal to encode only a subset or even none of the relevant items in a task. Our findings raise the possibility that cognitive "limitations" arise from rational cost minimization rather than from constraints.
Collapse
Affiliation(s)
| | - Wei Ji Ma
- Center for Neural Science and Department of PsychologyNew York UniversityNew YorkUnited States
| |
Collapse
|