1
|
Leticevscaia O, Brandman T, Peelen MV. Scene context and attention independently facilitate MEG decoding of object category. Vision Res 2024; 224:108484. [PMID: 39260230 DOI: 10.1016/j.visres.2024.108484] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Revised: 08/25/2024] [Accepted: 09/02/2024] [Indexed: 09/13/2024]
Abstract
Many of the objects we encounter in our everyday environments would be hard to recognize without any expectations about these objects. For example, a distant silhouette may be perceived as a car because we expect objects of that size, positioned on a road, to be cars. Reflecting the influence of such expectations on visual processing, neuroimaging studies have shown that when objects are poorly visible, expectations derived from scene context facilitate the representations of these objects in visual cortex from around 300 ms after scene onset. The current magnetoencephalography (MEG) study tested whether this facilitation occurs independently of attention and task relevance. Participants viewed degraded objects alone or within scene context while they either attended the scenes (attended condition) or the fixation cross (unattended condition), also temporally directing attention away from the scenes. Results showed that at 300 ms after stimulus onset, multivariate classifiers trained to distinguish clearly visible animate vs inanimate objects generalized to distinguish degraded objects in scenes better than degraded objects alone, despite the added clutter of the scene background. Attention also modulated object representations at this latency, with better category decoding in the attended than the unattended condition. The modulatory effects of context and attention were independent of each other. Finally, data from the current study and a previous study were combined (N = 51) to provide a more detailed temporal characterization of contextual facilitation. These results extend previous work by showing that facilitatory scene-object interactions are independent of the specific task performed on the visual input.
Collapse
Affiliation(s)
- Olga Leticevscaia
- University of Reading, Centre for Integrative Neuroscience and Neurodynamics, United Kingdom
| | - Talia Brandman
- Department of Brain Sciences, Weizmann Institute of Science, Rehovot 76100, Israel
| | - Marius V Peelen
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands.
| |
Collapse
|
2
|
Moerel D, Rich AN, Woolgar A. Selective Attention and Decision-Making Have Separable Neural Bases in Space and Time. J Neurosci 2024; 44:e0224242024. [PMID: 39107058 PMCID: PMC11411586 DOI: 10.1523/jneurosci.0224-24.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Revised: 06/21/2024] [Accepted: 07/19/2024] [Indexed: 08/09/2024] Open
Abstract
Attention and decision-making processes are fundamental to cognition. However, they are usually experimentally confounded, making it difficult to link neural observations to specific processes. Here we separated the effects of selective attention from the effects of decision-making on brain activity obtained from human participants (both sexes), using a two-stage task where the attended stimulus and decision were orthogonal and separated in time. Multivariate pattern analyses of multimodal neuroimaging data revealed the dynamics of perceptual and decision-related information coding through time with magnetoencephalography (MEG), through space with functional magnetic resonance imaging (fMRI), and their combination (MEG-fMRI fusion). Our MEG results showed an effect of attention before decision-making could begin, and fMRI results showed an attention effect in early visual and frontoparietal regions. Model-based MEG-fMRI fusion suggested that attention boosted stimulus information in the frontoparietal and early visual regions before decision-making was possible. Together, our results suggest that attention affects neural stimulus representations in the frontoparietal regions independent of decision-making.
Collapse
Affiliation(s)
- Denise Moerel
- School of Psychological Sciences, Macquarie University, Sydney 2109, New South Wales, Australia
- Perception in Action Research Centre, Macquarie University, Sydney 2109, New South Wales, Australia
- The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Sydney 2145, New South Wales, Australia
| | - Anina N Rich
- School of Psychological Sciences, Macquarie University, Sydney 2109, New South Wales, Australia
- Perception in Action Research Centre, Macquarie University, Sydney 2109, New South Wales, Australia
- Macquarie University Performance and Expertise Research Centre, Sydney 2109, New South Wales, Australia
| | - Alexandra Woolgar
- School of Psychological Sciences, Macquarie University, Sydney 2109, New South Wales, Australia
- Perception in Action Research Centre, Macquarie University, Sydney 2109, New South Wales, Australia
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge CB2 7EF, United Kingdom
| |
Collapse
|
3
|
Moerel D, Psihoyos J, Carlson TA. The Time-Course of Food Representation in the Human Brain. J Neurosci 2024; 44:e1101232024. [PMID: 38740441 PMCID: PMC11211715 DOI: 10.1523/jneurosci.1101-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Revised: 03/29/2024] [Accepted: 03/29/2024] [Indexed: 05/16/2024] Open
Abstract
Humans make decisions about food every day. The visual system provides important information that forms a basis for these food decisions. Although previous research has focused on visual object and category representations in the brain, it is still unclear how visually presented food is encoded by the brain. Here, we investigate the time-course of food representations in the brain. We used time-resolved multivariate analyses of electroencephalography (EEG) data, obtained from human participants (both sexes), to determine which food features are represented in the brain and whether focused attention is needed for this. We recorded EEG while participants engaged in two different tasks. In one task, the stimuli were task relevant, whereas in the other task, the stimuli were not task relevant. Our findings indicate that the brain can differentiate between food and nonfood items from ∼112 ms after the stimulus onset. The neural signal at later latencies contained information about food naturalness, how much the food was transformed, as well as the perceived caloric content. This information was present regardless of the task. Information about whether food is immediately ready to eat, however, was only present when the food was task relevant and presented at a slow presentation rate. Furthermore, the recorded brain activity correlated with the behavioral responses in an odd-item-out task. The fast representation of these food features, along with the finding that this information is used to guide food categorization decision-making, suggests that these features are important dimensions along which the representation of foods is organized.
Collapse
Affiliation(s)
- Denise Moerel
- School of Psychology, University of Sydney, Sydney, New South Wales 2050, Australia
| | - James Psihoyos
- School of Psychology, University of Sydney, Sydney, New South Wales 2050, Australia
| | - Thomas A Carlson
- School of Psychology, University of Sydney, Sydney, New South Wales 2050, Australia
| |
Collapse
|
4
|
Koenig-Robert R, Quek GL, Grootswagers T, Varlet M. Movement trajectories as a window into the dynamics of emerging neural representations. Sci Rep 2024; 14:11499. [PMID: 38769313 PMCID: PMC11106280 DOI: 10.1038/s41598-024-62135-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Accepted: 05/14/2024] [Indexed: 05/22/2024] Open
Abstract
The rapid transformation of sensory inputs into meaningful neural representations is critical to adaptive human behaviour. While non-invasive neuroimaging methods are the de-facto method for investigating neural representations, they remain expensive, not widely available, time-consuming, and restrictive. Here we show that movement trajectories can be used to measure emerging neural representations with fine temporal resolution. By combining online computer mouse-tracking and publicly available neuroimaging data via representational similarity analysis (RSA), we show that movement trajectories track the unfolding of stimulus- and category-wise neural representations along key dimensions of the human visual system. We demonstrate that time-resolved representational structures derived from movement trajectories overlap with those derived from M/EEG (albeit delayed) and those derived from fMRI in functionally-relevant brain areas. Our findings highlight the richness of movement trajectories and the power of the RSA framework to reveal and compare their information content, opening new avenues to better understand human perception.
Collapse
Affiliation(s)
- Roger Koenig-Robert
- The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Penrith, NSW, 2751, Australia
- School of Psychology, University of New South Wales, Sydney, NSW, Australia
| | - Genevieve L Quek
- The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Penrith, NSW, 2751, Australia
| | - Tijl Grootswagers
- The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Penrith, NSW, 2751, Australia
- School of Computer, Data and Mathematical Sciences, Western Sydney University, Penrith, NSW, 2751, Australia
| | - Manuel Varlet
- The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Penrith, NSW, 2751, Australia.
- School of Psychology, Western Sydney University, Sydney, NSW, 2751, Australia.
| |
Collapse
|
5
|
Grootswagers T, Robinson AK, Shatek SM, Carlson TA. Mapping the dynamics of visual feature coding: Insights into perception and integration. PLoS Comput Biol 2024; 20:e1011760. [PMID: 38190390 PMCID: PMC10798643 DOI: 10.1371/journal.pcbi.1011760] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Revised: 01/19/2024] [Accepted: 12/13/2023] [Indexed: 01/10/2024] Open
Abstract
The basic computations performed in the human early visual cortex are the foundation for visual perception. While we know a lot about these computations, a key missing piece is how the coding of visual features relates to our perception of the environment. To investigate visual feature coding, interactions, and their relationship to human perception, we investigated neural responses and perceptual similarity judgements to a large set of visual stimuli that varied parametrically along four feature dimensions. We measured neural responses using electroencephalography (N = 16) to 256 grating stimuli that varied in orientation, spatial frequency, contrast, and colour. We then mapped the response profiles of the neural coding of each visual feature and their interactions, and related these to independently obtained behavioural judgements of stimulus similarity. The results confirmed fundamental principles of feature coding in the visual system, such that all four features were processed simultaneously but differed in their dynamics, and there was distinctive conjunction coding for different combinations of features in the neural responses. Importantly, modelling of the behaviour revealed that every stimulus feature contributed to perceptual judgements, despite the untargeted nature of the behavioural task. Further, the relationship between neural coding and behaviour was evident from initial processing stages, signifying that the fundamental features, not just their interactions, contribute to perception. This study highlights the importance of understanding how feature coding progresses through the visual hierarchy and the relationship between different stages of processing and perception.
Collapse
Affiliation(s)
- Tijl Grootswagers
- The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Sydney, Australia
- School of Computer, Data and Mathematical Sciences, Western Sydney University, Sydney, Australia
| | - Amanda K. Robinson
- Queensland Brain Institute, The University of Queensland, Brisbane, Australia
| | - Sophia M. Shatek
- School of Psychology, The University of Sydney, Sydney, Australia
| | | |
Collapse
|
6
|
Marsicano G, Casartelli L, Federici A, Bertoni S, Vignali L, Molteni M, Facoetti A, Ronconi L. Prolonged neural encoding of visual information in autism. Autism Res 2024; 17:37-54. [PMID: 38009961 DOI: 10.1002/aur.3062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Accepted: 10/30/2023] [Indexed: 11/29/2023]
Abstract
Autism spectrum disorder (ASD) is associated with a hyper-focused visual attentional style, impacting higher-order social and affective domains. The understanding of such peculiarity can benefit from the use of multivariate pattern analysis (MVPA) of high-resolution electroencephalography (EEG) data, which has proved to be a powerful technique to investigate the hidden neural dynamics orchestrating sensory and cognitive processes. Here, we recorded EEG in typically developing (TD) children and in children with ASD during a visuo-spatial attentional task where attention was exogenously captured by a small (zoom-in) or large (zoom-out) cue in the visual field before the appearance of a target at different eccentricities. MVPA was performed both in the cue-locked period, to reveal potential differences in the modulation of the attentional focus, and in the target-locked period, to reveal potential cascade effects on stimulus processing. Cue-locked MVPA revealed that while in the TD group the pattern of neural activity contained information about the cue mainly before the target appearance, the ASD group showed a temporally sustained and topographically diffuse significant decoding of the cue neural response even after the target onset, suggesting a delayed extinction of cue-related neural activity. Crucially, this delayed extinction positively correlated with behavioral measures of attentional hyperfocusing. Results of target-locked MVPA were coherent with a hyper-focused attentional profile, highlighting an earlier and stronger decoding of target neural responses in small cue trials in the ASD group. The present findings document a spatially and temporally overrepresented encoding of visual information in ASD, which can constitute one of the main reasons behind their peculiar cognitive style.
Collapse
Affiliation(s)
- Gianluca Marsicano
- Department of Psychology, University of Bologna, Bologna, Italy
- Centre for Studies and Research in Cognitive Neuroscience, University of Bologna, Cesena, Italy
- Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Luca Casartelli
- Child Psychopathology Department, Theoretical and Cognitive Neuroscience Unit, Scientific Institute IRCCS E.MEDEA, Bosisio Parini, Italy
| | | | - Sara Bertoni
- Department of Human and Social Sciences, University of Bergamo, Bergamo, Italy
- Developmental and Cognitive Neuroscience Lab, Department of General Psychology, University of Padua, Padova, Italy
| | | | - Massimo Molteni
- Child Psychopathology Department, Theoretical and Cognitive Neuroscience Unit, Scientific Institute IRCCS E.MEDEA, Bosisio Parini, Italy
| | - Andrea Facoetti
- Developmental and Cognitive Neuroscience Lab, Department of General Psychology, University of Padua, Padova, Italy
| | - Luca Ronconi
- Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy
- School of Psychology, Vita-Salute San Raffaele University, Milan, Italy
| |
Collapse
|
7
|
Lowe BG, Robinson JE, Yamamoto N, Hogendoorn H, Johnston P. Same but different: The latency of a shared expectation signal interacts with stimulus attributes. Cortex 2023; 168:143-156. [PMID: 37716110 DOI: 10.1016/j.cortex.2023.08.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Revised: 07/13/2023] [Accepted: 08/07/2023] [Indexed: 09/18/2023]
Abstract
Predictive coding theories assert that perceptual inference is a hierarchical process of belief updating, wherein the onset of unexpected sensory data causes so-called prediction error responses that calibrate erroneous inferences. Given the functionally specialised organisation of visual cortex, it is assumed that prediction error propagation interacts with the specific visual attribute violating an expectation. We sought to test this within the temporal domain by applying time-resolved decoding methods to electroencephalography (EEG) data evoked by contextual trajectory violations of either brightness, size, or orientation within a bound stimulus. We found that following ∼170 ms post stimulus onset, responses to both size violations and orientation violations were decodable from physically identical control trials in which no attributes were violated. These two violation types were then directly compared, with attribute-specific signalling being decoded from 265 ms. Temporal generalisation suggested that this dissociation was driven by latency shifts in shared expectation signalling between the two conditions. Using a novel temporal bias method, we then found that this shared signalling occurred earlier for size violations than orientation violations. To our knowledge, we are among the first to decode expectation violations in humans using EEG and have demonstrated a temporal dissociation in attribute-specific expectancy violations.
Collapse
Affiliation(s)
- Benjamin G Lowe
- School of Psychology and Counselling, Queensland University of Technology (QUT), Kelvin Grove, QLD, Australia; Perception in Action Research Centre & School of Psychological Sciences, Macquarie University, Macquarie Park, NSW, Australia.
| | - Jonathan E Robinson
- Monash Centre for Consciousness & Contemplative Studies, Monash University, Clayton, VIC, Australia
| | - Naohide Yamamoto
- School of Psychology and Counselling, Queensland University of Technology (QUT), Kelvin Grove, QLD, Australia; Centre for Vision and Eye Research, Queensland University of Technology (QUT), Kelvin Grove, QLD, Australia
| | - Hinze Hogendoorn
- School of Psychology and Counselling, Queensland University of Technology (QUT), Kelvin Grove, QLD, Australia; Melbourne School of Psychological Science, University of Melbourne, Parkville, VIC, Australia
| | - Patrick Johnston
- School of Exercise Science and Nutrition Sciences, Queensland University of Technology (QUT), Kelvin Grove, QLD, Australia
| |
Collapse
|
8
|
Cerracchio E, Miletić S, Forstmann BU. Modelling decision-making biases. Front Comput Neurosci 2023; 17:1222924. [PMID: 37927545 PMCID: PMC10622807 DOI: 10.3389/fncom.2023.1222924] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Accepted: 10/09/2023] [Indexed: 11/07/2023] Open
Abstract
Biases are a fundamental aspect of everyday life decision-making. A variety of modelling approaches have been suggested to capture decision-making biases. Statistical models are a means to describe the data, but the results are usually interpreted according to a verbal theory. This can lead to an ambiguous interpretation of the data. Mathematical cognitive models of decision-making outline the structure of the decision process with formal assumptions, providing advantages in terms of prediction, simulation, and interpretability compared to statistical models. We compare studies that used both signal detection theory and evidence accumulation models as models of decision-making biases, concluding that the latter provides a more comprehensive account of the decision-making phenomena by including response time behavior. We conclude by reviewing recent studies investigating attention and expectation biases with evidence accumulation models. Previous findings, reporting an exclusive influence of attention on the speed of evidence accumulation and prior probability on starting point, are challenged by novel results suggesting an additional effect of attention on non-decision time and prior probability on drift rate.
Collapse
Affiliation(s)
- Ettore Cerracchio
- Department of Psychology, University of Amsterdam, Amsterdam, Netherlands
| | | | | |
Collapse
|
9
|
Robinson AK, Quek GL, Carlson TA. Visual Representations: Insights from Neural Decoding. Annu Rev Vis Sci 2023; 9:313-335. [PMID: 36889254 DOI: 10.1146/annurev-vision-100120-025301] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/10/2023]
Abstract
Patterns of brain activity contain meaningful information about the perceived world. Recent decades have welcomed a new era in neural analyses, with computational techniques from machine learning applied to neural data to decode information represented in the brain. In this article, we review how decoding approaches have advanced our understanding of visual representations and discuss efforts to characterize both the complexity and the behavioral relevance of these representations. We outline the current consensus regarding the spatiotemporal structure of visual representations and review recent findings that suggest that visual representations are at once robust to perturbations, yet sensitive to different mental states. Beyond representations of the physical world, recent decoding work has shone a light on how the brain instantiates internally generated states, for example, during imagery and prediction. Going forward, decoding has remarkable potential to assess the functional relevance of visual representations for human behavior, reveal how representations change across development and during aging, and uncover their presentation in various mental disorders.
Collapse
Affiliation(s)
- Amanda K Robinson
- Queensland Brain Institute, The University of Queensland, Brisbane, Australia;
| | - Genevieve L Quek
- The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Sydney, Australia;
| | | |
Collapse
|
10
|
Male AG, O’Shea RP. Attention is required for canonical brain signature of prediction error despite early encoding of the stimuli. PLoS Biol 2023; 21:e3001866. [PMID: 37339145 PMCID: PMC10281583 DOI: 10.1371/journal.pbio.3001866] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2022] [Accepted: 05/11/2023] [Indexed: 06/22/2023] Open
Abstract
Prediction error is a basic component of predictive-coding theory of brain processing. According to the theory, each stage of brain processing of sensory information generates a model of the current sensory input; subsequent input is compared against the model and only if there is a mismatch, a prediction error, is further processing performed. Recently, Smout and colleagues found that a signature of prediction error, the visual (v) mismatch negativity (MMN), for a fundamental property of visual input-its orientation-was absent without endogenous attention on the stimuli. This is remarkable because the weight of evidence for MMNs from audition and vision is that they occur without endogenous attention. To resolve this discrepancy, we conducted an experiment addressing 2 alternative explanations for Smout and colleagues' finding: that it was from a lack of reproducibility or that participants' visual systems did not encode the stimuli when attention was on something else. We conducted a similar experiment to that of Smout and colleagues. We showed 21 participants sequences of identically oriented Gabor patches, standards, and, unpredictably, otherwise identical, Gabor patches differing in orientation by ±15°, ±30°, and ±60°, deviants. To test whether participants encoded the orientation of the standards, we varied the number of standards preceding a deviant, allowing us to search for a decrease in activity with the number of repetitions of standards-repetition suppression. We diverted participants' attention from the oriented stimuli with a central, letter-detection task. We reproduced Smout and colleagues' finding of no vMMN without endogenous attention, strengthening their finding. We found that our participants showed repetition suppression: They did encode the stimuli preattentively. We also found early processing of deviants. We discuss various explanations why the earlier processing did not extend into the vMMN time window, including low precision of prediction.
Collapse
Affiliation(s)
- Alie G. Male
- Discipline of Psychology, College of Science, Health, Engineering and Education, Murdoch University, Perth, Australia
- Department of Psychiatry and Human Behavior, School of Medicine, University of California, Irvine, United States of America
| | - Robert P. O’Shea
- Discipline of Psychology, College of Science, Health, Engineering and Education, Murdoch University, Perth, Australia
- Wilhelm Wundt Institute for Psychology, University of Leipzig, Leipzig, Germany
| |
Collapse
|
11
|
Noah S, Meyyappan S, Ding M, Mangun GR. Time Courses of Attended and Ignored Object Representations. J Cogn Neurosci 2023; 35:645-658. [PMID: 36735619 PMCID: PMC10024573 DOI: 10.1162/jocn_a_01972] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
Selective attention prioritizes information that is relevant to behavioral goals. Previous studies have shown that attended visual information is processed and represented more efficiently, but distracting visual information is not fully suppressed, and may also continue to be represented in the brain. In natural vision, to-be-attended and to-be-ignored objects may be present simultaneously in the scene. Understanding precisely how each is represented in the visual system, and how these neural representations evolve over time, remains a key goal in cognitive neuroscience. In this study, we recorded EEG while participants performed a cued object-based attention task that involved attending to target objects and ignoring simultaneously presented and spatially overlapping distractor objects. We performed support vector machine classification on the stimulus-evoked EEG data to separately track the temporal dynamics of target and distractor representations. We found that (1) both target and distractor objects were decodable during the early phase of object processing (∼100 msec to ∼200 msec after target onset), and (2) the representations of both objects were sustained over time, remaining decodable above chance until ∼1000-msec latency. However, (3) the distractor object information faded significantly beginning after about 300-msec latency. These findings provide information about the fate of attended and ignored visual information in complex scene perception.
Collapse
Affiliation(s)
- Sean Noah
- University of California, Davis.,University of California, Berkeley
| | | | | | | |
Collapse
|