1
|
Yashiro R, Sawayama M, Amano K. Decoding time-resolved neural representations of orientation ensemble perception. Front Neurosci 2024; 18:1387393. [PMID: 39148524 PMCID: PMC11325722 DOI: 10.3389/fnins.2024.1387393] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2024] [Accepted: 07/15/2024] [Indexed: 08/17/2024] Open
Abstract
The visual system can compute summary statistics of several visual elements at a glance. Numerous studies have shown that an ensemble of different visual features can be perceived over 50-200 ms; however, the time point at which the visual system forms an accurate ensemble representation associated with an individual's perception remains unclear. This is mainly because most previous studies have not fully addressed time-resolved neural representations that occur during ensemble perception, particularly lacking quantification of the representational strength of ensembles and their correlation with behavior. Here, we conducted orientation ensemble discrimination tasks and electroencephalogram (EEG) recordings to decode orientation representations over time while human observers discriminated an average of multiple orientations. We modeled EEG signals as a linear sum of hypothetical orientation channel responses and inverted this model to quantify the representational strength of orientation ensemble. Our analysis using this inverted encoding model revealed stronger representations of the average orientation over 400-700 ms. We also correlated the orientation representation estimated from EEG signals with the perceived average orientation reported in the ensemble discrimination task with adjustment methods. We found that the estimated orientation at approximately 600-700 ms significantly correlated with the individual differences in perceived average orientation. These results suggest that although ensembles can be quickly and roughly computed, the visual system may gradually compute an orientation ensemble over several hundred milliseconds to achieve a more accurate ensemble representation.
Collapse
Affiliation(s)
- Ryuto Yashiro
- Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan
| | - Masataka Sawayama
- Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan
| | - Kaoru Amano
- Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan
| |
Collapse
|
2
|
Zeng T, Zhao Y, Cao B, Jia J. Perception of visual variance is mediated by subcortical mechanisms. Brain Cogn 2024; 175:106131. [PMID: 38219416 DOI: 10.1016/j.bandc.2024.106131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Revised: 01/01/2024] [Accepted: 01/04/2024] [Indexed: 01/16/2024]
Abstract
Variance characterizes the structure of the environment. This statistical concept plays a critical role in evaluating the reliability of evidence for human decision-making. The present study examined the involvement of subcortical structures in the processing of visual variance. To this end, we used a stereoscope to sequentially present two circle arrays in a dichoptic or monocular fashion while participants compared the perceived variance of the two arrays. In Experiment 1, two arrays were presented monocularly to the same eye, dichopticly to different eyes, or binocularly to both eyes. The variance judgment was less accurate in different-eye condition than the other conditions. In Experiment 2, the first circle array was split into a large-variance and a small-variance set, with either the large-variance or small-variance set preceding the presentation of the second circle array in the same eye. The variance of the first array was judged larger when the second array was preceded by the large-variance set in the same eye, showing that the perception of variance was modulated by the visual variance processed in the same eye. Taken together, these findings provide evidence for monocular processing of visual variance, suggesting that subcortical structures capture the statistical structure of the visual world.
Collapse
Affiliation(s)
- Ting Zeng
- Department of Psychology, Hangzhou Normal University, Hangzhou 311121, Zhejiang, China; School of Psychology, Jiangxi Normal University, Nanchang 330022, Jiangxi, China; School of Education, Nanchang Normal College of Applied Technology, Nanchang 330108, Jiangxi, China
| | - Yuqing Zhao
- Department of Psychology, Hangzhou Normal University, Hangzhou 311121, Zhejiang, China
| | - Bihua Cao
- School of Psychology, Jiangxi Normal University, Nanchang 330022, Jiangxi, China.
| | - Jianrong Jia
- Department of Psychology, Hangzhou Normal University, Hangzhou 311121, Zhejiang, China; Zhejiang Philosophy and Social Science Laboratory for Research in Early Development and Childcare, Hangzhou Normal University, Hangzhou 311121, Zhejiang, China.
| |
Collapse
|
3
|
Wang T, Zhao Y, Jia J. Nonadditive integration of visual information in ensemble processing. iScience 2023; 26:107988. [PMID: 37822498 PMCID: PMC10562869 DOI: 10.1016/j.isci.2023.107988] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 09/03/2023] [Accepted: 09/16/2023] [Indexed: 10/13/2023] Open
Abstract
Statistically summarizing information from a stimulus array into an ensemble representation (e.g., the mean) improves the efficiency of visual processing. However, little is known about how the brain computes the ensemble statistics. Here, we propose that ensemble processing is realized by nonadditive integration, rather than linear averaging, of individual items. We used a linear regression model approach to extract EEG responses to three levels of information: the individual items, their local interactions, and their global interaction. The local and global interactions, representing nonadditive integration of individual items, elicited rapid and independent neural responses. Critically, only the neural representation of the global interaction predicted the precision of the ensemble perception at the behavioral level. Furthermore, spreading attention over the global pattern to enhance ensemble processing directly promoted rapid neural representation of the global interaction. Taken together, these findings advocate a global, nonadditive mechanism of ensemble processing in the brain.
Collapse
Affiliation(s)
- Tongyu Wang
- Department of Psychology, Hangzhou Normal University, Hangzhou 311121, Zhejiang, China
| | - Yuqing Zhao
- Department of Psychology, Hangzhou Normal University, Hangzhou 311121, Zhejiang, China
| | - Jianrong Jia
- Department of Psychology, Hangzhou Normal University, Hangzhou 311121, Zhejiang, China
- Zhejiang Philosophy and Social Science Laboratory for Research in Early Development and Childcare, Hangzhou Normal University, Hangzhou 311121, Zhejiang, China
| |
Collapse
|
4
|
Marini F, Sutherland CAM, Ostrovska B, Manassi M. Three's a crowd: Fast ensemble perception of first impressions of trustworthiness. Cognition 2023; 239:105540. [PMID: 37478696 DOI: 10.1016/j.cognition.2023.105540] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Revised: 05/07/2023] [Accepted: 06/27/2023] [Indexed: 07/23/2023]
Abstract
Trustworthiness impressions are fundamental social judgements with far-reaching consequences in many aspects of society, including criminal justice, leadership selection and partner preferences. Thus far, most research has focused on facial characteristics that make a face individually appear more or less trustworthy. However, in everyday life, faces are not always perceived in isolation but are often encountered in crowds. It has been proposed that we deal with the large amount of facial information in a group by extracting summary statistics of the crowd, a phenomenon called ensemble perception. Prior research showed that ensemble perception occurs for various facial features, such as emotional expression, facial identity, and attractiveness. Here, we investigated whether observers can integrate the level of trustworthiness from multiple faces to extract an average impression of the crowd. Across four studies, participants were presented with crowds of faces and were asked to report their average level of trustworthiness with an adjustment (Experiment 1) and a rating task (Experiments 2 and 3). Participants were able to extract an ensemble perception of trustworthiness impressions from multiple faces. Moreover, observers were able to form a summary statistic of trustworthiness impressions from a group of faces as quickly as 250 ms (Experiment 4). Taken together, these results demonstrate that ensemble perception can occur at the level of impressions of trustworthiness. Thus, these critical social judgements not only occur for individual faces but are also integrated into a unique ensemble impression of crowds. Our findings contribute to the development of a more ecological approach to the study of trust impressions, since they provide an understanding of trustworthiness judgements not only on an individual level, but on a much broader social group level. Furthermore, our results drive forward new theory because they demonstrate for the first time that ensemble representations cover a broad range of phenomena than previously recognized, including complex high-level facial trait judgements such as trustworthiness impressions.
Collapse
Affiliation(s)
- Fiammetta Marini
- School of Psychology, University of Aberdeen, King's College, Aberdeen, UK.
| | - Clare A M Sutherland
- School of Psychology, University of Aberdeen, King's College, Aberdeen, UK; School of Psychological Science, University of Western Australia, Crawley, Western Australia, Australia
| | - Bārbala Ostrovska
- School of Psychology, University of Aberdeen, King's College, Aberdeen, UK
| | - Mauro Manassi
- School of Psychology, University of Aberdeen, King's College, Aberdeen, UK
| |
Collapse
|
5
|
Zhao Y, Zeng T, Wang T, Fang F, Pan Y, Jia J. Subcortical encoding of summary statistics in humans. Cognition 2023; 234:105384. [PMID: 36736077 DOI: 10.1016/j.cognition.2023.105384] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2022] [Revised: 01/18/2023] [Accepted: 01/20/2023] [Indexed: 02/04/2023]
Abstract
Statistical encoding compresses redundant information from multiple items into a single summary metric (e.g., mean). Such statistical representation has been suggested to be automatic, but at which stage it is extracted is unknown. Here, we examined the involvement of the subcortex in the processing of summary statistics. We presented an array of circles dichoptically or monocularly while matching the number of perceived circles after binocular fusion. Experiments 1 and 2 showed that interocularly suppressed, invisible circles were automatically involved in the summary statistical representation, but only when they were presented to the same eye as the visible circles. This same-eye effect was further observed for consciously processed circles in Experiment 3, in which the estimated mean size of the circles was biased toward the information transmitted by monocular channels. Together, we provide converging evidence that the processing of summary statistics, an assumed high-level cognitive process, is mediated by subcortical structures.
Collapse
Affiliation(s)
- Yuqing Zhao
- Department of Psychology, Hangzhou Normal University, Hangzhou 311121, Zhejiang, China; Center for Cognition and Brain Disorders, The Affiliated Hospital of Hangzhou Normal University, Hangzhou 310015, Zhejiang, China
| | - Ting Zeng
- Center for Cognition and Brain Disorders, The Affiliated Hospital of Hangzhou Normal University, Hangzhou 310015, Zhejiang, China; School of Psychology, Jiangxi Normal University, Nanchang 330022, Jiangxi, China
| | - Tongyu Wang
- Department of Psychology, Hangzhou Normal University, Hangzhou 311121, Zhejiang, China; Center for Cognition and Brain Disorders, The Affiliated Hospital of Hangzhou Normal University, Hangzhou 310015, Zhejiang, China
| | - Fang Fang
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing 100871, China; IDG/McGovern Institute for Brain Research, Peking University, Beijing 100871, China; Peking-Tsinghua Center for Life Sciences, Peking University, Beijing 100871, China
| | - Yi Pan
- Department of Psychology, Hangzhou Normal University, Hangzhou 311121, Zhejiang, China.
| | - Jianrong Jia
- Department of Psychology, Hangzhou Normal University, Hangzhou 311121, Zhejiang, China; Center for Cognition and Brain Disorders, The Affiliated Hospital of Hangzhou Normal University, Hangzhou 310015, Zhejiang, China.
| |
Collapse
|
6
|
Jia J, Wang T, Chen S, Ding N, Fang F. Ensemble size perception: Its neural signature and the role of global interaction over individual items. Neuropsychologia 2022; 173:108290. [PMID: 35697088 DOI: 10.1016/j.neuropsychologia.2022.108290] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2021] [Revised: 06/02/2022] [Accepted: 06/07/2022] [Indexed: 10/18/2022]
Abstract
To efficiently process complex visual scenes, the visual system often summarizes statistical information across individual items and represents them as an ensemble. However, due to the lack of techniques to disentangle the representation of the ensemble from that of the individual items constituting the ensemble, whether there exists a specialized neural mechanism for ensemble processing and how ensemble perception is computed in the brain remain unknown. To address these issues, we used a frequency-tagging EEG approach to track brain responses to periodically updated ensemble sizes. Neural responses tracking the ensemble size were detected in parieto-occipital electrodes, revealing a global and specialized neural mechanism of ensemble size perception. We then used the temporal response function to isolate neural responses to the individual sizes and their interactions. Notably, while the individual sizes and their local and global interactions were encoded in the EEG signals, only the global interaction contributed directly to the ensemble size perception. Finally, distributed attention to the global stimulus pattern enhanced the neural signature of the ensemble size, mainly by modulating the neural representation of the global interaction between all individual sizes. These findings advocate a specialized, global neural mechanism of ensemble size perception and suggest that global interaction between individual items contributes to ensemble perception.
Collapse
Affiliation(s)
- Jianrong Jia
- Center for Cognition and Brain Disorders, The Affiliated Hospital of Hangzhou Normal University, Hangzhou, 311121, China; Institute of Psychological Sciences, Hangzhou Normal University, Hangzhou, 311121, China
| | - Tongyu Wang
- Center for Cognition and Brain Disorders, The Affiliated Hospital of Hangzhou Normal University, Hangzhou, 311121, China; Institute of Psychological Sciences, Hangzhou Normal University, Hangzhou, 311121, China
| | - Siqi Chen
- Center for Cognition and Brain Disorders, The Affiliated Hospital of Hangzhou Normal University, Hangzhou, 311121, China; Institute of Psychological Sciences, Hangzhou Normal University, Hangzhou, 311121, China
| | - Nai Ding
- Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou, 311121, China; Research Center for Advanced Artificial Intelligence Theory, Zhejiang Lab, Hangzhou, 311121, China
| | - Fang Fang
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, 100871, China; IDG/McGovern Institute for Brain Research, Peking University, Beijing, 100871, China; Key Laboratory of Machine Perception (Ministry of Education), Peking University, Beijing, 100871, China; Peking-Tsinghua Center for Life Sciences, Peking University, Beijing, 100871, China.
| |
Collapse
|
7
|
Zhang K, Qian J. The role of ensemble average differs in working memory for depth and planar information. J Vis 2022; 22:4. [PMID: 35522260 PMCID: PMC9078066 DOI: 10.1167/jov.22.6.4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2021] [Accepted: 04/06/2022] [Indexed: 11/24/2022] Open
Abstract
The representation of individual planar locations and features stored in working memory can be affected by the average representation. However, less is known about how the average representation affects the short-term storage of depth information. To evaluate the possible different roles of the ensemble average in working memory for planar and depth information, we used mathematical models to fit the data collected from one study on working memory for depth and 12 studies on working memory for planar information. The pattern of recalled depth was well captured by models assuming that there was a probability of reporting the average depth instead of the individual depth, compressing the recalled front-back distance of the stimulus ensemble compared to the perceived distance. However, when modeling the recalled planar information, we found that participants tended to report individual nontarget features when the target was not memorized, and the assumption of reporting average information improves the data fitting only in very few studies. These results provide evidence for our hypothesis that average depth information can be used as a substitution for individual depth information stored in working memory, but for planar visual features, the substitution of target with the average works under a constraint that the average of to-be-remembered features is readily accessible.
Collapse
Affiliation(s)
- Ke Zhang
- Sun Yat-Sen University, Department of Psychology, Guangzhou, China
- Shaoxing University, Center for Brain, Mind, and Education, Shaoxing, China
| | - Jiehui Qian
- Sun Yat-Sen University, Department of Psychology, Guangzhou, China
| |
Collapse
|