1
|
Li Y, Yang H, Gu S. Enhancing neural encoding models for naturalistic perception with a multi-level integration of deep neural networks and cortical networks. Sci Bull (Beijing) 2024; 69:1738-1747. [PMID: 38490889 DOI: 10.1016/j.scib.2024.02.035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2023] [Revised: 06/27/2023] [Accepted: 02/23/2024] [Indexed: 03/17/2024]
Abstract
Cognitive neuroscience aims to develop computational models that can accurately predict and explain neural responses to sensory inputs in the cortex. Recent studies attempt to leverage the representation power of deep neural networks (DNNs) to predict the brain response and suggest a correspondence between artificial and biological neural networks in their feature representations. However, typical voxel-wise encoding models tend to rely on specific networks designed for computer vision tasks, leading to suboptimal brain-wide correspondence during cognitive tasks. To address this challenge, this work proposes a novel approach that upgrades voxel-wise encoding models through multi-level integration of features from DNNs and information from brain networks. Our approach combines DNN feature-level ensemble learning and brain atlas-level model integration, resulting in significant improvements in predicting whole-brain neural activity during naturalistic video perception. Furthermore, this multi-level integration framework enables a deeper understanding of the brain's neural representation mechanism, accurately predicting the neural response to complex visual concepts. We demonstrate that neural encoding models can be optimized by leveraging a framework that integrates both data-driven approaches and theoretical insights into the functional structure of the cortical networks.
Collapse
Affiliation(s)
- Yuanning Li
- School of Biomedical Engineering & State Key Laboratory of Advanced Medical Materials and Devices, ShanghaiTech University, Shanghai 201210, China.
| | - Huzheng Yang
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China; Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Shi Gu
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China; Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen 518110, China.
| |
Collapse
|
2
|
Hanson SJ, Yadav V, Hanson C. Dense Sample Deep Learning. Neural Comput 2024; 36:1228-1244. [PMID: 38669696 DOI: 10.1162/neco_a_01666] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 01/24/2024] [Indexed: 04/28/2024]
Abstract
Deep learning (DL), a variant of the neural network algorithms originally proposed in the 1980s (Rumelhart et al., 1986), has made surprising progress in artificial intelligence (AI), ranging from language translation, protein folding (Jumper et al., 2021), autonomous cars, and, more recently, human-like language models (chatbots). All that seemed intractable until very recently. Despite the growing use of DL networks, little is understood about the learning mechanisms and representations that make these networks effective across such a diverse range of applications. Part of the answer must be the huge scale of the architecture and, of course, the large scale of the data, since not much has changed since 1986. But the nature of deep learned representations remains largely unknown. Unfortunately, training sets with millions or billions of tokens have unknown combinatorics, and networks with millions or billions of hidden units can't easily be visualized and their mechanisms can't be easily revealed. In this letter, we explore these challenges with a large (1.24 million weights VGG) DL in a novel high-density sample task (five unique tokens with more than 500 exemplars per token), which allows us to more carefully follow the emergence of category structure and feature construction. We use various visualization methods for following the emergence of the classification and the development of the coupling of feature detectors and structures that provide a type of graphical bootstrapping. From these results, we harvest some basic observations of the learning dynamics of DL and propose a new theory of complex feature construction based on our results.
Collapse
Affiliation(s)
- Stephen José Hanson
- Rutgers Brain Imaging Center and Psychology Department, Rutgers University, Newark, NJ 07102, U.S.A.
| | - Vivek Yadav
- Rutgers Brain Imaging Center, Rutgers University, Newark, NJ 07102, U.S.A.
| | - Catherine Hanson
- Center for Molecular and Behavioral Neuroscience and Rutgers Brain Imaging Center, Rutgers University, Newark, NJ 07102, U.S.A.
| |
Collapse
|
3
|
Bornet A, Doerig A, Herzog MH, Francis G, Van der Burg E. Shrinking Bouma's window: How to model crowding in dense displays. PLoS Comput Biol 2021; 17:e1009187. [PMID: 34228703 PMCID: PMC8284675 DOI: 10.1371/journal.pcbi.1009187] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2020] [Revised: 07/16/2021] [Accepted: 06/16/2021] [Indexed: 11/22/2022] Open
Abstract
In crowding, perception of a target deteriorates in the presence of nearby flankers. Traditionally, it is thought that visual crowding obeys Bouma's law, i.e., all elements within a certain distance interfere with the target, and that adding more elements always leads to stronger crowding. Crowding is predominantly studied using sparse displays (a target surrounded by a few flankers). However, many studies have shown that this approach leads to wrong conclusions about human vision. Van der Burg and colleagues proposed a paradigm to measure crowding in dense displays using genetic algorithms. Displays were selected and combined over several generations to maximize human performance. In contrast to Bouma's law, only the target's nearest neighbours affected performance. Here, we tested various models to explain these results. We used the same genetic algorithm, but instead of selecting displays based on human performance we selected displays based on the model's outputs. We found that all models based on the traditional feedforward pooling framework of vision were unable to reproduce human behaviour. In contrast, all models involving a dedicated grouping stage explained the results successfully. We show how traditional models can be improved by adding a grouping stage.
Collapse
Affiliation(s)
- Alban Bornet
- Laboratory of Psychophysics, Brain Mind Institute, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Adrien Doerig
- Laboratory of Psychophysics, Brain Mind Institute, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| | - Michael H. Herzog
- Laboratory of Psychophysics, Brain Mind Institute, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Gregory Francis
- Department of Psychological Sciences, Purdue University, West Lafayette, Indiana, United States of America
| | - Erik Van der Burg
- TNO, Human Factors, Soesterberg, The Netherlands
- Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
4
|
Tsotsos JK, Abid O, Kotseruba I, Solbach MD. On the control of attentional processes in vision. Cortex 2021; 137:305-329. [PMID: 33677138 DOI: 10.1016/j.cortex.2021.01.001] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2020] [Revised: 12/17/2020] [Accepted: 01/07/2021] [Indexed: 11/26/2022]
Abstract
The study of attentional processing in vision has a long and deep history. Recently, several papers have presented insightful perspectives into how the coordination of multiple attentional functions in the brain might occur. These begin with experimental observations and the authors propose structures, processes, and computations that might explain those observations. Here, we consider a perspective that past works have not, as a complementary approach to the experimentally-grounded ones. We approach the same problem as past authors but from the other end of the computational spectrum, from the problem nature, as Marr's Computational Level would prescribe. What problem must the brain solve when orchestrating attentional processes in order to successfully complete one of the myriad possible visuospatial tasks at which we as humans excel? The hope, of course, is for the approaches to eventually meet and thus form a complete theory, but this is likely not soon. We make the first steps towards this by addressing the necessity of attentional control, examining the breadth and computational difficulty of the visuospatial and attentional tasks seen in human behavior, and suggesting a sketch of how attentional control might arise in the brain. The key conclusions of this paper are that an executive controller is necessary for human attentional function in vision, and that there is a 'first principles' computational approach to its understanding that is complementary to the previous approaches that focus on modelling or learning from experimental observations directly.
Collapse
|
5
|
Pan H, Zhang S, Pan D, Ye Z, Yu H, Ding J, Wang Q, Sun Q, Hua T. Characterization of Feedback Neurons in the High-Level Visual Cortical Areas That Project Directly to the Primary Visual Cortex in the Cat. Front Neuroanat 2021; 14:616465. [PMID: 33488364 PMCID: PMC7820340 DOI: 10.3389/fnana.2020.616465] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2020] [Accepted: 12/04/2020] [Indexed: 12/17/2022] Open
Abstract
Previous studies indicate that top-down influence plays a critical role in visual information processing and perceptual detection. However, the substrate that carries top-down influence remains poorly understood. Using a combined technique of retrograde neuronal tracing and immunofluorescent double labeling, we characterized the distribution and cell type of feedback neurons in cat's high-level visual cortical areas that send direct connections to the primary visual cortex (V1: area 17). Our results showed: (1) the high-level visual cortex of area 21a at the ventral stream and PMLS area at the dorsal stream have a similar proportion of feedback neurons back projecting to the V1 area, (2) the distribution of feedback neurons in the higher-order visual area 21a and PMLS was significantly denser than in the intermediate visual cortex of area 19 and 18, (3) feedback neurons in all observed high-level visual cortex were found in layer II-III, IV, V, and VI, with a higher proportion in layer II-III, V, and VI than in layer IV, and (4) most feedback neurons were CaMKII-positive excitatory neurons, and few of them were identified as inhibitory GABAergic neurons. These results may argue against the segregation of ventral and dorsal streams during visual information processing, and support "reverse hierarchy theory" or interactive model proposing that recurrent connections between V1 and higher-order visual areas constitute the functional circuits that mediate visual perception. Also, the corticocortical feedback neurons from high-level visual cortical areas to the V1 area are mostly excitatory in nature.
Collapse
Affiliation(s)
- Huijun Pan
- College of Life Sciences, Anhui Normal University, Wuhu, China
| | - Shen Zhang
- College of Life Sciences, Anhui Normal University, Wuhu, China
| | - Deng Pan
- College of Life Sciences, Anhui Normal University, Wuhu, China
| | - Zheng Ye
- College of Life Sciences, Anhui Normal University, Wuhu, China
| | - Hao Yu
- College of Life Sciences, Anhui Normal University, Wuhu, China
| | - Jian Ding
- College of Life Sciences, Anhui Normal University, Wuhu, China
| | - Qin Wang
- College of Life Sciences, Anhui Normal University, Wuhu, China
| | - Qingyan Sun
- College of Life Sciences, Anhui Normal University, Wuhu, China
| | - Tianmiao Hua
- College of Life Sciences, Anhui Normal University, Wuhu, China
| |
Collapse
|
6
|
Seijdel N, Tsakmakidis N, de Haan EHF, Bohte SM, Scholte HS. Depth in convolutional neural networks solves scene segmentation. PLoS Comput Biol 2020; 16:e1008022. [PMID: 32706770 PMCID: PMC7406083 DOI: 10.1371/journal.pcbi.1008022] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2019] [Revised: 08/05/2020] [Accepted: 06/06/2020] [Indexed: 01/25/2023] Open
Abstract
Feed-forward deep convolutional neural networks (DCNNs) are, under specific conditions, matching and even surpassing human performance in object recognition in natural scenes. This performance suggests that the analysis of a loose collection of image features could support the recognition of natural object categories, without dedicated systems to solve specific visual subtasks. Research in humans however suggests that while feedforward activity may suffice for sparse scenes with isolated objects, additional visual operations ('routines') that aid the recognition process (e.g. segmentation or grouping) are needed for more complex scenes. Linking human visual processing to performance of DCNNs with increasing depth, we here explored if, how, and when object information is differentiated from the backgrounds they appear on. To this end, we controlled the information in both objects and backgrounds, as well as the relationship between them by adding noise, manipulating background congruence and systematically occluding parts of the image. Results indicate that with an increase in network depth, there is an increase in the distinction between object- and background information. For more shallow networks, results indicated a benefit of training on segmented objects. Overall, these results indicate that, de facto, scene segmentation can be performed by a network of sufficient depth. We conclude that the human brain could perform scene segmentation in the context of object identification without an explicit mechanism, by selecting or "binding" features that belong to the object and ignoring other features, in a manner similar to a very deep convolutional neural network.
Collapse
Affiliation(s)
- Noor Seijdel
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
- Amsterdam Brain & Cognition (ABC) Center, University of Amsterdam, Amsterdam, The Netherlands
| | - Nikos Tsakmakidis
- Machine Learning Group, Centrum Wiskunde & Informatica, Amsterdam, the Netherlands
| | - Edward H. F. de Haan
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
- Amsterdam Brain & Cognition (ABC) Center, University of Amsterdam, Amsterdam, The Netherlands
| | - Sander M. Bohte
- Machine Learning Group, Centrum Wiskunde & Informatica, Amsterdam, the Netherlands
| | - H. Steven Scholte
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
- Amsterdam Brain & Cognition (ABC) Center, University of Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
7
|
Ben Naceur M, Akil M, Saouli R, Kachouri R. Fully automatic brain tumor segmentation with deep learning-based selective attention using overlapping patches and multi-class weighted cross-entropy. Med Image Anal 2020; 63:101692. [PMID: 32417714 DOI: 10.1016/j.media.2020.101692] [Citation(s) in RCA: 38] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2019] [Revised: 03/18/2020] [Accepted: 03/19/2020] [Indexed: 02/08/2023]
Abstract
In this paper, we present a new Deep Convolutional Neural Networks (CNNs) dedicated to fully automatic segmentation of Glioblastoma brain tumors with high- and low-grade. The proposed CNNs model is inspired by the Occipito-Temporal pathway which has a special function called selective attention that uses different receptive field sizes in successive layers to figure out the crucial objects in a scene. Thus, using selective attention technique to develop the CNNs model, helps to maximize the extraction of relevant features from MRI images. We have also addressed two more issues: class-imbalance, and the spatial relationship among image Patches. To address the first issue, we propose two steps: an equal sampling of images Patches and an experimental analysis of the effect of weighted cross-entropy loss function on the segmentation results. In addition, to overcome the second issue, we have studied the effect of Overlapping Patches against Adjacent Patches where the Overlapping Patches show better segmentation results due to the introduction of the global context as well as the local features of the image Patches compared to the conventionnel Adjacent Patches. Our experiment results are reported on BRATS-2018 dataset where our End-to-End Deep Learning model achieved state-of-the-art performance. The median Dice score of our fully automatic segmentation model is 0.90, 0.83, 0.83 for the whole tumor, tumor core, and enhancing tumor respectively compared to the Dice score of radiologist, that is in the range 74% - 85%. Moreover, our proposed CNNs model is not only computationally efficient at inference time, but it could segment the whole brain on average 12 seconds. Finally, the proposed Deep Learning model provides an accurate and reliable segmentation result, and that makes it suitable for adopting in research and as a part of different clinical settings.
Collapse
Affiliation(s)
- Mostefa Ben Naceur
- Gaspard Monge Computer Science Laboratory, Univ Gustave Eiffel, CNRS, ESIEE Paris, F-77454 Marne-la-Vallée, France; Smart Computer Sciences Laboratory, Computer Sciences Department, Exact.Sc, and SNL, University of Biskra, Algeria.
| | - Mohamed Akil
- Gaspard Monge Computer Science Laboratory, Univ Gustave Eiffel, CNRS, ESIEE Paris, F-77454 Marne-la-Vallée, France.
| | - Rachida Saouli
- Smart Computer Sciences Laboratory, Computer Sciences Department, Exact.Sc, and SNL, University of Biskra, Algeria.
| | - Rostom Kachouri
- Gaspard Monge Computer Science Laboratory, Univ Gustave Eiffel, CNRS, ESIEE Paris, F-77454 Marne-la-Vallée, France.
| |
Collapse
|
8
|
Jerath R, Beveridge C, Jensen M. On the Hierarchical Organization of Oscillatory Assemblies: Layered Superimposition and a Global Bioelectric Framework. Front Hum Neurosci 2019; 13:426. [PMID: 31866845 PMCID: PMC6904282 DOI: 10.3389/fnhum.2019.00426] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2018] [Accepted: 11/18/2019] [Indexed: 01/23/2023] Open
Abstract
Bioelectric oscillations occur throughout the nervous system of nearly all animals, revealed to play an important role in various aspects of cognitive activity such as information processing and feature binding. Modern research into this dynamic and intrinsic bioelectric activity of neural cells continues to raise questions regarding their role in consciousness and cognition. In this theoretical article, we assert a novel interpretation of the hierarchical nature of "brain waves" by identifying that the superposition of multiple oscillations varying in frequency corresponds to the superimposing of the contents of consciousness and cognition. In order to describe this isomorphism, we present a layered model of the global functional oscillations of various frequencies which act as a part of a unified metastable continuum described by the Operational Architectonics theory and suggested to be responsible for the emergence of the phenomenal mind. We detail the purposes, functions, and origins of each layer while proposing our main theory that the superimposition of these oscillatory layers mirrors the superimposition of the components of the integrated phenomenal experience as well as of cognition. In contrast to the traditional view that localizations of high and low-frequency activity are spatially distinct, many authors have suggested a hierarchical nature to oscillations. Our theoretical interpretation is founded in four layers which correlate not only in frequency but in evolutionary development. As other authors have done, we explore how these layers correlate to the phenomenology of human experience. Special importance is placed on the most basal layer of slow oscillations in coordinating and grouping all of the other layers. By detailing the isomorphism between the phenomenal and physiologic aspects of how lower frequency layers provide a foundation for higher frequency layers to be organized upon, we provide a further means to elucidate physiological and cognitive mechanisms of mind and for the well-researched outcomes of certain voluntary breathing patterns and meditative practices which modulate the mind and have therapeutic effects for psychiatric and other disorders.
Collapse
Affiliation(s)
- Ravinder Jerath
- Charitable Medical Healthcare Foundation, Augusta, GA, United States
| | - Connor Beveridge
- Charitable Medical Healthcare Foundation, Augusta, GA, United States
| | - Michael Jensen
- Department of Medical Illustration, Augusta University, Augusta, GA, United States
| |
Collapse
|
9
|
Tsotsos JK, Kotseruba I, Wloka C. Rapid visual categorization is not guided by early salience-based selection. PLoS One 2019; 14:e0224306. [PMID: 31648265 PMCID: PMC6812801 DOI: 10.1371/journal.pone.0224306] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2019] [Accepted: 10/11/2019] [Indexed: 11/19/2022] Open
Abstract
The current dominant visual processing paradigm in both human and machine research is the feedforward, layered hierarchy of neural-like processing elements. Within this paradigm, visual saliency is seen by many to have a specific role, namely that of early selection. Early selection is thought to enable very fast visual performance by limiting processing to only the most salient candidate portions of an image. This strategy has led to a plethora of saliency algorithms that have indeed improved processing time efficiency in machine algorithms, which in turn have strengthened the suggestion that human vision also employs a similar early selection strategy. However, at least one set of critical tests of this idea has never been performed with respect to the role of early selection in human vision. How would the best of the current saliency models perform on the stimuli used by experimentalists who first provided evidence for this visual processing paradigm? Would the algorithms really provide correct candidate sub-images to enable fast categorization on those same images? Do humans really need this early selection for their impressive performance? Here, we report on a new series of tests of these questions whose results suggest that it is quite unlikely that such an early selection process has any role in human rapid visual categorization.
Collapse
Affiliation(s)
- John K. Tsotsos
- Department of Electrical Engineering and Computer Science, York University, Toronto, ON, Canada
| | - Iuliia Kotseruba
- Department of Electrical Engineering and Computer Science, York University, Toronto, ON, Canada
| | - Calden Wloka
- Department of Electrical Engineering and Computer Science, York University, Toronto, ON, Canada
| |
Collapse
|
10
|
Greven CU, Lionetti F, Booth C, Aron EN, Fox E, Schendan HE, Pluess M, Bruining H, Acevedo B, Bijttebier P, Homberg J. Sensory Processing Sensitivity in the context of Environmental Sensitivity: A critical review and development of research agenda. Neurosci Biobehav Rev 2019; 98:287-305. [DOI: 10.1016/j.neubiorev.2019.01.009] [Citation(s) in RCA: 76] [Impact Index Per Article: 15.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2018] [Revised: 01/08/2019] [Accepted: 01/08/2019] [Indexed: 12/18/2022]
|
11
|
Neri P. Object segmentation controls image reconstruction from natural scenes. PLoS Biol 2017; 15:e1002611. [PMID: 28827801 PMCID: PMC5565198 DOI: 10.1371/journal.pbio.1002611] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2016] [Accepted: 07/25/2017] [Indexed: 11/22/2022] Open
Abstract
The structure of the physical world projects images onto our eyes. However, those images are often poorly representative of environmental structure: well-defined boundaries within the eye may correspond to irrelevant features of the physical world, while critical features of the physical world may be nearly invisible at the retinal projection. The challenge for the visual cortex is to sort these two types of features according to their utility in ultimately reconstructing percepts and interpreting the constituents of the scene. We describe a novel paradigm that enabled us to selectively evaluate the relative role played by these two feature classes in signal reconstruction from corrupted images. Our measurements demonstrate that this process is quickly dominated by the inferred structure of the environment, and only minimally controlled by variations of raw image content. The inferential mechanism is spatially global and its impact on early visual cortex is fast. Furthermore, it retunes local visual processing for more efficient feature extraction without altering the intrinsic transduction noise. The basic properties of this process can be partially captured by a combination of small-scale circuit models and large-scale network architectures. Taken together, our results challenge compartmentalized notions of bottom-up/top-down perception and suggest instead that these two modes are best viewed as an integrated perceptual mechanism. Biological vision is designed to discover the structure of the environment around us. To do this, it relies on ambiguous and often misleading information from the eyes: the boundary of a critical object may be invisible against a background of similar appearance, and may be overlooked in favour of the sharp contour projected by an irrelevant shadow. It remains unclear how human vision sorts different image features according to their relevance to the layout of objects within the scene. We demonstrate that vision achieves this goal via a specialized perceptual system for object segmentation that is one and the same with the feature extraction system: immediately after information is relayed to cortex by the eyes, the process of reconstructing image content from local features is controlled by a dedicated inferential mechanism that attempts to recover the underlying environmental structure; perception is quickly organized around the operation of this mechanism, which becomes the primary contextual influence on image reconstruction. The integrated nature of this perceptual mechanism defies current notions of separate top-down and bottom-up processes, offering a fresh view of how human vision operates on natural signals.
Collapse
Affiliation(s)
- Peter Neri
- Laboratoire des Systèmes Perceptifs, Département d'études cognitives, Ecole Normale Supérieure, PSL Research University, CNRS, Paris, France
- * E-mail:
| |
Collapse
|
12
|
Tsotsos JK. Complexity Level Analysis Revisited: What Can 30 Years of Hindsight Tell Us about How the Brain Might Represent Visual Information? Front Psychol 2017; 8:1216. [PMID: 28848458 PMCID: PMC5552749 DOI: 10.3389/fpsyg.2017.01216] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2016] [Accepted: 07/03/2017] [Indexed: 11/13/2022] Open
Abstract
Much has been written about how the biological brain might represent and process visual information, and how this might inspire and inform machine vision systems. Indeed, tremendous progress has been made, and especially during the last decade in the latter area. However, a key question seems too often, if not mostly, be ignored. This question is simply: do proposed solutions scale with the reality of the brain's resources? This scaling question applies equally to brain and to machine solutions. A number of papers have examined the inherent computational difficulty of visual information processing using theoretical and empirical methods. The main goal of this activity had three components: to understand the deep nature of the computational problem of visual information processing; to discover how well the computational difficulty of vision matches to the fixed resources of biological seeing systems; and, to abstract from the matching exercise the key principles that lead to the observed characteristics of biological visual performance. This set of components was termed complexity level analysis in Tsotsos (1987) and was proposed as an important complement to Marr's three levels of analysis. This paper revisits that work with the advantage that decades of hindsight can provide.
Collapse
Affiliation(s)
- John K Tsotsos
- Department of Electrical Engineering and Computer Science, York UniversityToronto, ON, Canada
| |
Collapse
|
13
|
Flashing Lights Induce Prolonged Distortions in Visual Cortical Responses and Visual Perception. eNeuro 2017; 4:eN-NWR-0304-16. [PMID: 28508035 PMCID: PMC5429040 DOI: 10.1523/eneuro.0304-16.2017] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2016] [Revised: 04/24/2017] [Accepted: 04/28/2017] [Indexed: 01/01/2023] Open
Abstract
The primary sensory neocortex generates an internal representation of the environment, and its circuit reorganization is thought to lead to a modification of sensory perception. This reorganization occurs primarily through activity-dependent plasticity and has been well documented in animals during early developmental stages. Here, we describe a new method for the noninvasive induction of long-term plasticity in the mature brain: simple transient visual stimuli (i.e., flashing lights) can be used to induce prolonged modifications in visual cortical processing and visually driven behaviors. Our previous studies have shown that, in the primary visual cortex (V1) of mice, a flashing light stimulus evokes a long-delayed response that persists for seconds. When the mice were repetitively presented with drifting grating stimuli (conditioned stimuli) during the flash stimulus-evoked delayed response period, the V1 neurons exhibited a long-lasting decrease in responsiveness to the conditioned stimuli. The flash stimulus-induced underrepresentation of the grating motion was specific to the direction of the conditioned stimuli and was associated with a decrease in the animal's ability to detect the motion of the drifting gratings. The neurophysiological and behavioral plasticity both persisted for at least several hours and required N-methyl-d-aspartate receptor activation in the visual cortex. We propose that flashing light stimuli can be used as an experimental tool to investigate the visual function and plasticity of neuronal representations and perception after a critical period of neocortical plasticity.
Collapse
|
14
|
Rodríguez-Sánchez AJ, Fallah M, Leonardis A. Editorial: Hierarchical Object Representations in the Visual Cortex and Computer Vision. Front Comput Neurosci 2015; 9:142. [PMID: 26635595 PMCID: PMC4653288 DOI: 10.3389/fncom.2015.00142] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2015] [Accepted: 11/06/2015] [Indexed: 11/29/2022] Open
Affiliation(s)
- Antonio J Rodríguez-Sánchez
- Intelligent and Interactive Systems, Department of Computer Science, University of Innsbruck Innsbruck, Austria
| | - Mazyar Fallah
- Visual Perception and Attention Laboratory, Centre for Vision Research, School of Kinesiology and Health Science, York University Toronto, ON, Canada
| | - Aleš Leonardis
- School of Computer Science, University of Birmingham Birmingham, UK
| |
Collapse
|
15
|
Tsotsos JK. Computational abstraction towards a theory of the brain. Curr Biol 2015. [DOI: 10.1016/j.cub.2015.06.074] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|