1
|
Hermansen E, Klindt DA, Dunn BA. Uncovering 2-D toroidal representations in grid cell ensemble activity during 1-D behavior. Nat Commun 2024; 15:5429. [PMID: 38926360 PMCID: PMC11208534 DOI: 10.1038/s41467-024-49703-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Accepted: 06/13/2024] [Indexed: 06/28/2024] Open
Abstract
Minimal experiments, such as head-fixed wheel-running and sleep, offer experimental advantages but restrict the amount of observable behavior, making it difficult to classify functional cell types. Arguably, the grid cell, and its striking periodicity, would not have been discovered without the perspective provided by free behavior in an open environment. Here, we show that by shifting the focus from single neurons to populations, we change the minimal experimental complexity required. We identify grid cell modules and show that the activity covers a similar, stable toroidal state space during wheel running as in open field foraging. Trajectories on grid cell tori correspond to single trial runs in virtual reality and path integration in the dark, and the alignment of the representation rapidly shifts with changes in experimental conditions. Thus, we provide a methodology to discover and study complex internal representations in even the simplest of experiments.
Collapse
Affiliation(s)
- Erik Hermansen
- Department of Mathematical Sciences, NTNU, Trondheim, Norway.
| | - David A Klindt
- Department of Mathematical Sciences, NTNU, Trondheim, Norway
- Cold Spring Harbor Laboratory, Cold Spring Harbor, Laurel Hollow, New York, USA
| | - Benjamin A Dunn
- Department of Mathematical Sciences, NTNU, Trondheim, Norway.
| |
Collapse
|
2
|
Sun SH, Killian NJ, Pezaris JS. More than expected: extracellular waveforms and functional responses in monkey LGN. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.11.22.568065. [PMID: 38798485 PMCID: PMC11118448 DOI: 10.1101/2023.11.22.568065] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/29/2024]
Abstract
Unlike the exhaustive determination of cell types in the retina, key populations in the lateral geniculate nucleus of the thalamus (LGN) may have been missed. Here, we have begun to characterize the full range of extracellular neuronal responses in the LGN of awake monkeys using multi-electrodes during the presentation of colored noise visual stimuli to identify any previously overlooked signals. Extracellular spike waveforms of single units were classified into seven distinct classes, revealing previously unrecognized diversity: four negative-dominant classes that were narrow or broad, one triphasic class, and two positive-dominant classes. Based on their mapped receptive field (RF), these units were further categorized into either magnocellular (M), parvocellular (P), koniocellular (K), or non-RF (N). We found correlations between spike shape and mapped RF and response characteristics, with negative and narrow spiking waveform units predominantly associated with P and N RFs, and positive waveforms mostly linked to M RFs. Responses from positive waveforms exhibited shorter latencies, larger RF sizes, and were associated with larger eccentricities in the visual field than the other waveform classes. Additionally, N cells, those without an estimated RF, were consistently responsive to the visually presented mapping stimulus at a lower and more sustained rate than units with an RF. These findings suggest that the LGN cell population may be more diverse than previously believed.
Collapse
Affiliation(s)
- Shi Hai Sun
- Department of Neurosurgery, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurosurgery, Harvard Medical School, Boston, MA, USA
| | - Nathaniel J Killian
- Department of Neurological Surgery, Albert Einstein College of Medicine, Bronx, NY, USA
| | - John S Pezaris
- Department of Neurosurgery, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurosurgery, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
3
|
Golan T, Taylor J, Schütt H, Peters B, Sommers RP, Seeliger K, Doerig A, Linton P, Konkle T, van Gerven M, Kording K, Richards B, Kietzmann TC, Lindsay GW, Kriegeskorte N. Deep neural networks are not a single hypothesis but a language for expressing computational hypotheses. Behav Brain Sci 2023; 46:e392. [PMID: 38054329 DOI: 10.1017/s0140525x23001553] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/07/2023]
Abstract
An ideal vision model accounts for behavior and neurophysiology in both naturalistic conditions and designed lab experiments. Unlike psychological theories, artificial neural networks (ANNs) actually perform visual tasks and generate testable predictions for arbitrary inputs. These advantages enable ANNs to engage the entire spectrum of the evidence. Failures of particular models drive progress in a vibrant ANN research program of human vision.
Collapse
Affiliation(s)
- Tal Golan
- Department of Cognitive and Brain Sciences, Ben-Gurion University of the Negev, Be'er Sheva, Israel
| | - JohnMark Taylor
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA ://linton.vision/
| | - Heiko Schütt
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA ://linton.vision/
- Center for Neural Science, New York University, New York, NY, USA
| | - Benjamin Peters
- School of Psychology & Neuroscience, University of Glasgow, Glasgow, UK
| | - Rowan P Sommers
- Department of Neurobiology of Language, Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
| | - Katja Seeliger
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Adrien Doerig
- Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany
| | - Paul Linton
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA ://linton.vision/
- Presidential Scholars in Society and Neuroscience, Center for Science and Society, Columbia University, New York, NY, USA
- Italian Academy for Advanced Studies in America, Columbia University, New York, NY, USA
| | - Talia Konkle
- Department of Psychology and Center for Brain Sciences, Harvard University, Cambridge, MA, USA ://konklab.fas.harvard.edu/
| | - Marcel van Gerven
- Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlandsartcogsys.com
| | - Konrad Kording
- Departments of Bioengineering and Neuroscience, University of Pennsylvania, Philadelphia, PA, USA
- Learning in Machines and Brains Program, CIFAR, Toronto, ON, Canada
| | - Blake Richards
- Learning in Machines and Brains Program, CIFAR, Toronto, ON, Canada
- Mila, Montreal, QC, Canada
- School of Computer Science, McGill University, Montreal, QC, Canada
- Department of Neurology & Neurosurgery, McGill University, Montreal, QC, Canada
- Montreal Neurological Institute, Montreal, QC, Canada
| | - Tim C Kietzmann
- Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany
| | - Grace W Lindsay
- Department of Psychology and Center for Data Science, New York University, New York, NY, USA
| | - Nikolaus Kriegeskorte
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA ://linton.vision/
- Departments of Psychology, Neuroscience, and Electrical Engineering, Columbia University, New York, NY, USA
| |
Collapse
|
4
|
Fang Z, Bloem IM, Olsson C, Ma WJ, Winawer J. Normalization by orientation-tuned surround in human V1-V3. PLoS Comput Biol 2023; 19:e1011704. [PMID: 38150484 PMCID: PMC10793941 DOI: 10.1371/journal.pcbi.1011704] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Revised: 01/17/2024] [Accepted: 11/20/2023] [Indexed: 12/29/2023] Open
Abstract
An influential account of neuronal responses in primary visual cortex is the normalized energy model. This model is often implemented as a multi-stage computation. The first stage is linear filtering. The second stage is the extraction of contrast energy, whereby a complex cell computes the squared and summed outputs of a pair of the linear filters in quadrature phase. The third stage is normalization, in which a local population of complex cells mutually inhibit one another. Because the population includes cells tuned to a range of orientations and spatial frequencies, the result is that the responses are effectively normalized by the local stimulus contrast. Here, using evidence from human functional MRI, we show that the classical model fails to account for the relative responses to two classes of stimuli: straight, parallel, band-passed contours (gratings), and curved, band-passed contours (snakes). The snakes elicit fMRI responses that are about twice as large as the gratings, yet a traditional divisive normalization model predicts responses that are about the same. Motivated by these observations and others from the literature, we implement a divisive normalization model in which cells matched in orientation tuning ("tuned normalization") preferentially inhibit each other. We first show that this model accounts for differential responses to these two classes of stimuli. We then show that the model successfully generalizes to other band-pass textures, both in V1 and in extrastriate cortex (V2 and V3). We conclude that even in primary visual cortex, complex features of images such as the degree of heterogeneity, can have large effects on neural responses.
Collapse
Affiliation(s)
- Zeming Fang
- Department of Psychology and Center for Neural Science, New York University, New York City, New York, United States of America
- Department of Cognitive Science, Rensselaer Polytechnic Institute, Troy, New York, United States of America
| | - Ilona M. Bloem
- Department of Psychology and Center for Neural Science, New York University, New York City, New York, United States of America
| | - Catherine Olsson
- Department of Psychology and Center for Neural Science, New York University, New York City, New York, United States of America
| | - Wei Ji Ma
- Department of Psychology and Center for Neural Science, New York University, New York City, New York, United States of America
| | - Jonathan Winawer
- Department of Psychology and Center for Neural Science, New York University, New York City, New York, United States of America
| |
Collapse
|
5
|
Anada S, Nomura Y, Yamamoto K. Enhancing performance of electron holography with mathematical and machine learning-based denoising techniques. Microscopy (Oxf) 2023; 72:461-484. [PMID: 37428597 DOI: 10.1093/jmicro/dfad037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 06/13/2023] [Accepted: 07/09/2023] [Indexed: 07/12/2023] Open
Abstract
Electron holography is a useful tool for analyzing functional properties, such as electromagnetic fields and strains of materials and devices. The performance of electron holography is limited by the 'shot noise' inherent in electron micrographs (holograms), which are composed of a finite number of electrons. A promising approach for addressing this issue is to use mathematical and machine learning-based image-processing techniques for hologram denoising. With the advancement of information science, denoising methods have become capable of extracting signals that are completely buried in noise, and they are being applied to electron microscopy, including electron holography. However, these advanced denoising methods are complex and have many parameters to be tuned; therefore, it is necessary to understand their principles in depth and use them carefully. Herein, we present an overview of the principles and usage of sparse coding, the wavelet hidden Markov model and tensor decomposition, which have been applied to electron holography. We also present evaluation results for the denoising performance of these methods obtained through their application to simulated and experimentally recorded holograms. Our analysis, review and comparison of the methods clarify the impact of denoising on electron holography research.
Collapse
Affiliation(s)
- Satoshi Anada
- Nanostructures Research Laboratory, Japan Fine Ceramics Center, 2-4-1 Mutsuno, Atsuta-ku, Nagoya, Aichi 456-8587, Japan
| | - Yuki Nomura
- Nanostructures Research Laboratory, Japan Fine Ceramics Center, 2-4-1 Mutsuno, Atsuta-ku, Nagoya, Aichi 456-8587, Japan
| | - Kazuo Yamamoto
- Nanostructures Research Laboratory, Japan Fine Ceramics Center, 2-4-1 Mutsuno, Atsuta-ku, Nagoya, Aichi 456-8587, Japan
| |
Collapse
|
6
|
Cowley BR, Stan PL, Pillow JW, Smith MA. Compact deep neural network models of visual cortex. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.11.22.568315. [PMID: 38045255 PMCID: PMC10690296 DOI: 10.1101/2023.11.22.568315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/05/2023]
Abstract
A powerful approach to understanding the computations carried out in visual cortex is to develop models that predict neural responses to arbitrary images. Deep neural network (DNN) models have worked remarkably well at predicting neural responses [1, 2, 3], yet their underlying computations remain buried in millions of parameters. Have we simply replaced one complicated system in vivo with another in silico ? Here, we train a data-driven deep ensemble model that predicts macaque V4 responses ∼50% more accurately than currently-used task-driven DNN models. We then compress this deep ensemble to identify compact models that have 5,000x fewer parameters yet equivalent accuracy as the deep ensemble. We verified that the stimulus preferences of the compact models matched those of the real V4 neurons by measuring V4 responses to both 'maximizing' and adversarial images generated using compact models. We then analyzed the inner workings of the compact models and discovered a common circuit motif: Compact models share a similar set of filters in early stages of processing but then specialize by heavily consolidating this shared representation with a precise readout. This suggests that a V4 neuron's stimulus preference is determined entirely by its consolidation step. To demonstrate this, we investigated the compression step of a dot-detecting compact model and found a set of simple computations that may be carried out by dot-selective V4 neurons. Overall, our work demonstrates that the DNN models currently used in computational neuroscience are needlessly large; our approach provides a new way forward for obtaining explainable, high-accuracy models of visual cortical neurons.
Collapse
|
7
|
Li X, Wang S. Simple and complex cells revisited: toward a selectivity-invariance model of object recognition. Front Comput Neurosci 2023; 17:1282828. [PMID: 37905187 PMCID: PMC10613527 DOI: 10.3389/fncom.2023.1282828] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Accepted: 09/19/2023] [Indexed: 11/02/2023] Open
Abstract
This paper presents a theoretical perspective on modeling ventral stream processing by revisiting the computational abstraction of simple and complex cells. In parallel to David Marr's vision theory, we organize the new perspective into three levels. At the computational level, we abstract simple and complex cells into space partitioning and composition in a topological space based on the redundancy exploitation hypothesis of Horace Barlow. At the algorithmic level, we present a hierarchical extension of sparse coding by exploiting the manifold constraint in high-dimensional space (i.e., the blessing of dimensionality). The resulting over-parameterized models for object recognition differ from existing hierarchical models by disentangling the objectives of selectivity and invariance computation. It is possible to interpret our hierarchical construction as a computational implementation of cortically local subspace untangling for object recognition and face representation, which are closely related to exemplar-based and axis-based coding in the medial temporal lobe. At the implementation level, we briefly discuss two possible implementations based on asymmetric sparse autoencoders and divergent spiking neural networks.
Collapse
Affiliation(s)
- Xin Li
- Department of Computer Science, University at Albany, Albany, NY, United States
| | - Shuo Wang
- Department of Radiology, Washington University at St. Louis, St. Louis, MO, United States
| |
Collapse
|
8
|
Luna R, Zabaleta I, Bertalmío M. State-of-the-art image and video quality assessment with a metric based on an intrinsically non-linear neural summation model. Front Neurosci 2023; 17:1222815. [PMID: 37559700 PMCID: PMC10408451 DOI: 10.3389/fnins.2023.1222815] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Accepted: 06/30/2023] [Indexed: 08/11/2023] Open
Abstract
The development of automatic methods for image and video quality assessment that correlate well with the perception of human observers is a very challenging open problem in vision science, with numerous practical applications in disciplines such as image processing and computer vision, as well as in the media industry. In the past two decades, the goal of image quality research has been to improve upon classical metrics by developing models that emulate some aspects of the visual system, and while the progress has been considerable, state-of-the-art quality assessment methods still share a number of shortcomings, like their performance dropping considerably when they are tested on a database that is quite different from the one used to train them, or their significant limitations in predicting observer scores for high framerate videos. In this work we propose a novel objective method for image and video quality assessment that is based on the recently introduced Intrinsically Non-linear Receptive Field (INRF) formulation, a neural summation model that has been shown to be better at predicting neural activity and visual perception phenomena than the classical linear receptive field. Here we start by optimizing, on a classic image quality database, the four parameters of a very simple INRF-based metric, and proceed to test this metric on three other databases, showing that its performance equals or surpasses that of the state-of-the-art methods, some of them having millions of parameters. Next, we extend to the temporal domain this INRF image quality metric, and test it on several popular video quality datasets; again, the results of our proposed INRF-based video quality metric are shown to be very competitive.
Collapse
Affiliation(s)
- Raúl Luna
- Institute of Optics, Spanish National Research Council (CSIC), Madrid, Spain
| | - Itziar Zabaleta
- Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain
| | - Marcelo Bertalmío
- Institute of Optics, Spanish National Research Council (CSIC), Madrid, Spain
| |
Collapse
|
9
|
de Vries SEJ, Siegle JH, Koch C. Sharing neurophysiology data from the Allen Brain Observatory. eLife 2023; 12:e85550. [PMID: 37432073 PMCID: PMC10335829 DOI: 10.7554/elife.85550] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Accepted: 06/27/2023] [Indexed: 07/12/2023] Open
Abstract
Nullius in verba ('trust no one'), chosen as the motto of the Royal Society in 1660, implies that independently verifiable observations-rather than authoritative claims-are a defining feature of empirical science. As the complexity of modern scientific instrumentation has made exact replications prohibitive, sharing data is now essential for ensuring the trustworthiness of one's findings. While embraced in spirit by many, in practice open data sharing remains the exception in contemporary systems neuroscience. Here, we take stock of the Allen Brain Observatory, an effort to share data and metadata associated with surveys of neuronal activity in the visual system of laboratory mice. Data from these surveys have been used to produce new discoveries, to validate computational algorithms, and as a benchmark for comparison with other data, resulting in over 100 publications and preprints to date. We distill some of the lessons learned about open surveys and data reuse, including remaining barriers to data sharing and what might be done to address these.
Collapse
|
10
|
Benigno GB, Budzinski RC, Davis ZW, Reynolds JH, Muller L. Waves traveling over a map of visual space can ignite short-term predictions of sensory input. Nat Commun 2023; 14:3409. [PMID: 37296131 PMCID: PMC10256723 DOI: 10.1038/s41467-023-39076-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Accepted: 05/25/2023] [Indexed: 06/12/2023] Open
Abstract
Recent analyses have found waves of neural activity traveling across entire visual cortical areas in awake animals. These traveling waves modulate the excitability of local networks and perceptual sensitivity. The general computational role of these spatiotemporal patterns in the visual system, however, remains unclear. Here, we hypothesize that traveling waves endow the visual system with the capacity to predict complex and naturalistic inputs. We present a network model whose connections can be rapidly and efficiently trained to predict individual natural movies. After training, a few input frames from a movie trigger complex wave patterns that drive accurate predictions many frames into the future solely from the network's connections. When the recurrent connections that drive waves are randomly shuffled, both traveling waves and the ability to predict are eliminated. These results suggest traveling waves may play an essential computational role in the visual system by embedding continuous spatiotemporal structures over spatial maps.
Collapse
Affiliation(s)
- Gabriel B Benigno
- Department of Mathematics, Western University, London, ON, Canada
- Brain and Mind Institute, Western University, London, ON, Canada
- Western Academy for Advanced Research, Western University, London, ON, Canada
| | - Roberto C Budzinski
- Department of Mathematics, Western University, London, ON, Canada
- Brain and Mind Institute, Western University, London, ON, Canada
- Western Academy for Advanced Research, Western University, London, ON, Canada
| | - Zachary W Davis
- The Salk Institute for Biological Studies, La Jolla, CA, USA
| | - John H Reynolds
- The Salk Institute for Biological Studies, La Jolla, CA, USA
| | - Lyle Muller
- Department of Mathematics, Western University, London, ON, Canada.
- Brain and Mind Institute, Western University, London, ON, Canada.
- Western Academy for Advanced Research, Western University, London, ON, Canada.
| |
Collapse
|
11
|
Qiu Y, Klindt DA, Szatko KP, Gonschorek D, Hoefling L, Schubert T, Busse L, Bethge M, Euler T. Efficient coding of natural scenes improves neural system identification. PLoS Comput Biol 2023; 19:e1011037. [PMID: 37093861 PMCID: PMC10159360 DOI: 10.1371/journal.pcbi.1011037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Revised: 05/04/2023] [Accepted: 03/20/2023] [Indexed: 04/25/2023] Open
Abstract
Neural system identification aims at learning the response function of neurons to arbitrary stimuli using experimentally recorded data, but typically does not leverage normative principles such as efficient coding of natural environments. Visual systems, however, have evolved to efficiently process input from the natural environment. Here, we present a normative network regularization for system identification models by incorporating, as a regularizer, the efficient coding hypothesis, which states that neural response properties of sensory representations are strongly shaped by the need to preserve most of the stimulus information with limited resources. Using this approach, we explored if a system identification model can be improved by sharing its convolutional filters with those of an autoencoder which aims to efficiently encode natural stimuli. To this end, we built a hybrid model to predict the responses of retinal neurons to noise stimuli. This approach did not only yield a higher performance than the "stand-alone" system identification model, it also produced more biologically plausible filters, meaning that they more closely resembled neural representation in early visual systems. We found these results applied to retinal responses to different artificial stimuli and across model architectures. Moreover, our normatively regularized model performed particularly well in predicting responses of direction-of-motion sensitive retinal neurons. The benefit of natural scene statistics became marginal, however, for predicting the responses to natural movies. In summary, our results indicate that efficiently encoding environmental inputs can improve system identification models, at least for noise stimuli, and point to the benefit of probing the visual system with naturalistic stimuli.
Collapse
Affiliation(s)
- Yongrong Qiu
- Institute for Ophthalmic Research, U Tübingen, Tübingen, Germany
- Centre for Integrative Neuroscience (CIN), U Tübingen, Tübingen, Germany
- Graduate Training Centre of Neuroscience (GTC), International Max Planck Research School, U Tübingen, Tübingen, Germany
| | - David A Klindt
- Institute for Ophthalmic Research, U Tübingen, Tübingen, Germany
- Centre for Integrative Neuroscience (CIN), U Tübingen, Tübingen, Germany
- Department of Mathematical Sciences, Norwegian University of Science and Technology, Trondheim, Norway
| | - Klaudia P Szatko
- Institute for Ophthalmic Research, U Tübingen, Tübingen, Germany
- Centre for Integrative Neuroscience (CIN), U Tübingen, Tübingen, Germany
- Graduate Training Centre of Neuroscience (GTC), International Max Planck Research School, U Tübingen, Tübingen, Germany
- Bernstein Center for Computational Neuroscience, Tübingen, Germany
| | - Dominic Gonschorek
- Institute for Ophthalmic Research, U Tübingen, Tübingen, Germany
- Centre for Integrative Neuroscience (CIN), U Tübingen, Tübingen, Germany
- Research Training Group 2381, U Tübingen, Tübingen, Germany
| | - Larissa Hoefling
- Institute for Ophthalmic Research, U Tübingen, Tübingen, Germany
- Centre for Integrative Neuroscience (CIN), U Tübingen, Tübingen, Germany
- Bernstein Center for Computational Neuroscience, Tübingen, Germany
| | - Timm Schubert
- Institute for Ophthalmic Research, U Tübingen, Tübingen, Germany
- Centre for Integrative Neuroscience (CIN), U Tübingen, Tübingen, Germany
| | - Laura Busse
- Division of Neurobiology, Faculty of Biology, LMU Munich, Planegg-Martinsried, Germany
- Bernstein Center for Computational Neuroscience, Planegg-Martinsried, Germany
| | - Matthias Bethge
- Centre for Integrative Neuroscience (CIN), U Tübingen, Tübingen, Germany
- Bernstein Center for Computational Neuroscience, Tübingen, Germany
- Institute for Theoretical Physics, U Tübingen, Tübingen, Germany
| | - Thomas Euler
- Institute for Ophthalmic Research, U Tübingen, Tübingen, Germany
- Centre for Integrative Neuroscience (CIN), U Tübingen, Tübingen, Germany
- Bernstein Center for Computational Neuroscience, Tübingen, Germany
| |
Collapse
|
12
|
Esposito A, Chiarella SG, Raffone A, Nikolaev AR, van Leeuwen C. Perceptual bias contextualized in visually ambiguous stimuli. Cognition 2023; 230:105284. [PMID: 36174260 DOI: 10.1016/j.cognition.2022.105284] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2021] [Revised: 09/06/2022] [Accepted: 09/09/2022] [Indexed: 11/28/2022]
Abstract
The visual appearance of an object is a function of stimulus properties as well as perceptual biases imposed by the observer. The context-specific trade-off between both can be measured accurately in a perceptual judgment task, involving grouping by proximity in ambiguous dot lattices. Such grouping depends lawfully on a stimulus parameter of the dot lattices known as their aspect ratio (AR), whose effect is modulated by a perceptual bias representing the preference for a cardinal orientation. In two experiments, we investigated how preceding context can lead to bias modulation, either in a top-down fashion via visual working memory (VWM) or bottom-up via sensory priming. In Experiment 1, we embedded the perceptual judgment task in a change detection paradigm and studied how the factors of VWM load (complexity of the memory array) and content (congruency in orientation to the ensuing dot lattice) affect the prominence of perceptual bias. A robust vertical orientation bias was observed, which was increased by VWM load and modulated by congruent VWM content. In Experiment 2, dot lattices were preceded by oriented primes. Here, primes regardless of orientation elicited a vertical orientation bias in dot lattices compared to a neutral baseline. Taken together, the two experiments demonstrate that top-down context (VWM load and content) effectively controls orientation bias modulation, while bottom-up context (i.e., priming) merely acts as an undifferentiated trigger to perceptual bias. These findings characterize the temporal context sensitivity of Gestalt perception, shed light on the processes responsible for different perceptual outcomes of ambiguous stimuli, and identify some of the mechanisms controlling perceptual bias.
Collapse
Affiliation(s)
- Antonino Esposito
- Department of Psychology, Sapienza University of Rome, Italy; Brain and Cognition Research Unit, KU Leuven, Belgium.
| | - Salvatore Gaetano Chiarella
- Department of Psychology, Sapienza University of Rome, Italy; Brain and Cognition Research Unit, KU Leuven, Belgium
| | | | - Andrey R Nikolaev
- Brain and Cognition Research Unit, KU Leuven, Belgium; Department of Psychology, Lund University, Sweden
| | - Cees van Leeuwen
- Brain and Cognition Research Unit, KU Leuven, Belgium; Center for Cognitive Science, TU Kaiserslautern, Germany
| |
Collapse
|
13
|
Hofstetter S, Dumoulin SO. Assessing the ecological validity of numerosity-selective neuronal populations with real-world natural scenes. iScience 2022; 25:105267. [PMID: 36274951 PMCID: PMC9579010 DOI: 10.1016/j.isci.2022.105267] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Revised: 07/18/2022] [Accepted: 09/26/2022] [Indexed: 11/21/2022] Open
Abstract
Animals and humans are able to quickly and effortlessly estimate the number of items in a set: their numerosity. Numerosity perception is thought to be critical to behavior, from feeding to escaping predators to human mathematical cognition. Virtually, all scientific studies on numerosity mechanisms use well controlled but artificial stimuli to isolate the numerosity dimension from other physical quantities. Here, we probed the ecological validity of these artificial stimuli and evaluate whether an important component in numerosity processing, the numerosity-selective neural populations, also respond to numerosity of items in real-world natural scenes. Using 7T MRI and natural images from a wide range of categories, we provide evidence that the numerosity-tuned neuronal populations show numerosity-selective responses when viewing images from a real-world natural scene. Our findings strengthen the role of numerosity-selective neurons in numerosity perception and provide an important link to their function in numerosity perception in real-world settings.
Collapse
Affiliation(s)
- Shir Hofstetter
- The Spinoza Centre for Neuroimaging, Amsterdam, the Netherlands,Department of Computational Cognitive Neuroscience and Neuroimaging, Netherlands Institute for Neuroscience, Amsterdam, the Netherlands,Corresponding author
| | - Serge O. Dumoulin
- The Spinoza Centre for Neuroimaging, Amsterdam, the Netherlands,Department of Computational Cognitive Neuroscience and Neuroimaging, Netherlands Institute for Neuroscience, Amsterdam, the Netherlands,Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, the Netherlands,Department of Experimental and Applied Psychology, VU University, Amsterdam, the Netherlands,Corresponding author
| |
Collapse
|
14
|
Visual evoked feedforward-feedback traveling waves organize neural activity across the cortical hierarchy in mice. Nat Commun 2022; 13:4754. [PMID: 35963850 PMCID: PMC9376099 DOI: 10.1038/s41467-022-32378-x] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2021] [Accepted: 07/27/2022] [Indexed: 12/26/2022] Open
Abstract
Sensory processing is distributed among many brain regions that interact via feedforward and feedback signaling. Neuronal oscillations have been shown to mediate intercortical feedforward and feedback interactions. Yet, the macroscopic structure of the multitude of such oscillations remains unclear. Here, we show that simple visual stimuli reliably evoke two traveling waves with spatial wavelengths that cover much of the cerebral hemisphere in awake mice. 30-50 Hz feedforward waves arise in primary visual cortex (V1) and propagate rostrally, while 3-6 Hz feedback waves originate in the association cortex and flow caudally. The phase of the feedback wave modulates the amplitude of the feedforward wave and synchronizes firing between V1 and parietal cortex. Altogether, these results provide direct experimental evidence that visual evoked traveling waves percolate through the cerebral cortex and coordinate neuronal activity across broadly distributed networks mediating visual processing.
Collapse
|
15
|
Jarvis J, Triantaphillidou S, Gupta G. Contrast discrimination in images of natural scenes. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2022; 39:B50-B64. [PMID: 36215527 DOI: 10.1364/josaa.447390] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/04/2021] [Accepted: 03/30/2022] [Indexed: 06/16/2023]
Abstract
Contrast discrimination determines the threshold contrast required to distinguish between two suprathreshold visual stimuli. It is typically measured using sine-wave gratings. We first present a modification to Barten's semi-mechanistic contrast discrimination model to account for spatial frequency effects and demonstrate how the model can successfully predict visual thresholds obtained from published classical contrast discrimination studies. Contrast discrimination functions are then measured from images of natural scenes, using a psychophysical paradigm based on that employed in our previous study of contrast detection sensitivity. The proposed discrimination model modification is shown to successfully predict discrimination thresholds for structurally very different types of natural image stimuli. A comparison of results shows that, for normal contrast levels in natural scene viewing, contextual contrast detection and discrimination are approximately the same and almost independent of spatial frequency within the range of 1-20 c/deg. At higher frequencies, both sensitivities decrease in magnitude due to optical limitations of the eye. The results are discussed in relation to current image quality models.
Collapse
|
16
|
Miller CT, Gire D, Hoke K, Huk AC, Kelley D, Leopold DA, Smear MC, Theunissen F, Yartsev M, Niell CM. Natural behavior is the language of the brain. Curr Biol 2022; 32:R482-R493. [PMID: 35609550 PMCID: PMC10082559 DOI: 10.1016/j.cub.2022.03.031] [Citation(s) in RCA: 54] [Impact Index Per Article: 27.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
The breadth and complexity of natural behaviors inspires awe. Understanding how our perceptions, actions, and internal thoughts arise from evolved circuits in the brain has motivated neuroscientists for generations. Researchers have traditionally approached this question by focusing on stereotyped behaviors, either natural or trained, in a limited number of model species. This approach has allowed for the isolation and systematic study of specific brain operations, which has greatly advanced our understanding of the circuits involved. At the same time, the emphasis on experimental reductionism has left most aspects of the natural behaviors that have shaped the evolution of the brain largely unexplored. However, emerging technologies and analytical tools make it possible to comprehensively link natural behaviors to neural activity across a broad range of ethological contexts and timescales, heralding new modes of neuroscience focused on natural behaviors. Here we describe a three-part roadmap that aims to leverage the wealth of behaviors in their naturally occurring distributions, linking their variance with that of underlying neural processes to understand how the brain is able to successfully navigate the everyday challenges of animals' social and ecological landscapes. To achieve this aim, experimenters must harness one challenge faced by all neurobiological systems, namely variability, in order to gain new insights into the language of the brain.
Collapse
Affiliation(s)
- Cory T Miller
- Cortical Systems and Behavior Laboratory, University of California San Diego, 9500 Gilman Drive, La Jolla, CA 92039, USA.
| | - David Gire
- Department of Psychology, University of Washington, Guthrie Hall, Seattle, WA 98105, USA
| | - Kim Hoke
- Department of Biology, Colorado State University, 1878 Campus Delivery, Fort Collins, CO 80523, USA
| | - Alexander C Huk
- Center for Perceptual Systems, Departments of Neuroscience and Psychology, University of Texas at Austin, 116 Inner Campus Drive, Austin, TX 78712, USA
| | - Darcy Kelley
- Department of Biological Sciences, Columbia University, 1212 Amsterdam Avenue, New York, NY 10027, USA
| | - David A Leopold
- Section of Cognitive Neurophysiology and Imaging, National Institute of Mental Health, 49 Convent Drive, Bethesda, MD 20892, USA
| | - Matthew C Smear
- Department of Psychology and Institute of Neuroscience, University of Oregon, 1227 University Street, Eugene, OR 97403, USA
| | - Frederic Theunissen
- Department of Psychology, University of California Berkeley, 2121 Berkeley Way, Berkeley, CA 94720, USA
| | - Michael Yartsev
- Department of Bioengineering, University of California Berkeley, 306 Stanley Hall, Berkeley, CA 94720, USA
| | - Cristopher M Niell
- Department of Biology and Institute of Neuroscience, University of Oregon, 222 Huestis Hall, Eugene, OR 97403, USA.
| |
Collapse
|
17
|
Cho S, Roy A, Liu CJ, Idiyatullin D, Zhu W, Zhang Y, Zhu XH, O'Herron P, Leikvoll A, Chen W, Kara P, Uğurbil K. Cortical layer-specific differences in stimulus selectivity revealed with high-field fMRI and single-vessel resolution optical imaging of the primary visual cortex. Neuroimage 2022; 251:118978. [PMID: 35143974 PMCID: PMC9048976 DOI: 10.1016/j.neuroimage.2022.118978] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2021] [Revised: 01/26/2022] [Accepted: 02/05/2022] [Indexed: 11/23/2022] Open
Abstract
The mammalian neocortex exhibits a stereotypical laminar organization, with feedforward inputs arriving primarily into layer 4, local computations shaping response selectivity in layers 2/3, and outputs to other brain areas emanating via layers 2/3, 5 and 6. It cannot be assumed a priori that these signatures of laminar differences in neuronal circuitry are reflected in hemodynamic signals that form the basis of functional magnetic resonance imaging (fMRI). Indeed, optical imaging of single-vessel functional responses has highlighted the potential limits of using vascular signals as surrogates for mapping the selectivity of neural responses. Therefore, before fMRI can be employed as an effective tool for studying critical aspects of laminar processing, validation with single-vessel resolution is needed. The primary visual cortex (V1) in cats, with its precise neuronal functional micro-architecture, offers an ideal model system to examine laminar differences in stimulus selectivity across imaging modalities. Here we used cerebral blood volume weighted (wCBV) fMRI to examine if layer-specific orientation-selective responses could be detected in cat V1. We found orientation preference maps organized tangential to the cortical surface that typically extended across depth in a columnar fashion. We then examined arterial dilation and blood velocity responses to identical visual stimuli by using two- and three- photon optical imaging at single-vessel resolution-which provides a measure of the hemodynamic signals with the highest spatial resolution. Both fMRI and optical imaging revealed a consistent laminar response pattern in which orientation selectivity in cortical layer 4 was significantly lower compared to layer 2/3. This systematic change in selectivity across cortical layers has a clear underpinning in neural circuitry, particularly when comparing layer 4 to other cortical layers.
Collapse
Affiliation(s)
- Shinho Cho
- Center for Magnetic Resonance Research (CMRR), University of Minnesota, MN 55455, United States; Department of Radiology, University of Minnesota, MN 55455, United States
| | - Arani Roy
- Center for Magnetic Resonance Research (CMRR), University of Minnesota, MN 55455, United States; Department of Neuroscience, University of Minnesota, MN 55455, United States
| | - Chao J Liu
- Center for Magnetic Resonance Research (CMRR), University of Minnesota, MN 55455, United States; Department of Neuroscience, University of Minnesota, MN 55455, United States
| | - Djaudat Idiyatullin
- Center for Magnetic Resonance Research (CMRR), University of Minnesota, MN 55455, United States; Department of Radiology, University of Minnesota, MN 55455, United States
| | - Wei Zhu
- Center for Magnetic Resonance Research (CMRR), University of Minnesota, MN 55455, United States; Department of Radiology, University of Minnesota, MN 55455, United States
| | - Yi Zhang
- Center for Magnetic Resonance Research (CMRR), University of Minnesota, MN 55455, United States; Department of Radiology, University of Minnesota, MN 55455, United States
| | - Xiao-Hong Zhu
- Center for Magnetic Resonance Research (CMRR), University of Minnesota, MN 55455, United States; Department of Radiology, University of Minnesota, MN 55455, United States
| | - Phillip O'Herron
- Department of Neuroscience, Medical University of South Carolina, Charleston, SC 29425, United States
| | - Austin Leikvoll
- Center for Magnetic Resonance Research (CMRR), University of Minnesota, MN 55455, United States; Department of Neuroscience, University of Minnesota, MN 55455, United States
| | - Wei Chen
- Center for Magnetic Resonance Research (CMRR), University of Minnesota, MN 55455, United States; Department of Radiology, University of Minnesota, MN 55455, United States
| | - Prakash Kara
- Center for Magnetic Resonance Research (CMRR), University of Minnesota, MN 55455, United States; Department of Neuroscience, University of Minnesota, MN 55455, United States; Department of Neuroscience, Medical University of South Carolina, Charleston, SC 29425, United States.
| | - Kâmil Uğurbil
- Center for Magnetic Resonance Research (CMRR), University of Minnesota, MN 55455, United States; Department of Radiology, University of Minnesota, MN 55455, United States.
| |
Collapse
|
18
|
Zhang Y, Bu T, Zhang J, Tang S, Yu Z, Liu JK, Huang T. Decoding Pixel-Level Image Features from Two-Photon Calcium Signals of Macaque Visual Cortex. Neural Comput 2022; 34:1369-1397. [PMID: 35534008 DOI: 10.1162/neco_a_01498] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Accepted: 12/20/2021] [Indexed: 11/04/2022]
Abstract
Images of visual scenes comprise essential features important for visual cognition of the brain. The complexity of visual features lies at different levels, from simple artificial patterns to natural images with different scenes. It has been a focus of using stimulus images to predict neural responses. However, it remains unclear how to extract features from neuronal responses. Here we address this question by leveraging two-photon calcium neural data recorded from the visual cortex of awake macaque monkeys. With stimuli including various categories of artificial patterns and diverse scenes of natural images, we employed a deep neural network decoder inspired by image segmentation technique. Consistent with the notation of sparse coding for natural images, a few neurons with stronger responses dominated the decoding performance, whereas decoding of ar tificial patterns needs a large number of neurons. When natural images using the model pretrained on artificial patterns are decoded, salient features of natural scenes can be extracted, as well as the conventional category information. Altogether, our results give a new perspective on studying neural encoding principles using reverse-engineering decoding strategies.
Collapse
Affiliation(s)
- Yijun Zhang
- Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240.,Department of Computer Science and Technology, Peking University, Peking 100871, P.R.C.
| | - Tong Bu
- Department of Computer Science and Technology, Peking University, Beijing 100871, P.R.C.
| | - Jiyuan Zhang
- Department of Computer Science and Technology, Peking University, Beijing 100871, P.R.C.
| | - Shiming Tang
- School of Life Sciences and Peking-Tsinghua Center for Life Sciences, Peking University, Beijing 100871, P.R.C.
| | - Zhaofei Yu
- Department of Computer Science and Technology and In stitute for Artificial Intelligence, Peking University, Beijing 100871, P.R.C.
| | - Jian K Liu
- School of Computing, University of Leeds, Leeds LS2 9JT, U.K.
| | - Tiejun Huang
- Department of Computer Science and Technology and Institute for Artificial Intelligence, Peking University, Beijing 100871, P.R.C.,Beijing Academy of Artificial Intelligence, Beijing 100190, P.R.C.
| |
Collapse
|
19
|
Neural oscillations promoting perceptual stability and perceptual memory during bistable perception. Sci Rep 2022; 12:2760. [PMID: 35177702 PMCID: PMC8854562 DOI: 10.1038/s41598-022-06570-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2021] [Accepted: 02/01/2022] [Indexed: 12/15/2022] Open
Abstract
Ambiguous images elicit bistable perception, wherein periods of momentary perceptual stability are interrupted by sudden perceptual switches. When intermittently presented, ambiguous images trigger a perceptual memory trace in the intervening blank periods. Understanding the neural bases of perceptual stability and perceptual memory during bistable perception may hold clues for explaining the apparent stability of visual experience in the natural world, where ambiguous and fleeting images are prevalent. Motivated by recent work showing the involvement of the right inferior frontal gyrus (rIFG) in bistable perception, we conducted a transcranial direct-current stimulation (tDCS) study with a double-blind, within-subject cross-over design to test a potential causal role of rIFG in these processes. Subjects viewed ambiguous images presented continuously or intermittently while under EEG recording. We did not find any significant tDCS effect on perceptual behavior. However, the fluctuations of oscillatory power in the alpha and beta bands predicted perceptual stability, with higher power corresponding to longer percept durations. In addition, higher alpha and beta power predicted enhanced perceptual memory during intermittent viewing. These results reveal a unified neurophysiological mechanism sustaining perceptual stability and perceptual memory when the visual system is faced with ambiguous input.
Collapse
|
20
|
Inhibition, but not excitation, recovers from partial cone loss with greater spatiotemporal integration, synapse density, and frequency. Cell Rep 2022; 38:110317. [PMID: 35108533 PMCID: PMC8865908 DOI: 10.1016/j.celrep.2022.110317] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2021] [Revised: 11/30/2021] [Accepted: 01/07/2022] [Indexed: 12/30/2022] Open
Abstract
Neural circuits function in the face of changing inputs, either caused by normal variation in stimuli or by cell death. To maintain their ability to perform essential computations with partial inputs, neural circuits make modifications. Here, we study the retinal circuit’s responses to changes in light stimuli or in photoreceptor inputs by inducing partial cone death in the mature mouse retina. Can the retina withstand or recover from input loss? We find that the excitatory pathways exhibit functional loss commensurate with cone death and with some aspects predicted by partial light stimulation. However, inhibitory pathways recover functionally from lost input by increasing spatiotemporal integration in a way that is not recapitulated by partially stimulating the control retina. Anatomically, inhibitory synapses are upregulated on secondary bipolar cells and output ganglion cells. These findings demonstrate the greater capacity for inhibition, compared with excitation, to modify spatiotemporal processing with fewer cone inputs. Lee et al. find partial cone loss triggers inhibition, but not excitation, to increase spatiotemporal integration, recover contrast gain, and increase synaptic release onto retinal ganglion cells. Natural images filtered by cone-loss receptive fields perceptually match those of controls. Thus, inhibition compensates for fewer cones to potentially preserve perception.
Collapse
|
21
|
Rideaux R, West RK, Wallis TSA, Bex PJ, Mattingley JB, Harrison WJ. Spatial structure, phase, and the contrast of natural images. J Vis 2022; 22:4. [PMID: 35006237 PMCID: PMC8762697 DOI: 10.1167/jov.22.1.4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2021] [Accepted: 11/25/2021] [Indexed: 11/24/2022] Open
Abstract
The sensitivity of the human visual system is thought to be shaped by environmental statistics. A major endeavor in vision science, therefore, is to uncover the image statistics that predict perceptual and cognitive function. When searching for targets in natural images, for example, it has recently been proposed that target detection is inversely related to the spatial similarity of the target to its local background. We tested this hypothesis by measuring observers' sensitivity to targets that were blended with natural image backgrounds. Targets were designed to have a spatial structure that was either similar or dissimilar to the background. Contrary to masking from similarity, we found that observers were most sensitive to targets that were most similar to their backgrounds. We hypothesized that a coincidence of phase alignment between target and background results in a local contrast signal that facilitates detection when target-background similarity is high. We confirmed this prediction in a second experiment. Indeed, we show that, by solely manipulating the phase of a target relative to its background, the target can be rendered easily visible or undetectable. Our study thus reveals that, in addition to its structural similarity, the phase of the target relative to the background must be considered when predicting detection sensitivity in natural images.
Collapse
Affiliation(s)
- Reuben Rideaux
- Queensland Brain Institute, University of Queensland, St. Lucia, Queensland, Australia
| | - Rebecca K West
- School of Psychology, University of Queensland, St. Lucia, Queensland, Australia
| | - Thomas S A Wallis
- Institut für Psychologie & Centre for Cognitive Science, Technische Universität Darmstadt, Darmstadt, Germany
| | - Peter J Bex
- Department of Psychology, Northeastern University, Boston, MA, USA
| | - Jason B Mattingley
- Queensland Brain Institute, University of Queensland, St. Lucia, Queensland, Australia
- School of Psychology, University of Queensland, St. Lucia, Queensland, Australia
| | - William J Harrison
- Queensland Brain Institute, University of Queensland, St. Lucia, Queensland, Australia
- School of Psychology, University of Queensland, St. Lucia, Queensland, Australia
| |
Collapse
|
22
|
Smith DM, Yang YY, Subramanian DL, Miller AMP, Bulkin DA, Law LM. The limbic memory circuit and the neural basis of contextual memory. Neurobiol Learn Mem 2022; 187:107557. [PMID: 34808337 PMCID: PMC8755583 DOI: 10.1016/j.nlm.2021.107557] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2021] [Revised: 11/12/2021] [Accepted: 11/15/2021] [Indexed: 01/03/2023]
Abstract
The hippocampus, retrosplenial cortex and anterior thalamus are key components of a neural circuit known to be involved in a variety of memory functions, including spatial, contextual and episodic memory. In this review, we focus on the role of this circuit in contextual memory processes. The background environment, or context, is a powerful cue for memory retrieval, and neural representations of the context provide a mechanism for efficiently retrieving relevant memories while avoiding interference from memories that belong to other contexts. Data from experimental lesions and neural manipulation techniques indicate that each of these regions is critical for contextual memory. Neurophysiological evidence from the hippocampus and retrosplenial cortex suggest that contextual information is represented within this circuit by population-level neural firing patterns that reliably differentiate each context a subject encounters. These findings indicate that encoding contextual information to support context-dependent memory retrieval is a key function of this circuit.
Collapse
Affiliation(s)
- David M Smith
- Department of Psychology, Cornell University, Ithaca, NY, United States.
| | - Yan Yu Yang
- Department of Psychology, Cornell University, Ithaca, NY, United States
| | | | - Adam M P Miller
- Department of Psychology, Cornell University, Ithaca, NY, United States
| | - David A Bulkin
- Department of Psychology, Cornell University, Ithaca, NY, United States
| | - L Matthew Law
- Department of Psychology, Cornell University, Ithaca, NY, United States
| |
Collapse
|
23
|
Hardstone R, Zhu M, Flinker A, Melloni L, Devore S, Friedman D, Dugan P, Doyle WK, Devinsky O, He BJ. Long-term priors influence visual perception through recruitment of long-range feedback. Nat Commun 2021; 12:6288. [PMID: 34725348 PMCID: PMC8560909 DOI: 10.1038/s41467-021-26544-w] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2020] [Accepted: 10/08/2021] [Indexed: 11/10/2022] Open
Abstract
Perception results from the interplay of sensory input and prior knowledge. Despite behavioral evidence that long-term priors powerfully shape perception, the neural mechanisms underlying these interactions remain poorly understood. We obtained direct cortical recordings in neurosurgical patients as they viewed ambiguous images that elicit constant perceptual switching. We observe top-down influences from the temporal to occipital cortex, during the preferred percept that is congruent with the long-term prior. By contrast, stronger feedforward drive is observed during the non-preferred percept, consistent with a prediction error signal. A computational model based on hierarchical predictive coding and attractor networks reproduces all key experimental findings. These results suggest a pattern of large-scale information flow change underlying long-term priors' influence on perception and provide constraints on theories about long-term priors' influence on perception.
Collapse
Affiliation(s)
- Richard Hardstone
- Neuroscience Institute, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Michael Zhu
- Neuroscience Institute, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Adeen Flinker
- Department of Neurology, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Lucia Melloni
- Department of Neurology, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Sasha Devore
- Department of Neurology, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Daniel Friedman
- Department of Neurology, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Patricia Dugan
- Department of Neurology, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Werner K Doyle
- Department of Neurosurgery, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Orrin Devinsky
- Department of Neurology, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Biyu J He
- Neuroscience Institute, New York University Grossman School of Medicine, New York, NY, 10016, USA.
- Department of Neurology, New York University Grossman School of Medicine, New York, NY, 10016, USA.
- Department of Neuroscience and Physiology, New York University Grossman School of Medicine, New York, NY, 10016, USA.
- Department of Radiology, New York University Grossman School of Medicine, New York, NY, 10016, USA.
| |
Collapse
|
24
|
Visual exposure enhances stimulus encoding and persistence in primary cortex. Proc Natl Acad Sci U S A 2021; 118:2105276118. [PMID: 34663727 PMCID: PMC8639370 DOI: 10.1073/pnas.2105276118] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/07/2021] [Indexed: 11/28/2022] Open
Abstract
Experience shapes sensory responses, already at the earliest stages of cortical processing. We provide evidence that, in the primary visual cortex of anesthetized cats, brief repetitive exposure to a set of simple, abstract stimuli expands the range and decreases the variability of neuronal responses that persist after stimulus offset. These refinements increase the stimulus-specific clustering of neuronal population responses and result in a more efficient encoding of both stimulus identity and stimulus structure, thus potentially benefiting simple readouts in higher cortical areas. Similar results can be achieved via local plasticity mechanisms in recurrent networks, through self-organized refinements of internal dynamics that do not require changes in firing amplitudes. The brain adapts to the sensory environment. For example, simple sensory exposure can modify the response properties of early sensory neurons. How these changes affect the overall encoding and maintenance of stimulus information across neuronal populations remains unclear. We perform parallel recordings in the primary visual cortex of anesthetized cats and find that brief, repetitive exposure to structured visual stimuli enhances stimulus encoding by decreasing the selectivity and increasing the range of the neuronal responses that persist after stimulus presentation. Low-dimensional projection methods and simple classifiers demonstrate that visual exposure increases the segregation of persistent neuronal population responses into stimulus-specific clusters. These observed refinements preserve the representational details required for stimulus reconstruction and are detectable in postexposure spontaneous activity. Assuming response facilitation and recurrent network interactions as the core mechanisms underlying stimulus persistence, we show that the exposure-driven segregation of stimulus responses can arise through strictly local plasticity mechanisms, also in the absence of firing rate changes. Our findings provide evidence for the existence of an automatic, unguided optimization process that enhances the encoding power of neuronal populations in early visual cortex, thus potentially benefiting simple readouts at higher stages of visual processing.
Collapse
|
25
|
Xia J, Marks TD, Goard MJ, Wessel R. Stable representation of a naturalistic movie emerges from episodic activity with gain variability. Nat Commun 2021; 12:5170. [PMID: 34453045 PMCID: PMC8397750 DOI: 10.1038/s41467-021-25437-2] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2020] [Accepted: 08/11/2021] [Indexed: 01/08/2023] Open
Abstract
Visual cortical responses are known to be highly variable across trials within an experimental session. However, the long-term stability of visual cortical responses is poorly understood. Here using chronic imaging of V1 in mice we show that neural responses to repeated natural movie clips are unstable across weeks. Individual neuronal responses consist of sparse episodic activity which are stable in time but unstable in gain across weeks. Further, we find that the individual episode, instead of neuron, serves as the basic unit of the week-to-week fluctuation. To investigate how population activity encodes the stimulus, we extract a stable one-dimensional representation of the time in the natural movie, using an unsupervised method. Most week-to-week fluctuation is perpendicular to the stimulus encoding direction, thus leaving the stimulus representation largely unaffected. We propose that precise episodic activity with coordinated gain changes are keys to maintain a stable stimulus representation in V1.
Collapse
Affiliation(s)
- Ji Xia
- Department of Physics, Washington University in St. Louis, St. Louis, MO, USA.
| | - Tyler D Marks
- Neuroscience Research Institute, University of California, Santa Barbara, CA, USA
| | - Michael J Goard
- Neuroscience Research Institute, University of California, Santa Barbara, CA, USA
- Department of Molecular, Cellular, and Developmental Biology, University of California, Santa Barbara, CA, USA
- Department of Psychological & Brain Sciences, University of California, Santa Barbara, CA, USA
| | - Ralf Wessel
- Department of Physics, Washington University in St. Louis, St. Louis, MO, USA
| |
Collapse
|
26
|
Pospisil DA, Bair W. The unbiased estimation of the fraction of variance explained by a model. PLoS Comput Biol 2021; 17:e1009212. [PMID: 34347786 PMCID: PMC8367013 DOI: 10.1371/journal.pcbi.1009212] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2020] [Revised: 08/16/2021] [Accepted: 06/24/2021] [Indexed: 11/28/2022] Open
Abstract
The correlation coefficient squared, r2, is commonly used to validate quantitative models on neural data, yet it is biased by trial-to-trial variability: as trial-to-trial variability increases, measured correlation to a model’s predictions decreases. As a result, models that perfectly explain neural tuning can appear to perform poorly. Many solutions to this problem have been proposed, but no consensus has been reached on which is the least biased estimator. Some currently used methods substantially overestimate model fit, and the utility of even the best performing methods is limited by the lack of confidence intervals and asymptotic analysis. We provide a new estimator, r^ER2, that outperforms all prior estimators in our testing, and we provide confidence intervals and asymptotic guarantees. We apply our estimator to a variety of neural data to validate its utility. We find that neural noise is often so great that confidence intervals of the estimator cover the entire possible range of values ([0, 1]), preventing meaningful evaluation of the quality of a model’s predictions. This leads us to propose the use of the signal-to-noise ratio (SNR) as a quality metric for making quantitative comparisons across neural recordings. Analyzing a variety of neural data sets, we find that up to ∼ 40% of some state-of-the-art neural recordings do not pass even a liberal SNR criterion. Moving toward more reliable estimates of correlation, and quantitatively comparing quality across recording modalities and data sets, will be critical to accelerating progress in modeling biological phenomena. Quantifying the similarity between a model and noisy data is fundamental to advancing the scientific understanding of biological phenomena, and it is particularly relevant to modeling neuronal responses. A ubiquitous metric of similarity is the correlation coefficient, but this metric depends on a variety of factors that are irrelevant to the similarity between a model and data. While neuroscientists have recognized this problem and proposed corrected methods, no consensus has been reached as to which are effective. Prior methods have wide variation in their accuracy, and even the most successful methods lack confidence intervals, leaving uncertainty about the reliability of any particular estimate. We address these issues by developing a new estimator along with an associated confidence interval that outperforms all prior methods. We also demonstrate how a signal-to-noise ratio can be used to usefully threshold and compare noisy experimental data across studies and recording paradigms.
Collapse
Affiliation(s)
- Dean A. Pospisil
- Department of Biological Structure, University of Washington, Seattle, Washington, United States of America
- * E-mail:
| | - Wyeth Bair
- Department of Biological Structure, University of Washington, Seattle, Washington, United States of America
- Washington National Primate Research Center, University of Washington, Seattle, Washington, United States of America
- University of Washington Institute for Neuroengineering, Seattle, Washington, United States of America
- Computational Neuroscience Center, University of Washington, Seattle, Washington, United States of America
| |
Collapse
|
27
|
Brick C, Hood B, Ekroll V, de-Wit L. Illusory Essences: A Bias Holding Back Theorizing in Psychological Science. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2021; 17:491-506. [PMID: 34283676 PMCID: PMC8902028 DOI: 10.1177/1745691621991838] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
The reliance in psychology on verbal definitions means that psychological research is unusually moored to how humans think and communicate about categories. Psychological concepts (e.g., intelligence, attention) are easily assumed to represent objective, definable categories with an underlying essence. Like the “vital forces” previously thought to animate life, these assumed essences can create an illusion of understanding. By synthesizing a wide range of research lines from cognitive, clinical, and biological psychology and neuroscience, we describe a pervasive tendency across psychological science to assume that essences explain phenomena. Labeling a complex phenomenon can appear as theoretical progress before there is sufficient evidence that the described category has a definable essence or known boundary conditions. Category labels can further undermine progress by masking contingent and contextual relationships and obscuring the need to specify mechanisms. Finally, we highlight examples of promising methods that circumvent the lure of essences and suggest four concrete strategies for identifying and avoiding essentialist intuitions in theory development.
Collapse
Affiliation(s)
- C Brick
- Department of Psychology, University of Amsterdam.,Department of Psychology, University of Cambridge
| | - B Hood
- School of Psychological Science, University of Bristol
| | - V Ekroll
- Department of Psychosocial Science, University of Bergen
| | - L de-Wit
- Department of Psychology, University of Cambridge
| |
Collapse
|
28
|
Does task-irrelevant music affect gaze allocation during real-world scene viewing? Psychon Bull Rev 2021; 28:1944-1960. [PMID: 34159530 DOI: 10.3758/s13423-021-01947-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/02/2021] [Indexed: 11/08/2022]
Abstract
Gaze control manifests from a dynamic integration of visual and auditory information, with sound providing important cues for how a viewer should behave. Some past research suggests that music, even if entirely irrelevant to the current task demands, may also sway the timing and frequency of fixations. The current work sought to further assess this idea as well as investigate whether task-irrelevant music could also impact how gaze is spatially allocated. In preparation for a later memory test, participants studied pictures of urban scenes in silence or while simultaneously listening to one of two types of music. Eye tracking was recorded, and nine gaze behaviors were measured to characterize the temporal and spatial aspects of gaze control. Findings showed that while these gaze behaviors changed over the course of viewing, music had no impact. Participants in the music conditions, however, did show better memory performance than those who studied in silence. These findings are discussed within theories of multimodal gaze control.
Collapse
|
29
|
Ringach DL. Sparse thalamocortical convergence. Curr Biol 2021; 31:2199-2202.e2. [PMID: 33705713 DOI: 10.1016/j.cub.2021.02.032] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2020] [Revised: 01/28/2021] [Accepted: 02/15/2021] [Indexed: 11/16/2022]
Abstract
How many thalamic neurons converge onto a cortical cell? This is an important question, because the organization of thalamocortical projections can influence the cortical architecture.1,2 Here, we estimate the degree of thalamocortical convergence in primary visual cortex by taking advantage of the cortical expansion-neurons within a restricted volume in primary visual cortex have overlapping receptive fields driven by a smaller set of inputs from the lateral geniculate nucleus.3-5 Under these conditions, the measurements of cortical receptive fields in a population can be used to infer the receptive fields of their geniculate inputs and the weights of their projections using non-negative matrix factorization.6 The analysis reveals sparse connectivity,7 where a handful (~2-6) of thalamic inputs account for 90% of the total synaptic weight to a cortical neuron. Together with previous findings,8 these results paint a picture consistent with the idea that convergence of a few inputs partly determine the retinotopy and tuning properties of cortical cells.8-13.
Collapse
Affiliation(s)
- Dario L Ringach
- Departments of Psychology and Neurobiology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA 90095, USA.
| |
Collapse
|
30
|
Hsu WMM, Kastner DB, Baccus SA, Sharpee TO. How inhibitory neurons increase information transmission under threshold modulation. Cell Rep 2021; 35:109158. [PMID: 34038717 PMCID: PMC8846953 DOI: 10.1016/j.celrep.2021.109158] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2020] [Revised: 02/14/2021] [Accepted: 04/29/2021] [Indexed: 11/28/2022] Open
Abstract
Modulation of neuronal thresholds is ubiquitous in the brain. Phenomena such as figure-ground segmentation, motion detection, stimulus anticipation, and shifts in attention all involve changes in a neuron’s threshold based on signals from larger scales than its primary inputs. However, this modulation reduces the accuracy with which neurons can represent their primary inputs, creating a mystery as to why threshold modulation is so widespread in the brain. We find that modulation is less detrimental than other forms of neuronal variability and that its negative effects can be nearly completely eliminated if modulation is applied selectively to sparsely responding neurons in a circuit by inhibitory neurons. We verify these predictions in the retina where we find that inhibitory amacrine cells selectively deliver modulation signals to sparsely responding ganglion cell types. Our findings elucidate the central role that inhibitory neurons play in maximizing information transmission under modulation. Modulation of neuronal thresholds is ubiquitous in the brain but reduces the accuracy of neural signaling. Hsu et al. show that the negative impact of threshold modulation can be almost completely eliminated when modulation is not delivered uniformly to all neurons but only to a subset and via inhibitory neurons.
Collapse
Affiliation(s)
- Wei-Mien M Hsu
- Computational Neurobiology Laboratory, Salk Institute for Biological Studies, La Jolla, CA, USA; Department of Physics, University of California, San Diego, La Jolla, CA, USA
| | - David B Kastner
- Department of Psychiatry and Behavioral Sciences, University of California, San Francisco, School of Medicine, San Francisco, CA, USA; Department of Neurobiology, Stanford University School of Medicine, Stanford, CA, USA
| | - Stephen A Baccus
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA, USA
| | - Tatyana O Sharpee
- Computational Neurobiology Laboratory, Salk Institute for Biological Studies, La Jolla, CA, USA; Department of Physics, University of California, San Diego, La Jolla, CA, USA.
| |
Collapse
|
31
|
Anomalous Behavior Detection Framework Using HTM-Based Semantic Folding Technique. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2021; 2021:5585238. [PMID: 33790986 PMCID: PMC7987406 DOI: 10.1155/2021/5585238] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/23/2021] [Revised: 02/17/2021] [Accepted: 02/26/2021] [Indexed: 11/17/2022]
Abstract
Upon the working principles of the human neocortex, the Hierarchical Temporal Memory model has been developed which is a proposed theoretical framework for sequence learning. Both categorical and numerical types of data are handled by HTM. Semantic Folding Theory (SFT) is based on HTM to represent a data stream for processing in the form of sparse distributed representation (SDR). For natural language perception and production, SFT delivers a solid structural background for semantic evidence description to the fundamentals of the semantic foundation during the phase of language learning. Anomalies are the patterns from data streams that do not follow the expected behavior. Any stream of data patterns could have a number of anomaly types. In a data stream, a single pattern or combination of closely related patterns that diverges and deviates from standard, normal, or expected is called a static (spatial) anomaly. A temporal anomaly is a set of unexpected changes between patterns. When a change first appears, this is recorded as an anomaly. If this change looks a number of times, then it is set to a “new normal” and terminated as an anomaly. An HTM system detects the anomaly, and due to continuous learning nature, it quickly learns when they become the new normal. A robust anomalous behavior detection framework using HTM-based SFT for improving decision-making (SDR-ABDF/P2) is a proposed framework or model in this research. The researcher claims that the proposed model would be able to learn the order of several variables continuously in temporal sequences by using an unsupervised learning rule.
Collapse
|
32
|
Abstract
It has often been conjectured that the effectiveness of line drawings can be explained by the similarity of edge images to line drawings. This article presents several problems with explaining line drawing perception in terms of edges, and how the recently proposed Realism Hypothesis resolves these problems. There is nonetheless existing evidence that edges are often the best features for predicting where people draw lines; this article describes how the Realism Hypothesis can explain this evidence.
Collapse
|
33
|
Abstract
In this paper, we define a new method for analyzing object-scene contextual relationships using computational linguistics: Linguistic Analysis of Scene Semantics, or LASS. LASS uses linguistic semantic similarity relationships between scene object and context labels embedded in a vector-space language model: Facebook Research's fastText. Importantly, the use of fastText permits semantic similarity score calculation between any set of strings and thus elements of any set of image data for which labels are available. Scene semantic similarity scores are then embedded in object segmentation mask locations in the image, creating a semantic similarity map. LASS can also be fully automated by generating context and object labels, as well as object segmentation masks, using deep learning. We compare semantic similarity maps between human- and neural network-generated annotations on a corpus of images taken from the LabelMe database. Semantic similarity maps produced by the fully automated LASS have a number of desirable properties, while maintaining a high degree of spatial and semantic similarity to them. Finally, we use LASS to evaluate the distribution of semantically consistent scene elements in space. Both show relatively uniform distributions of semantic relatedness to scene context, suggesting that contextually appropriate objects are likely to be found in all image regions. Taken together, these results suggest that LASS is accurate, automatic, flexible, and useful in a number of research contexts such as scene grammar and novelty detection.
Collapse
|
34
|
Siegle JH, Jia X, Durand S, Gale S, Bennett C, Graddis N, Heller G, Ramirez TK, Choi H, Luviano JA, Groblewski PA, Ahmed R, Arkhipov A, Bernard A, Billeh YN, Brown D, Buice MA, Cain N, Caldejon S, Casal L, Cho A, Chvilicek M, Cox TC, Dai K, Denman DJ, de Vries SEJ, Dietzman R, Esposito L, Farrell C, Feng D, Galbraith J, Garrett M, Gelfand EC, Hancock N, Harris JA, Howard R, Hu B, Hytnen R, Iyer R, Jessett E, Johnson K, Kato I, Kiggins J, Lambert S, Lecoq J, Ledochowitsch P, Lee JH, Leon A, Li Y, Liang E, Long F, Mace K, Melchior J, Millman D, Mollenkopf T, Nayan C, Ng L, Ngo K, Nguyen T, Nicovich PR, North K, Ocker GK, Ollerenshaw D, Oliver M, Pachitariu M, Perkins J, Reding M, Reid D, Robertson M, Ronellenfitch K, Seid S, Slaughterbeck C, Stoecklin M, Sullivan D, Sutton B, Swapp J, Thompson C, Turner K, Wakeman W, Whitesell JD, Williams D, Williford A, Young R, Zeng H, Naylor S, Phillips JW, Reid RC, Mihalas S, Olsen SR, Koch C. Survey of spiking in the mouse visual system reveals functional hierarchy. Nature 2021; 592:86-92. [PMID: 33473216 PMCID: PMC10399640 DOI: 10.1038/s41586-020-03171-x] [Citation(s) in RCA: 181] [Impact Index Per Article: 60.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2019] [Accepted: 12/09/2020] [Indexed: 12/14/2022]
Abstract
The anatomy of the mammalian visual system, from the retina to the neocortex, is organized hierarchically1. However, direct observation of cellular-level functional interactions across this hierarchy is lacking due to the challenge of simultaneously recording activity across numerous regions. Here we describe a large, open dataset-part of the Allen Brain Observatory2-that surveys spiking from tens of thousands of units in six cortical and two thalamic regions in the brains of mice responding to a battery of visual stimuli. Using cross-correlation analysis, we reveal that the organization of inter-area functional connectivity during visual stimulation mirrors the anatomical hierarchy from the Allen Mouse Brain Connectivity Atlas3. We find that four classical hierarchical measures-response latency, receptive-field size, phase-locking to drifting gratings and response decay timescale-are all correlated with the hierarchy. Moreover, recordings obtained during a visual task reveal that the correlation between neural activity and behavioural choice also increases along the hierarchy. Our study provides a foundation for understanding coding and signal propagation across hierarchically organized cortical and thalamic visual areas.
Collapse
Affiliation(s)
| | - Xiaoxuan Jia
- Allen Institute for Brain Science, Seattle, WA, USA.
| | | | - Sam Gale
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | - Nile Graddis
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | | | - Hannah Choi
- Allen Institute for Brain Science, Seattle, WA, USA.,Department of Applied Mathematics, University of Washington, Seattle, WA, USA
| | | | | | | | | | - Amy Bernard
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | - Dillan Brown
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | - Nicolas Cain
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | - Linzy Casal
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Andrew Cho
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | - Timothy C Cox
- University of Missouri-Kansas City School of Dentistry, Kansas City, MO, USA
| | - Kael Dai
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Daniel J Denman
- Allen Institute for Brain Science, Seattle, WA, USA.,The University of Colorado Denver, Anschutz Medical Campus, Aurora, CO, USA
| | | | | | | | | | - David Feng
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | | | | | | | | | | | - Brian Hu
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Ross Hytnen
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | | | | | - India Kato
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | | | - Jerome Lecoq
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | | | - Arielle Leon
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Yang Li
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | - Fuhui Long
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Kyla Mace
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | | | | | | | - Lydia Ng
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Kiet Ngo
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | | | - Kat North
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | | | | | | | - Jed Perkins
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | - David Reid
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | | | - Sam Seid
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | | | | | - Ben Sutton
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Jackie Swapp
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | | | | | | | | | | | - Rob Young
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Hongkui Zeng
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Sarah Naylor
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | - R Clay Reid
- Allen Institute for Brain Science, Seattle, WA, USA
| | | | - Shawn R Olsen
- Allen Institute for Brain Science, Seattle, WA, USA.
| | | |
Collapse
|
35
|
Ahmadi A, Kashefi M, Shahrokhi H, Nazari MA. Computer aided diagnosis system using deep convolutional neural networks for ADHD subtypes. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2020.102227] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
|
36
|
A Gradient of Sharpening Effects by Perceptual Prior across the Human Cortical Hierarchy. J Neurosci 2020; 41:167-178. [PMID: 33208472 DOI: 10.1523/jneurosci.2023-20.2020] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2020] [Revised: 11/06/2020] [Accepted: 11/06/2020] [Indexed: 11/21/2022] Open
Abstract
Prior knowledge profoundly influences perceptual processing. Previous studies have revealed consistent suppression of predicted stimulus information in sensory areas, but how prior knowledge modulates processing higher up in the cortical hierarchy remains poorly understood. In addition, the mechanism leading to suppression of predicted sensory information remains unclear, and studies thus far have revealed a mixed pattern of results in support of either the "sharpening" or "dampening" model. Here, using 7T fMRI in humans (both sexes), we observed that prior knowledge acquired from fast, one-shot perceptual learning sharpens neural representation throughout the ventral visual stream, generating suppressed sensory responses. In contrast, the frontoparietal and default mode networks exhibit similar sharpening of content-specific neural representation, but in the context of unchanged and enhanced activity magnitudes, respectively: a pattern we refer to as "selective enhancement." Together, these results reveal a heretofore unknown macroscopic gradient of prior knowledge's sharpening effect on neural representations across the cortical hierarchy.SIGNIFICANCE STATEMENT A fundamental question in neuroscience is how prior knowledge shapes perceptual processing. Perception is constantly informed by internal priors in the brain acquired from past experiences, but the neural mechanisms underlying this process are poorly understood. To date, research on this question has focused on early visual regions, reporting a consistent downregulation when predicted stimuli are encountered. Here, using a dramatic one-shot perceptual learning paradigm, we observed that prior knowledge results in sharper neural representations across the cortical hierarchy of the human brain through a gradient of mechanisms. In visual regions, neural responses tuned away from internal predictions are suppressed. In frontoparietal regions, neural activity consistent with priors is selectively enhanced. These results deepen our understanding of how prior knowledge informs perception.
Collapse
|
37
|
Nastase SA, Goldstein A, Hasson U. Keep it real: rethinking the primacy of experimental control in cognitive neuroscience. Neuroimage 2020; 222:117254. [PMID: 32800992 PMCID: PMC7789034 DOI: 10.1016/j.neuroimage.2020.117254] [Citation(s) in RCA: 117] [Impact Index Per Article: 29.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2019] [Revised: 07/08/2020] [Accepted: 08/04/2020] [Indexed: 01/17/2023] Open
Abstract
Naturalistic experimental paradigms in neuroimaging arose from a pressure to test the validity of models we derive from highly-controlled experiments in real-world contexts. In many cases, however, such efforts led to the realization that models developed under particular experimental manipulations failed to capture much variance outside the context of that manipulation. The critique of non-naturalistic experiments is not a recent development; it echoes a persistent and subversive thread in the history of modern psychology. The brain has evolved to guide behavior in a multidimensional world with many interacting variables. The assumption that artificially decoupling and manipulating these variables will lead to a satisfactory understanding of the brain may be untenable. We develop an argument for the primacy of naturalistic paradigms, and point to recent developments in machine learning as an example of the transformative power of relinquishing control. Naturalistic paradigms should not be deployed as an afterthought if we hope to build models of brain and behavior that extend beyond the laboratory into the real world.
Collapse
Affiliation(s)
- Samuel A Nastase
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
| | - Ariel Goldstein
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Uri Hasson
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA; Department of Psychology, Princeton University, Princeton, NJ, USA
| |
Collapse
|
38
|
Chauhan T, Héjja-Brichard Y, Cottereau BR. Modelling binocular disparity processing from statistics in natural scenes. Vision Res 2020; 176:27-39. [DOI: 10.1016/j.visres.2020.07.009] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2019] [Revised: 07/19/2020] [Accepted: 07/20/2020] [Indexed: 11/25/2022]
|
39
|
Bertalmío M, Gomez-Villa A, Martín A, Vazquez-Corral J, Kane D, Malo J. Evidence for the intrinsically nonlinear nature of receptive fields in vision. Sci Rep 2020; 10:16277. [PMID: 33004868 PMCID: PMC7530701 DOI: 10.1038/s41598-020-73113-0] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2019] [Accepted: 09/11/2020] [Indexed: 11/10/2022] Open
Abstract
The responses of visual neurons, as well as visual perception phenomena in general, are highly nonlinear functions of the visual input, while most vision models are grounded on the notion of a linear receptive field (RF). The linear RF has a number of inherent problems: it changes with the input, it presupposes a set of basis functions for the visual system, and it conflicts with recent studies on dendritic computations. Here we propose to model the RF in a nonlinear manner, introducing the intrinsically nonlinear receptive field (INRF). Apart from being more physiologically plausible and embodying the efficient representation principle, the INRF has a key property of wide-ranging implications: for several vision science phenomena where a linear RF must vary with the input in order to predict responses, the INRF can remain constant under different stimuli. We also prove that Artificial Neural Networks with INRF modules instead of linear filters have a remarkably improved performance and better emulate basic human perception. Our results suggest a change of paradigm for vision science as well as for artificial intelligence.
Collapse
Affiliation(s)
| | | | | | | | - David Kane
- Universitat Pompeu Fabra, Barcelona, Spain
| | - Jesús Malo
- Universitat de Valencia, Valencia, Spain
| |
Collapse
|
40
|
Gomez-Villa A, Martín A, Vazquez-Corral J, Bertalmío M, Malo J. Color illusions also deceive CNNs for low-level vision tasks: Analysis and implications. Vision Res 2020; 176:156-174. [PMID: 32896717 DOI: 10.1016/j.visres.2020.07.010] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2019] [Revised: 07/10/2020] [Accepted: 07/22/2020] [Indexed: 11/18/2022]
Abstract
The study of visual illusions has proven to be a very useful approach in vision science. In this work we start by showing that, while convolutional neural networks (CNNs) trained for low-level visual tasks in natural images may be deceived by brightness and color illusions, some network illusions can be inconsistent with the perception of humans. Next, we analyze where these similarities and differences may come from. On one hand, the proposed linear eigenanalysis explains the overall similarities: in simple CNNs trained for tasks like denoising or deblurring, the linear version of the network has center-surround receptive fields, and global transfer functions are very similar to the human achromatic and chromatic contrast sensitivity functions in human-like opponent color spaces. These similarities are consistent with the long-standing hypothesis that considers low-level visual illusions as a by-product of the optimization to natural environments. Specifically, here human-like features emerge from error minimization. On the other hand, the observed differences must be due to the behavior of the human visual system not explained by the linear approximation. However, our study also shows that more 'flexible' network architectures, with more layers and a higher degree of nonlinearity, may actually have a worse capability of reproducing visual illusions. This implies, in line with other works in the vision science literature, a word of caution on using CNNs to study human vision: on top of the intrinsic limitations of the L + NL formulation of artificial networks to model vision, the nonlinear behavior of flexible architectures may easily be markedly different from that of the visual system.
Collapse
Affiliation(s)
- A Gomez-Villa
- Dept. Inf. Comm. Tech., Universitat Pompeu Fabra, Barcelona, Spain.
| | - A Martín
- Dept. Inf. Comm. Tech., Universitat Pompeu Fabra, Barcelona, Spain.
| | - J Vazquez-Corral
- Dept. Inf. Comm. Tech., Universitat Pompeu Fabra, Barcelona, Spain.
| | - M Bertalmío
- Dept. Inf. Comm. Tech., Universitat Pompeu Fabra, Barcelona, Spain.
| | - J Malo
- Image Proc., Lab, Universitat de València, València, Spain.
| |
Collapse
|
41
|
Natsubori A, Tsunematsu T, Karashima A, Imamura H, Kabe N, Trevisiol A, Hirrlinger J, Kodama T, Sanagi T, Masamoto K, Takata N, Nave KA, Matsui K, Tanaka KF, Honda M. Intracellular ATP levels in mouse cortical excitatory neurons varies with sleep-wake states. Commun Biol 2020; 3:491. [PMID: 32895482 PMCID: PMC7477120 DOI: 10.1038/s42003-020-01215-6] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2019] [Accepted: 07/31/2020] [Indexed: 02/06/2023] Open
Abstract
Whilst the brain is assumed to exert homeostatic functions to keep the cellular energy status constant under physiological conditions, this has not been experimentally proven. Here, we conducted in vivo optical recordings of intracellular concentration of adenosine 5'-triphosphate (ATP), the major cellular energy metabolite, using a genetically encoded sensor in the mouse brain. We demonstrate that intracellular ATP levels in cortical excitatory neurons fluctuate in a cortex-wide manner depending on the sleep-wake states, correlating with arousal. Interestingly, ATP levels profoundly decreased during rapid eye movement sleep, suggesting a negative energy balance in neurons despite a simultaneous increase in cerebral hemodynamics for energy supply. The reduction in intracellular ATP was also observed in response to local electrical stimulation for neuronal activation, whereas the hemodynamics were simultaneously enhanced. These observations indicate that cerebral energy metabolism may not always meet neuronal energy demands, consequently resulting in physiological fluctuations of intracellular ATP levels in neurons.
Collapse
Affiliation(s)
- Akiyo Natsubori
- Sleep Disorders Project, Tokyo Metropolitan Institute of Medical Science, 2-1-6 Kamikitazawa, Setagaya-Ku, Tokyo, 156-8506, Japan.
| | - Tomomi Tsunematsu
- Super-network Brain Physiology, Graduate School of Life Sciences, Tohoku University, Sendai, 980-8578, Japan
- Precursory Research for Embryonic Science and Technology, Japan Science and Technology Agency, Kawaguchi, 332-0012, Japan
- Advanced Interdisciplinary Research Division, Frontier Research Institute for Interdisciplinary Sciences, Tohoku University, Sendai, 980-8578, Japan
| | - Akihiro Karashima
- Tohoku Institute of Technology, 35-1, Yagiyama Kasumi-cho, Taihaku-ku, Sendai, 982-8577, Japan
| | - Hiromi Imamura
- Graduate School of Biostudies, Kyoto University, Yoshida-konoe-cho, Sakyo-ku, Kyoto, 606-8501, Japan
| | - Naoya Kabe
- Neural Prosthesis Project, Tokyo Metropolitan Institute of Medical Science, 2-1-6 Kamikitazawa, Setagaya-ku, Tokyo, 156-8506, Japan
| | - Andrea Trevisiol
- Department of Neurogenetics, Max-Planck-Institute for Experimental Medicine, Gottingen, 37075, Germany
| | - Johannes Hirrlinger
- Department of Neurogenetics, Max-Planck-Institute for Experimental Medicine, Gottingen, 37075, Germany
- Carl-Ludwig-Institute for Physiology, University of Leipzig, Liebigstrasse 27, 04103, Leipzig, Germany
| | - Tohru Kodama
- Sleep Disorders Project, Tokyo Metropolitan Institute of Medical Science, 2-1-6 Kamikitazawa, Setagaya-Ku, Tokyo, 156-8506, Japan
| | - Tomomi Sanagi
- Advanced Interdisciplinary Research Division, Frontier Research Institute for Interdisciplinary Sciences, Tohoku University, Sendai, 980-8578, Japan
| | - Kazuto Masamoto
- Department of Mechanical and Intelligent Systems Engineering, University of Electro-Communications, 1-5-1 Chofugaoka, Chofu, Tokyo, 182-8585, Japan
| | - Norio Takata
- Department of Neuropsychiatry, Keio University School of Medicine, 35 Shinanomachi, Shinjuku-ku, Tokyo, 160-8582, Japan
| | - Klaus-Armin Nave
- Department of Neurogenetics, Max-Planck-Institute for Experimental Medicine, Gottingen, 37075, Germany
| | - Ko Matsui
- Super-network Brain Physiology, Graduate School of Life Sciences, Tohoku University, Sendai, 980-8578, Japan
| | - Kenji F Tanaka
- Department of Neuropsychiatry, Keio University School of Medicine, 35 Shinanomachi, Shinjuku-ku, Tokyo, 160-8582, Japan
| | - Makoto Honda
- Sleep Disorders Project, Tokyo Metropolitan Institute of Medical Science, 2-1-6 Kamikitazawa, Setagaya-Ku, Tokyo, 156-8506, Japan
| |
Collapse
|
42
|
Krasich K, Huffman G, Faber M, Brockmole JR. Where the eyes wander: The relationship between mind wandering and fixation allocation to visually salient and semantically informative static scene content. J Vis 2020; 20:10. [PMID: 32926071 PMCID: PMC7490225 DOI: 10.1167/jov.20.9.10] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Vision is crucial for many everyday activities, but the mind is not always focused on what the eyes see. Mind wandering occurs frequently and is associated with attenuated visual and cognitive processing of external information. Corresponding changes in gaze behavior—namely, fewer, longer, and more dispersed fixations—suggest a shift in how the visual system samples external information. Using three computational models of visual salience and two innovative approaches for measuring semantic informativeness, the current work assessed whether these changes reflect how the visual system prioritizes visually salient and semantically informative scene content, two major determinants in most theoretical frameworks and computational models of gaze control. Findings showed that, in a static scene viewing task, fixations were allocated to scene content that was more visually salient 10 seconds prior to probe-caught, self-reported mind wandering compared to self-reported attentive viewing. The relationship between mind wandering and semantic content was more equivocal, with weaker evidence that fixations are more likely to fall on locally informative scene regions. This indicates that the visual system is still able to discriminate visually salient and semantically informative scene content during mind wandering and may fixate on such information more frequently than during attentive viewing. Theoretical implications are discussed in light of these findings.
Collapse
Affiliation(s)
- Kristina Krasich
- Department of Psychology, University of Notre Dame, Notre Dame, IN, USA
| | - Greg Huffman
- Department of Psychology, University of Notre Dame, Notre Dame, IN, USA.,Present address: Leidos, Reston, VA, USA
| | - Myrthe Faber
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Medical Center, Nijmegen, The Netherlands
| | - James R Brockmole
- Department of Psychology, University of Notre Dame, Notre Dame, IN, USA
| |
Collapse
|
43
|
Sanzeni A, Histed MH, Brunel N. Response nonlinearities in networks of spiking neurons. PLoS Comput Biol 2020; 16:e1008165. [PMID: 32941457 PMCID: PMC7524009 DOI: 10.1371/journal.pcbi.1008165] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2019] [Revised: 09/29/2020] [Accepted: 07/19/2020] [Indexed: 01/18/2023] Open
Abstract
Combining information from multiple sources is a fundamental operation performed by networks of neurons in the brain, whose general principles are still largely unknown. Experimental evidence suggests that combination of inputs in cortex relies on nonlinear summation. Such nonlinearities are thought to be fundamental to perform complex computations. However, these non-linearities are inconsistent with the balanced-state model, one of the most popular models of cortical dynamics, which predicts networks have a linear response. This linearity is obtained in the limit of very large recurrent coupling strength. We investigate the stationary response of networks of spiking neurons as a function of coupling strength. We show that, while a linear transfer function emerges at strong coupling, nonlinearities are prominent at finite coupling, both at response onset and close to saturation. We derive a general framework to classify nonlinear responses in these networks and discuss which of them can be captured by rate models. This framework could help to understand the diversity of non-linearities observed in cortical networks.
Collapse
Affiliation(s)
- Alessandro Sanzeni
- National institute of Mental Health Intramural Program, NIH, Bethesda, MD, USA
- Department of Neurobiology, Duke University, Durham, NC, USA
| | - Mark H. Histed
- National institute of Mental Health Intramural Program, NIH, Bethesda, MD, USA
| | - Nicolas Brunel
- Department of Neurobiology, Duke University, Durham, NC, USA
- Department of Physics, Duke University, Durham, NC, USA
| |
Collapse
|
44
|
Yu Z, Zhang C, Wang L, Tong L, Yan B. A Comparative Analysis of Visual Encoding Models Based on Classification and Segmentation Task-Driven CNNs. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2020; 2020:5408942. [PMID: 32802150 PMCID: PMC7416280 DOI: 10.1155/2020/5408942] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/14/2020] [Revised: 05/31/2020] [Accepted: 06/06/2020] [Indexed: 11/17/2022]
Abstract
Nowadays, visual encoding models use convolution neural networks (CNNs) with outstanding performance in computer vision to simulate the process of human information processing. However, the prediction performances of encoding models will have differences based on different networks driven by different tasks. Here, the impact of network tasks on encoding models is studied. Using functional magnetic resonance imaging (fMRI) data, the features of natural visual stimulation are extracted using a segmentation network (FCN32s) and a classification network (VGG16) with different visual tasks but similar network structure. Then, using three sets of features, i.e., segmentation, classification, and fused features, the regularized orthogonal matching pursuit (ROMP) method is used to establish the linear mapping from features to voxel responses. The analysis results indicate that encoding models based on networks performing different tasks can effectively but differently predict stimulus-induced responses measured by fMRI. The prediction accuracy of the encoding model based on VGG is found to be significantly better than that of the model based on FCN in most voxels but similar to that of fused features. The comparative analysis demonstrates that the CNN performing the classification task is more similar to human visual processing than that performing the segmentation task.
Collapse
Affiliation(s)
- Ziya Yu
- PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China
| | - Chi Zhang
- PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China
| | - Linyuan Wang
- PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China
| | - Li Tong
- PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China
| | - Bin Yan
- PLA Strategy Support Force Information Engineering University, Zhengzhou 450001, China
| |
Collapse
|
45
|
Parker PRL, Brown MA, Smear MC, Niell CM. Movement-Related Signals in Sensory Areas: Roles in Natural Behavior. Trends Neurosci 2020; 43:581-595. [PMID: 32580899 PMCID: PMC8000520 DOI: 10.1016/j.tins.2020.05.005] [Citation(s) in RCA: 66] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2020] [Revised: 05/02/2020] [Accepted: 05/24/2020] [Indexed: 11/24/2022]
Abstract
Recent studies have demonstrated prominent and widespread movement-related signals in the brain of head-fixed mice, even in primary sensory areas. However, it is still unknown what role these signals play in sensory processing. Why are these sensory areas 'contaminated' by movement signals? During natural behavior, animals actively acquire sensory information as they move through the environment and use this information to guide ongoing actions. In this context, movement-related signals could allow sensory systems to predict self-induced sensory changes and extract additional information about the environment. In this review we summarize recent findings on the presence of movement-related signals in sensory areas and discuss how their study, in the context of natural freely moving behaviors, could advance models of sensory processing.
Collapse
Affiliation(s)
- Philip R L Parker
- Institute of Neuroscience, University of Oregon, Eugene, OR 97403, USA.
| | - Morgan A Brown
- Institute of Neuroscience, University of Oregon, Eugene, OR 97403, USA
| | - Matthew C Smear
- Institute of Neuroscience, University of Oregon, Eugene, OR 97403, USA; Department of Psychology, University of Oregon, Eugene, OR 97403, USA
| | - Cristopher M Niell
- Institute of Neuroscience, University of Oregon, Eugene, OR 97403, USA; Department of Biology, University of Oregon, Eugene, OR 97403, USA.
| |
Collapse
|
46
|
Sanzeni A, Akitake B, Goldbach HC, Leedy CE, Brunel N, Histed MH. Inhibition stabilization is a widespread property of cortical networks. eLife 2020; 9:e54875. [PMID: 32598278 PMCID: PMC7324160 DOI: 10.7554/elife.54875] [Citation(s) in RCA: 69] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2020] [Accepted: 05/21/2020] [Indexed: 12/19/2022] Open
Abstract
Many cortical network models use recurrent coupling strong enough to require inhibition for stabilization. Yet it has been experimentally unclear whether inhibition-stabilized network (ISN) models describe cortical function well across areas and states. Here, we test several ISN predictions, including the counterintuitive (paradoxical) suppression of inhibitory firing in response to optogenetic inhibitory stimulation. We find clear evidence for ISN operation in mouse visual, somatosensory, and motor cortex. Simple two-population ISN models describe the data well and let us quantify coupling strength. Although some models predict a non-ISN to ISN transition with increasingly strong sensory stimuli, we find ISN effects without sensory stimulation and even during light anesthesia. Additionally, average paradoxical effects result only with transgenic, not viral, opsin expression in parvalbumin (PV)-positive neurons; theory and expression data show this is consistent with ISN operation. Taken together, these results show strong coupling and inhibition stabilization are common features of the cortex.
Collapse
Affiliation(s)
- Alessandro Sanzeni
- NIMH Intramural Program, National Institutes of HealthBethesdaUnited States
- Department of Neurobiology, Duke UniversityDurhamUnited States
| | - Bradley Akitake
- NIMH Intramural Program, National Institutes of HealthBethesdaUnited States
| | - Hannah C Goldbach
- NIMH Intramural Program, National Institutes of HealthBethesdaUnited States
| | - Caitlin E Leedy
- NIMH Intramural Program, National Institutes of HealthBethesdaUnited States
| | - Nicolas Brunel
- Department of Neurobiology, Duke UniversityDurhamUnited States
| | - Mark H Histed
- NIMH Intramural Program, National Institutes of HealthBethesdaUnited States
| |
Collapse
|
47
|
Sorooshyari SK, Sheng H, Poor HV. Object Recognition at Higher Regions of the Ventral Visual Stream via Dynamic Inference. Front Comput Neurosci 2020; 14:46. [PMID: 32655388 PMCID: PMC7325008 DOI: 10.3389/fncom.2020.00046] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2020] [Accepted: 04/30/2020] [Indexed: 11/13/2022] Open
Affiliation(s)
- Siamak K. Sorooshyari
- Department of Integrative Biology, University of California, Berkeley, Berkeley, CA, United States
- *Correspondence: Siamak K. Sorooshyari
| | - Huanjie Sheng
- Department of Integrative Biology, University of California, Berkeley, Berkeley, CA, United States
| | - H. Vincent Poor
- Department of Electrical Engineering, Princeton University, Princeton, NJ, United States
| |
Collapse
|
48
|
Anada S, Nomura Y, Hirayama T, Yamamoto K. Simulation-Trained Sparse Coding for High-Precision Phase Imaging in Low-Dose Electron Holography. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2020; 26:429-438. [PMID: 32513331 DOI: 10.1017/s1431927620001452] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
We broaden the applicability of sparse coding, a machine learning method, to low-dose electron holography by using simulated holograms for learning and validation processes. The holograms, with shot noise, are prepared to generate a model, or a dictionary, that includes basic features representing interference fringes. The dictionary is applied to sparse representations of other simulated holograms with various signal-to-noise ratios (SNRs). Results demonstrate that this approach successfully removes noise for holograms with an extremely small SNR of 0.10, and that the denoised holograms provide the accurate phase distribution. Furthermore, this study demonstrates that the dictionary learned from the simulated holograms can be applied to denoising of experimental holograms of a p-n junction specimen recorded with different exposure times. The results indicate that the simulation-trained sparse coding is suitable for use over a wide range of imaging conditions, in particular for observing electron beam-sensitive materials.
Collapse
Affiliation(s)
- Satoshi Anada
- Nanostructures Research Laboratory, Japan Fine Ceramics Center, 2-4-1 Mutsuno, Atsuta-ku, Nagoya, Aichi456-8587, Japan
| | - Yuki Nomura
- Technology Innovation Division, Panasonic Corporation, 3-1-1 Yagumo-Nakamachi, Moriguchi, Osaka570-8501, Japan
| | - Tsukasa Hirayama
- Nanostructures Research Laboratory, Japan Fine Ceramics Center, 2-4-1 Mutsuno, Atsuta-ku, Nagoya, Aichi456-8587, Japan
| | - Kazuo Yamamoto
- Nanostructures Research Laboratory, Japan Fine Ceramics Center, 2-4-1 Mutsuno, Atsuta-ku, Nagoya, Aichi456-8587, Japan
| |
Collapse
|
49
|
Almasi A, Meffin H, Cloherty SL, Wong Y, Yunzab M, Ibbotson MR. Mechanisms of Feature Selectivity and Invariance in Primary Visual Cortex. Cereb Cortex 2020; 30:5067-5087. [PMID: 32368778 DOI: 10.1093/cercor/bhaa102] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2019] [Revised: 03/26/2020] [Accepted: 03/26/2020] [Indexed: 11/14/2022] Open
Abstract
Visual object identification requires both selectivity for specific visual features that are important to the object's identity and invariance to feature manipulations. For example, a hand can be shifted in position, rotated, or contracted but still be recognized as a hand. How are the competing requirements of selectivity and invariance built into the early stages of visual processing? Typically, cells in the primary visual cortex are classified as either simple or complex. They both show selectivity for edge-orientation but complex cells develop invariance to edge position within the receptive field (spatial phase). Using a data-driven model that extracts the spatial structures and nonlinearities associated with neuronal computation, we quantitatively describe the balance between selectivity and invariance in complex cells. Phase invariance is frequently partial, while invariance to orientation and spatial frequency are more extensive than expected. The invariance arises due to two independent factors: (1) the structure and number of filters and (2) the form of nonlinearities that act upon the filter outputs. Both vary more than previously considered, so primary visual cortex forms an elaborate set of generic feature sensitivities, providing the foundation for more sophisticated object processing.
Collapse
Affiliation(s)
- Ali Almasi
- National Vision Research Institute, Australian College of Optometry, Carlton VIC 3053, Australia
| | - Hamish Meffin
- National Vision Research Institute, Australian College of Optometry, Carlton VIC 3053, Australia.,Department of Biomedical Engineering, The University of Melbourne, Parkville VIC 3010, Australia
| | - Shaun L Cloherty
- School of Engineering, RMIT University, Melbourne VIC 3001, Australia
| | - Yan Wong
- Department of Electrical and Computer Systems Engineering and Department of Physiology, Monash University, Clayton VIC 3800, Australia
| | - Molis Yunzab
- National Vision Research Institute, Australian College of Optometry, Carlton VIC 3053, Australia
| | - Michael R Ibbotson
- National Vision Research Institute, Australian College of Optometry, Carlton VIC 3053, Australia.,Department of Optometry and Vision Sciences, The University of Melbourne, Parkville VIC 3010, Australia
| |
Collapse
|
50
|
Levy M, Sporns O, MacLean JN. Network Analysis of Murine Cortical Dynamics Implicates Untuned Neurons in Visual Stimulus Coding. Cell Rep 2020; 31:107483. [PMID: 32294431 PMCID: PMC7218481 DOI: 10.1016/j.celrep.2020.03.047] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2019] [Revised: 01/22/2020] [Accepted: 03/13/2020] [Indexed: 02/02/2023] Open
Abstract
Unbiased and dense sampling of large populations of layer 2/3 pyramidal neurons in mouse primary visual cortex (V1) reveals two functional sub-populations: neurons tuned and untuned to drifting gratings. Whether functional interactions between these two groups contribute to the representation of visual stimuli is unclear. To examine these interactions, we summarize the population partial pairwise correlation structure as a directed and weighted graph. We find that tuned and untuned neurons have distinct topological properties, with untuned neurons occupying central positions in functional networks (FNs). Implementation of a decoder that utilizes the topology of these FNs yields accurate decoding of visual stimuli. We further show that decoding performance degrades comparably following manipulations of either tuned or untuned neurons. Our results demonstrate that untuned neurons are an integral component of V1 FNs and suggest that network interactions contain information about the stimulus that is accessible to downstream elements.
Collapse
Affiliation(s)
- Maayan Levy
- Committee on Computational Neuroscience, The University of Chicago, Chicago, IL 60637, USA
| | - Olaf Sporns
- Indiana University Network Science Institute, Indiana University, Bloomington, IN 47405, USA; Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN 47405, USA
| | - Jason N MacLean
- Committee on Computational Neuroscience, The University of Chicago, Chicago, IL 60637, USA; Department of Neurobiology, The University of Chicago, Chicago, IL 60637, USA; Grossman Institute for Neuroscience, Quantitative Biology and Human Behavior.
| |
Collapse
|