1
|
Exploitation of image statistics with sparse coding in the case of stereo vision. Neural Netw 2020; 135:158-176. [PMID: 33388507 DOI: 10.1016/j.neunet.2020.12.016] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2020] [Revised: 12/06/2020] [Accepted: 12/14/2020] [Indexed: 11/23/2022]
Abstract
The sparse coding algorithm has served as a model for early processing in mammalian vision. It has been assumed that the brain uses sparse coding to exploit statistical properties of the sensory stream. We hypothesize that sparse coding discovers patterns from the data set, which can be used to estimate a set of stimulus parameters by simple readout. In this study, we chose a model of stereo vision to test our hypothesis. We used the Locally Competitive Algorithm (LCA), followed by a naïve Bayes classifier, to infer stereo disparity. From the results we report three observations. First, disparity inference was successful with this naturalistic processing pipeline. Second, an expanded, highly redundant representation is required to robustly identify the input patterns. Third, the inference error can be predicted from the number of active coefficients in the LCA representation. We conclude that sparse coding can generate a suitable general representation for subsequent inference tasks.
Collapse
|
2
|
Johnston WJ, Palmer SE, Freedman DJ. Nonlinear mixed selectivity supports reliable neural computation. PLoS Comput Biol 2020; 16:e1007544. [PMID: 32069273 PMCID: PMC7048320 DOI: 10.1371/journal.pcbi.1007544] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2019] [Revised: 02/28/2020] [Accepted: 11/12/2019] [Indexed: 12/17/2022] Open
Abstract
Neuronal activity in the brain is variable, yet both perception and behavior are generally reliable. How does the brain achieve this? Here, we show that the conjunctive coding of multiple stimulus features, commonly known as nonlinear mixed selectivity, may be used by the brain to support reliable information transmission using unreliable neurons. Nonlinearly mixed feature representations have been observed throughout primary sensory, decision-making, and motor brain areas. In these areas, different features are almost always nonlinearly mixed to some degree, rather than represented separately or with only additive (linear) mixing, which we refer to as pure selectivity. Mixed selectivity has been previously shown to support flexible linear decoding for complex behavioral tasks. Here, we show that it has another important benefit: in many cases, it makes orders of magnitude fewer decoding errors than pure selectivity even when both forms of selectivity use the same number of spikes. This benefit holds for sensory, motor, and more abstract, cognitive representations. Further, we show experimental evidence that mixed selectivity exists in the brain even when it does not enable behaviorally useful linear decoding. This suggests that nonlinear mixed selectivity may be a general coding scheme exploited by the brain for reliable and efficient neural computation. Neurons in the brain are unreliable, while both perception and behavior are generally reliable. In this work, we study how the neural population response to sensory, motor, and cognitive features can produce this reliability. Across the brain, single neurons have been shown to respond to particular conjunctions of multiple features, termed nonlinear mixed selectivity. In this work, we show that populations of these mixed selective neurons lead to many fewer decoding errors than populations without mixed selectivity, even when both neural codes are given the same number of spikes. We show that the reliability benefits from mixed selectivity are quite general, holding under different assumptions about metabolic costs and neural noise as well as for both categorical and sensory errors. Further, previous theoretical work has shown that mixed selectivity enables the learning of complex behaviors with simple decoders. Through the analysis of neural data, we show that the brain implements mixed selectivity even when it would not serve this purpose. Thus, we argue that the brain also implements mixed selectivity to exploit its general benefits for reliable and efficient neural computation.
Collapse
Affiliation(s)
- W. Jeffrey Johnston
- Graduate Program in Computational Neuroscience, The University of Chicago, Chicago, Illinois, United States of America
- Department of Neurobiology, The University of Chicago, Chicago, Illinois, United States of America
- * E-mail:
| | - Stephanie E. Palmer
- Graduate Program in Computational Neuroscience, The University of Chicago, Chicago, Illinois, United States of America
- Grossman Institute for Neuroscience, Quantitative Biology, and Human Behavior, The University of Chicago, Chicago, Illinois, United States of America
- Department of Organismal Biology and Anatomy, The University of Chicago, Chicago, Illinois, United States of America
- Department of Physics, The University of Chicago, Chicago, Illinois, United States of America
| | - David J. Freedman
- Graduate Program in Computational Neuroscience, The University of Chicago, Chicago, Illinois, United States of America
- Department of Neurobiology, The University of Chicago, Chicago, Illinois, United States of America
- Grossman Institute for Neuroscience, Quantitative Biology, and Human Behavior, The University of Chicago, Chicago, Illinois, United States of America
| |
Collapse
|
3
|
Nightingale SJ, Wade KA, Watson DG. Can people identify original and manipulated photos of real-world scenes? COGNITIVE RESEARCH-PRINCIPLES AND IMPLICATIONS 2017; 2:30. [PMID: 28776002 PMCID: PMC5514174 DOI: 10.1186/s41235-017-0067-2] [Citation(s) in RCA: 48] [Impact Index Per Article: 6.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/20/2016] [Accepted: 06/12/2017] [Indexed: 05/29/2023]
Abstract
Advances in digital technology mean that the creation of visually compelling photographic fakes is growing at an incredible speed. The prevalence of manipulated photos in our everyday lives invites an important, yet largely unanswered, question: Can people detect photo forgeries? Previous research using simple computer-generated stimuli suggests people are poor at detecting geometrical inconsistencies within a scene. We do not know, however, whether such limitations also apply to real-world scenes that contain common properties that the human visual system is attuned to processing. In two experiments we asked people to detect and locate manipulations within images of real-world scenes. Subjects demonstrated a limited ability to detect original and manipulated images. Furthermore, across both experiments, even when subjects correctly detected manipulated images, they were often unable to locate the manipulation. People’s ability to detect manipulated images was positively correlated with the extent of disruption to the underlying structure of the pixels in the photo. We also explored whether manipulation type and individual differences were associated with people’s ability to identify manipulations. Taken together, our findings show, for the first time, that people have poor ability to identify whether a real-world image is original or has been manipulated. The results have implications for professionals working with digital images in legal, media, and other domains.
Collapse
Affiliation(s)
| | - Kimberley A Wade
- Department of Psychology, University of Warwick, Coventry, CV4 7AL UK
| | - Derrick G Watson
- Department of Psychology, University of Warwick, Coventry, CV4 7AL UK
| |
Collapse
|
4
|
Trinh AT, Harvey-Girard E, Teixeira F, Maler L. Cryptic laminar and columnar organization in the dorsolateral pallium of a weakly electric fish. J Comp Neurol 2015; 524:408-28. [PMID: 26234725 DOI: 10.1002/cne.23874] [Citation(s) in RCA: 29] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2015] [Revised: 07/28/2015] [Accepted: 07/28/2015] [Indexed: 01/25/2023]
Abstract
In the weakly electric gymnotiform fish, Apteronotus leptorhynchus, the dorsolateral pallium (DL) receives diencephalic inputs representing electrosensory input utilized for communication and navigation. Cell counts reveal that, similar to thalamocortical projections, many more cells are present in DL than in the diencephalic nucleus that provides it with sensory input. DL is implicated in learning and memory and considered homologous to medial and/or dorsal pallium. The gymnotiform DL has an apparently simple architecture with a random distribution of simple multipolar neurons. We used multiple neurotracer injections in order to study the microcircuitry of DL. Surprisingly, we demonstrated that the intrinsic connectivity of DL is highly organized. It consists of orthogonal laminar and vertical excitatory synaptic connections. The laminar synaptic connections are symmetric sparse, random, and drop off exponentially with distance; they parcellate DL into narrow (60 μm) overlapping cryptic layers. At distances greater than 100 μm, the laminar connections generate a strongly connected directed graph architecture within DL. The vertical connectivity suggests that DL is also organized into cryptic columns; these connections are highly asymmetric, with superficial DL cells preferentially projecting towards deeper cells. Our experimental analyses suggest that the overlapping cryptic columns have a width of 100 μm, in agreement with the minimal distance for strong connectivity. The architecture of DL and the expansive representation of its input, taken together with the strong expression of N-methyl-D-aspartate (NMDA) receptors by its cells, are consistent with theoretical ideas concerning the cortical computations of pattern separation and memory storage via bump attractors.
Collapse
Affiliation(s)
- Anh-Tuan Trinh
- Department of Cellular and Molecular Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Erik Harvey-Girard
- Department of Cellular and Molecular Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Fellipe Teixeira
- Department of Cellular and Molecular Medicine, University of Ottawa, Ottawa, Ontario, Canada.,Departamento de Biofísica, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil
| | - Leonard Maler
- Department of Cellular and Molecular Medicine, University of Ottawa, Ottawa, Ontario, Canada.,Center for Neural Dynamics, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
5
|
Giassi AC, Ellis W, Maler L. Organization of the gymnotiform fish pallium in relation to learning and memory: III. Intrinsic connections. J Comp Neurol 2012; 520:3369-94. [DOI: 10.1002/cne.23108] [Citation(s) in RCA: 33] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
|
6
|
Barlow HB. Single units and sensation: a neuron doctrine for perceptual psychology? Perception 2010; 38:795-8. [PMID: 19806956 DOI: 10.1068/pmkbar] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
7
|
Westover MB, O'Sullivan JA. Achievable Rates for Pattern Recognition. IEEE TRANSACTIONS ON INFORMATION THEORY 2008; 54:299-320. [PMID: 32153303 PMCID: PMC7062371 DOI: 10.1109/tit.2007.911296] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
Biological and machine pattern recognition systems face a common challenge: Given sensory data about an unknown pattern, classify the pattern by searching for the best match within a library of representations stored in memory. In many cases, the number of patterns to be discriminated and the richness of the raw data force recognition systems to internally represent memory and sensory information in a compressed format. However, these representations must preserve enough information to accommodate the variability and complexity of the environment, otherwise recognition will be unreliable. Thus, there is an intrinsic tradeoff between the amount of resources devoted to data representation and the complexity of the environment in which a recognition system may reliably operate. In this paper, we describe a mathematical model for pattern recognition systems subject to resource constraints, and show how the aforementioned resource-complexity tradeoff can be characterized in terms of three rates related to the number of bits available for representing memory and sensory data, and the number of patterns populating a given statistical environment. We prove single-letter information-theoretic bounds governing the achievable rates, and investigate in detail two illustrative cases where the pattern data is either binary or Gaussian.
Collapse
Affiliation(s)
- M Brandon Westover
- Department of Neurology, Massachusetts General Hospital, Boston, MA 02114-2622 USA
| | - Joseph A O'Sullivan
- Department of Electrical engineering, Washington University, St. Louis, MO 63130 USA
| |
Collapse
|
8
|
Földiák P, Xiao D, Keysers C, Edwards R, Perrett DI. Rapid serial visual presentation for the determination of n eural selectivity in area STSa. PROGRESS IN BRAIN RESEARCH 2004; 144:107-16. [PMID: 14650843 DOI: 10.1016/s0079-6123(03)14407-x] [Citation(s) in RCA: 14] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Abstract
We show that rapid serial visual presentation (RSVP) in combination with a progressive reduction of the stimulus set is an efficient method for describing the selectivity properties of high-level cortical neurons in single-cell electrophysiological recording experiments. Rapid presentation allows the experimental testing of a significantly larger number of stimuli, which can reduce the subjectivity of the results due to stimulus selection and the lack of sufficient control stimuli. We prove the reliability of the rapid presentation and stimulus reduction methods by repeated experiments and the comparison of different testing conditions. Our results from neurons in area STSa of the macaque temporal cortex provide a well-controlled confirmation for the existence of a population of cells that respond selectively to stimuli containing faces. View tuning properties measured using this method also confirmed earlier results. In addition, we found a population of cells that respond reliably to complex non-face stimuli, though their tuning properties are not obvious.
Collapse
Affiliation(s)
- Peter Földiák
- School of Psychology, University of St. Andrews, St. Andrews, Fife, KY16 9JU, UK.
| | | | | | | | | |
Collapse
|
9
|
Abstract
Much of perception, learning and high-level cognition involves finding patterns in data. But there are always infinitely many patterns compatible with any finite amount of data. How does the cognitive system choose 'sensible' patterns? A long tradition in epistemology, philosophy of science, and mathematical and computational theories of learning argues that patterns 'should' be chosen according to how simply they explain the data. This article reviews research exploring the idea that simplicity drives a wide range of cognitive processes. We outline mathematical theory, computational results and empirical data that underpin this viewpoint.
Collapse
Affiliation(s)
- Nick Chater
- Institute for Applied Cognitive Science, Department of Psychology, University of Warwick, CV4 7AL, Coventry, UK
| | | |
Collapse
|
10
|
|