Three aspects of representation in neuroscience

Mental state decoders: game-changers or wishful thinking?

Decoding mental and perceptual states using fMRI has become increasingly popular over the past two decades, with numerous highly-cited studies published in high-profile journals. Nevertheless, what have we learned from these decoders? In this opinion, we argue that fMRI-based decoders are not neurophysiologically informative and are not, and likely cannot be, applicable to real-world decision-making. The former point stems from the fact that decoding models cannot disentangle neural mechanisms from their epiphenomena. The latter point stems from both logical and ethical constraints. Constructing decoders requires precious time and resources that should instead be directed toward scientific endeavors more likely to yield meaningful scientific progress.

Formalising the role of behaviour in neuroscience

We develop a mathematical approach to formally proving that certain neural computations and representations exist based on patterns observed in an organism's behaviour. To illustrate, we provide a simple set of conditions under which an ant's ability to determine how far it is from its nest would logically imply neural structures isomorphic to the natural numbers ℕ . We generalise these results to arbitrary behaviours and representations and show what mathematical characterisation of neural computation and representation is simplest while being maximally predictive of behaviour. We develop this framework in detail using a path integration example, where an organism's ability to search for its nest in the correct location implies representational structures isomorphic to two-dimensional coordinates under addition. We also study a system for processinga n b n strings common in comparative work. Our approach provides an objective way to determine what theory of a physical system is best, addressing a fundamental challenge in neuroscientific inference. These results motivate considering which neurobiological structures have the requisite formal structure and are otherwise physically plausible given relevant physical considerations such as generalisability, information density, thermodynamic stability and energetic cost.

Computation for cognitive science: Analog versus digital

Cognitive science was founded on the idea that the mind/brain can be understood in computational terms. While computational modeling in science is ubiquitous, cognitive science takes the stronger stance that the mind/brain literally performs computations. Moreover, performing computations is crucial to explaining what the mind/brain does, qua mind/brain. Unfortunately, most scientists fail to consider analog computation as a legitimate and theoretically useful type of computation in addition to digital computation; to the extent that analog computation is acknowledged, it is mostly based on a simplistic and incomplete understanding. Taking computation to consist of only one type (i.e., digital) while ignoring another, interestingly distinct type (i.e., analog) leads to an impoverished understanding of what it could mean for minds/brains to compute. A full appreciation and understanding of analog computation-particularly in relation to digital computation-allows researchers to develop computational frameworks and hypotheses in new and exciting ways. Thus, somewhat counterintuitively, looking to the once-dominant computing paradigm of yesteryear can provide novel computational ways of thinking about the mind and brain. This article is categorized under: Philosophy > Foundations of Cognitive Science.

Connectivity analyses for task-based fMRI

Functional connectivity is conventionally defined by measuring the similarity between brain signals from two regions. The technique has become widely adopted in the analysis of functional magnetic resonance imaging (fMRI) data, where it has provided cognitive neuroscientists with abundant information on how brain regions interact to support complex cognition. However, in the past decade the notion of "connectivity" has expanded in both the complexity and heterogeneity of its application to cognitive neuroscience, resulting in greater difficulty of interpretation, replication, and cross-study comparisons. In this paper, we begin with the canonical notions of functional connectivity and then introduce recent methodological developments that either estimate some alternative form of connectivity or extend the analytical framework, with the hope of bringing better clarity for cognitive neuroscience researchers.

Compositionality in perception: A framework

Perception involves the processing of content or information about the world. In what form is this content represented? I argue that perception is widely compositional. The perceptual system represents many stimulus features (including shape, orientation, and motion) in terms of combinations of other features (such as shape parts, slant and tilt, common and residual motion vectors). But compositionality can take a variety of forms. The ways in which perceptual representations compose are markedly different from the ways in which sentences or thoughts are thought to be composed. I suggest that the thesis that perception is compositional is not itself a concrete hypothesis with specific predictions; rather it affords a productive framework for developing and evaluating specific empirical hypotheses about the form and content of perceptual representations. The question is not just whether perception is compositional, but how. Answering this latter question can provide fundamental insights into perception. This article is categorized under: Philosophy > Representation Philosophy > Foundations of Cognitive Science Psychology > Perception and Psychophysics.

Neural representation in active inference: Using generative models to interact with-and understand-the lived world

This paper considers neural representation through the lens of active inference, a normative framework for understanding brain function. It delves into how living organisms employ generative models to minimize the discrepancy between predictions and observations (as scored with variational free energy). The ensuing analysis suggests that the brain learns generative models to navigate the world adaptively, not (or not solely) to understand it. Different living organisms may possess an array of generative models, spanning from those that support action-perception cycles to those that underwrite planning and imagination; namely, from explicit models that entail variables for predicting concurrent sensations, like objects, faces, or people-to action-oriented models that predict action outcomes. It then elucidates how generative models and belief dynamics might link to neural representation and the implications of different types of generative models for understanding an agent's cognitive capabilities in relation to its ecological niche. The paper concludes with open questions regarding the evolution of generative models and the development of advanced cognitive abilities-and the gradual transition from pragmatic to detached neural representations. The analysis on offer foregrounds the diverse roles that generative models play in cognitive processes and the evolution of neural representation.

Downstream network transformations dissociate neural activity from causal functional contributions

Neuroscientists rely on distributed spatio-temporal patterns of neural activity to understand how neural units contribute to cognitive functions and behavior. However, the extent to which neural activity reliably indicates a unit's causal contribution to the behavior is not well understood. To address this issue, we provide a systematic multi-site perturbation framework that captures time-varying causal contributions of elements to a collectively produced outcome. Applying our framework to intuitive toy examples and artificial neural networks revealed that recorded activity patterns of neural elements may not be generally informative of their causal contribution due to activity transformations within a network. Overall, our findings emphasize the limitations of inferring causal mechanisms from neural activities and offer a rigorous lesioning framework for elucidating causal neural contributions.

Seeing social interactions

Seeing the interactions between other people is a critical part of our everyday visual experience, but recognizing the social interactions of others is often considered outside the scope of vision and grouped with higher-level social cognition like theory of mind. Recent work, however, has revealed that recognition of social interactions is efficient and automatic, is well modeled by bottom-up computational algorithms, and occurs in visually-selective regions of the brain. We review recent evidence from these three methodologies (behavioral, computational, and neural) that converge to suggest the core of social interaction perception is visual. We propose a computational framework for how this process is carried out in the brain and offer directions for future interdisciplinary investigations of social perception.

Circular and unified analysis in network neuroscience

Genuinely new discovery transcends existing knowledge. Despite this, many analyses in systems neuroscience neglect to test new speculative hypotheses against benchmark empirical facts. Some of these analyses inadvertently use circular reasoning to present existing knowledge as new discovery. Here, I discuss that this problem can confound key results and estimate that it has affected more than three thousand studies in network neuroscience over the last decade. I suggest that future studies can reduce this problem by limiting the use of speculative evidence, integrating existing knowledge into benchmark models, and rigorously testing proposed discoveries against these models. I conclude with a summary of practical challenges and recommendations.

Visual Representations: Insights from Neural Decoding

Patterns of brain activity contain meaningful information about the perceived world. Recent decades have welcomed a new era in neural analyses, with computational techniques from machine learning applied to neural data to decode information represented in the brain. In this article, we review how decoding approaches have advanced our understanding of visual representations and discuss efforts to characterize both the complexity and the behavioral relevance of these representations. We outline the current consensus regarding the spatiotemporal structure of visual representations and review recent findings that suggest that visual representations are at once robust to perturbations, yet sensitive to different mental states. Beyond representations of the physical world, recent decoding work has shone a light on how the brain instantiates internally generated states, for example, during imagery and prediction. Going forward, decoding has remarkable potential to assess the functional relevance of visual representations for human behavior, reveal how representations change across development and during aging, and uncover their presentation in various mental disorders.

The 21st century engram

The search for the engram-the neural mechanism of memory-has been a guiding research project for neuroscience since its emergence as a distinct scientific field. Recent developments in the tools and techniques available for investigating the mechanisms of memory have allowed researchers to proclaimed the search is over. While there is ongoing debate about the justification for that claim, renewed interest in the engram is clear. This attention highlights the impoverished status of the engram concept. As research accelerates, the simple characterization of the engram as an enduring physical change is stretched thin. Now that the engram commitment has been made more explicit, it must also be made more precise. If the project of 20th century neurobiology was finding the engram, the project of the 21st must be supplying a richer account of what's been found. This paper sketches a history of the engram, and a way forward. This article is categorized under: Philosophy > Foundations of Cognitive Science.

Distinct ventral stream and prefrontal cortex representational dynamics during sustained conscious visual perception

Instances of sustained stationary sensory input are ubiquitous. However, previous work focused almost exclusively on transient onset responses. This presents a critical challenge for neural theories of consciousness, which should account for the full temporal extent of experience. To address this question, we use intracranial recordings from ten human patients with epilepsy to view diverse images of multiple durations. We reveal that, in sensory regions, despite dramatic changes in activation magnitude, the distributed representation of categories and exemplars remains sustained and stable. In contrast, in frontoparietal regions, we find transient content representation at stimulus onset. Our results highlight the connection between the anatomical and temporal correlates of experience. To the extent perception is sustained, it may rely on sensory representations and to the extent perception is discrete, centered on perceptual updating, it may rely on frontoparietal representations.

Investigating the concept of representation in the neural and psychological sciences

The concept of representation is commonly treated as indispensable to research on brains, behavior, and cognition. Nevertheless, systematic evidence about the ways the concept is applied remains scarce. We present the results of an experiment aimed at elucidating what researchers mean by "representation." Participants were an international group of psychologists, neuroscientists, and philosophers (N = 736). Applying elicitation methodology, participants responded to a survey with experimental scenarios aimed at invoking applications of "representation" and five other ways of describing how the brain responds to stimuli. While we find little disciplinary variation in the application of "representation" and other expressions (e.g., "about" and "carry information"), the results suggest that researchers exhibit uncertainty about what sorts of brain activity involve representations or not; they also prefer non-representational, causal characterizations of the brain's response to stimuli. Potential consequences of these findings are explored, such as reforming or eliminating the concept of representation from use.

When Neural Activity Fails to Reveal Causal Contributions

Neuroscientists rely on distributed spatio-temporal patterns of neural activity to understand how neural units contribute to cognitive functions and behavior. However, the extent to which neural activity reliably indicates a unit's causal contribution to the behavior is not well understood. To address this issue, we provide a systematic multi-site perturbation framework that captures time-varying causal contributions of elements to a collectively produced outcome. Applying our framework to intuitive toy examples and artificial neuronal networks revealed that recorded activity patterns of neural elements may not be generally informative of their causal contribution due to activity transformations within a network. Overall, our findings emphasize the limitations of inferring causal mechanisms from neural activities and offer a rigorous lesioning framework for elucidating causal neural contributions.

Comparing representations and computations in single neurons versus neural networks

Single-neuron-level explanations have been the gold standard in neuroscience for decades. Recently, however, neural-network-level explanations have become increasingly popular. This increase in popularity is driven by the fact that the analysis of neural networks can solve problems that cannot be addressed by analyzing neurons independently. In this opinion article, I argue that while both frameworks employ the same general logic to link physical and mental phenomena, in many cases the neural network framework provides better explanatory objects to understand representations and computations related to mental phenomena. I discuss what constitutes a mechanistic explanation in neural systems, provide examples, and conclude by highlighting a number of the challenges and considerations associated with the use of analyses of neural networks to study brain function.

Nine insights from internet engineering that help us understand brain network communication

Philosophers have long recognized the value of metaphor as a tool that opens new avenues of investigation. By seeing brains as having the goal of representation, the computer metaphor in its various guises has helped systems neuroscience approach a wide array of neuronal behaviors at small and large scales. Here I advocate a complementary metaphor, the internet. Adopting this metaphor shifts our focus from computing to communication, and from seeing neuronal signals as localized representational elements to seeing neuronal signals as traveling messages. In doing so, we can take advantage of a comparison with the internet's robust and efficient routing strategies to understand how the brain might meet the challenges of network communication. I lay out nine engineering strategies that help the internet solve routing challenges similar to those faced by brain networks. The internet metaphor helps us by reframing neuronal activity across the brain as, in part, a manifestation of routing, which may, in different parts of the system, resemble the internet more, less, or not at all. I describe suggestive evidence consistent with the brain's use of internet-like routing strategies and conclude that, even if empirical data do not directly implicate internet-like routing, the metaphor is valuable as a reference point for those investigating the difficult problem of network communication in the brain and in particular the problem of routing.