Synergistic Coding of Visual Information in Columnar Networks

KETAMINE: Neural- and network-level changes

Ketamine is a widely used clinical drug that has several functional and clinical applications, including its use as an anaesthetic, analgesic, anti-depressive, anti-suicidal agent, among others. Among its diverse behavioral effects, it influences short-term memory and induces psychedelic effects. At the neural level across different brain areas, it modulates neural firing rates, neural tuning, brain oscillations, and modularity, while promoting hypersynchrony and random connectivity between neurons. In our recent studies we demonstrated that topical application of ketamine on the visual cortex alters neural tuning and promotes vigorous connectivity between neurons by decreasing their firing variability. Here, we begin with a brief review of the literature, followed by results from our lab, where we synthesize a dendritic model of neural tuning and network changes following ketamine application. This model has potential implications for focused modulation of cortical networks in clinical settings. Finally, we identify current gaps in research and suggest directions for future studies, particularly emphasizing the need for more animal experiments to establish a platform for effective translation and synergistic therapies combining ketamine with other protocols such as training and adaptation. In summary, investigating ketamine's broader systemic effects, not only provides deeper insight into cognitive functions and consciousness but also paves the way to advance therapies for neuropsychiatric disorders.

Comparison of orientation encoding across layers within single columns of primate V1 revealed by high-density recordings

Primary visual cortex (V1) has been the focus of extensive neurophysiological investigations, with its laminar organization serving as a crucial model for understanding the functional logic of neocortical microcircuits. Utilizing newly developed high-density, Neuropixels probes, we measured visual responses from large populations of simultaneously recorded neurons distributed across layers of macaque V1. Within single recordings, myriad differences in the functional properties of neuronal subpopulations could be observed. Notably, while standard measurements of orientation selectivity showed only minor differences between laminar compartments, decoding stimulus orientation from layer 4C responses outperformed both superficial and deep layers within the same cortical column. The superior orientation discrimination within layer 4C was associated with greater response reliability of individual neurons rather than lower correlated activity within neuronal populations. Our results underscore the efficacy of high-density electrophysiology in revealing the functional organization and network properties of neocortical microcircuits within single experiments.

Neural interactions in the human frontal cortex dissociate reward and punishment learning

How human prefrontal and insular regions interact while maximizing rewards and minimizing punishments is unknown. Capitalizing on human intracranial recordings, we demonstrate that the functional specificity toward reward or punishment learning is better disentangled by interactions compared to local representations. Prefrontal and insular cortices display non-selective neural populations to rewards and punishments. Non-selective responses, however, give rise to context-specific interareal interactions. We identify a reward subsystem with redundant interactions between the orbitofrontal and ventromedial prefrontal cortices, with a driving role of the latter. In addition, we find a punishment subsystem with redundant interactions between the insular and dorsolateral cortices, with a driving role of the insula. Finally, switching between reward and punishment learning is mediated by synergistic interactions between the two subsystems. These results provide a unifying explanation of distributed cortical representations and interactions supporting reward and punishment learning.

Refinement of efficient encodings of movement in the dorsolateral striatum throughout learning

The striatum is required for normal action selection, movement, and sensorimotor learning. Although action-specific striatal ensembles have been well documented, it is not well understood how these ensembles are formed and how their dynamics may evolve throughout motor learning. Here we used longitudinal 2-photon Ca2+ imaging of dorsal striatal neurons in head-fixed mice as they learned to self-generate locomotion. We observed a significant activation of both direct- and indirect-pathway spiny projection neurons (dSPNs and iSPNs, respectively) during early locomotion bouts and sessions that gradually decreased over time. For dSPNs, onset- and offset-ensembles were gradually refined from active motion-nonspecific cells. iSPN ensembles emerged from neurons initially active during opponent actions before becoming onset- or offset-specific. Our results show that as striatal ensembles are progressively refined, the number of active nonspecific striatal neurons decrease and the overall efficiency of the striatum information encoding for learned actions increases.

Nonuniform and pathway-specific laminar processing of spatial frequencies in the primary visual cortex of primates

The neocortex comprises six cortical layers that play a crucial role in information processing; however, it remains unclear whether laminar processing is consistent across all regions within a single cortex. In this study, we demonstrate diverse laminar response patterns in the primary visual cortex (V1) of three male macaque monkeys when exposed to visual stimuli at different spatial frequencies (SFs). These response patterns can be categorized into two groups. One group exhibit suppressed responses in the output layers for all SFs, while the other type shows amplified responses specifically at high SFs. Further analysis suggests that both magnocellular (M) and parvocellular (P) pathways contribute to the suppressive effect through feedforward mechanisms, whereas amplification is specific to local recurrent mechanisms within the parvocellular pathway. These findings highlight the non-uniform distribution of neural mechanisms involved in laminar processing and emphasize how pathway-specific amplification selectively enhances representations of high-SF information in primate V1.

Distributed representations of prediction error signals across the cortical hierarchy are synergistic

A relevant question concerning inter-areal communication in the cortex is whether these interactions are synergistic. Synergy refers to the complementary effect of multiple brain signals conveying more information than the sum of each isolated signal. Redundancy, on the other hand, refers to the common information shared between brain signals. Here, we dissociated cortical interactions encoding complementary information (synergy) from those sharing common information (redundancy) during prediction error (PE) processing. We analyzed auditory and frontal electrocorticography (ECoG) signals in five common awake marmosets performing two distinct auditory oddball tasks and investigated to what extent event-related potentials (ERP) and broadband (BB) dynamics encoded synergistic and redundant information about PE processing. The information conveyed by ERPs and BB signals was synergistic even at lower stages of the hierarchy in the auditory cortex and between auditory and frontal regions. Using a brain-constrained neural network, we simulated the synergy and redundancy observed in the experimental results and demonstrated that the emergence of synergy between auditory and frontal regions requires the presence of strong, long-distance, feedback, and feedforward connections. These results indicate that distributed representations of PE signals across the cortical hierarchy can be highly synergistic.

Information decomposition and the informational architecture of the brain

To explain how the brain orchestrates information-processing for cognition, we must understand information itself. Importantly, information is not a monolithic entity. Information decomposition techniques provide a way to split information into its constituent elements: unique, redundant, and synergistic information. We review how disentangling synergistic and redundant interactions is redefining our understanding of integrative brain function and its neural organisation. To explain how the brain navigates the trade-offs between redundancy and synergy, we review converging evidence integrating the structural, molecular, and functional underpinnings of synergy and redundancy; their roles in cognition and computation; and how they might arise over evolution and development. Overall, disentangling synergistic and redundant information provides a guiding principle for understanding the informational architecture of the brain and cognition.

Dissecting muscle synergies in the task space

The muscle synergy is a guiding concept in motor control research that relies on the general notion of muscles 'working together' towards task performance. However, although the synergy concept has provided valuable insights into motor coordination, muscle interactions have not been fully characterised with respect to task performance. Here, we address this research gap by proposing a novel perspective to the muscle synergy that assigns specific functional roles to muscle couplings by characterising their task-relevance. Our novel perspective provides nuance to the muscle synergy concept, demonstrating how muscular interactions can 'work together' in different ways: (1) irrespective of the task at hand but also (2) redundantly or (3) complementarily towards common task-goals. To establish this perspective, we leverage information- and network-theory and dimensionality reduction methods to include discrete and continuous task parameters directly during muscle synergy extraction. Specifically, we introduce co-information as a measure of the task-relevance of muscle interactions and use it to categorise such interactions as task-irrelevant (present across tasks), redundant (shared task information), or synergistic (different task information). To demonstrate these types of interactions in real data, we firstly apply the framework in a simple way, revealing its added functional and physiological relevance with respect to current approaches. We then apply the framework to large-scale datasets and extract generalizable and scale-invariant representations consisting of subnetworks of synchronised muscle couplings and distinct temporal patterns. The representations effectively capture the functional interplay between task end-goals and biomechanical affordances and the concurrent processing of functionally similar and complementary task information. The proposed framework unifies the capabilities of current approaches in capturing distinct motor features while providing novel insights and research opportunities through a nuanced perspective to the muscle synergy.

Behavioural relevance of redundant and synergistic stimulus information between functionally connected neurons in mouse auditory cortex

Measures of functional connectivity have played a central role in advancing our understanding of how information is transmitted and processed within the brain. Traditionally, these studies have focused on identifying redundant functional connectivity, which involves determining when activity is similar across different sites or neurons. However, recent research has highlighted the importance of also identifying synergistic connectivity-that is, connectivity that gives rise to information not contained in either site or neuron alone. Here, we measured redundant and synergistic functional connectivity between neurons in the mouse primary auditory cortex during a sound discrimination task. Specifically, we measured directed functional connectivity between neurons simultaneously recorded with calcium imaging. We used Granger Causality as a functional connectivity measure. We then used Partial Information Decomposition to quantify the amount of redundant and synergistic information about the presented sound that is carried by functionally connected or functionally unconnected pairs of neurons. We found that functionally connected pairs present proportionally more redundant information and proportionally less synergistic information about sound than unconnected pairs, suggesting that their functional connectivity is primarily redundant. Further, synergy and redundancy coexisted both when mice made correct or incorrect perceptual discriminations. However, redundancy was much higher (both in absolute terms and in proportion to the total information available in neuron pairs) in correct behavioural choices compared to incorrect ones, whereas synergy was higher in absolute terms but lower in relative terms in correct than in incorrect behavioural choices. Moreover, the proportion of redundancy reliably predicted perceptual discriminations, with the proportion of synergy adding no extra predictive power. These results suggest a crucial contribution of redundancy to correct perceptual discriminations, possibly due to the advantage it offers for information propagation, and also suggest a role of synergy in enhancing information level during correct discriminations.

Evidence for adaptive myelination of subcortical shortcuts for visual motion perception in healthy adults

Conscious visual motion information follows a cortical pathway from the retina to the lateral geniculate nucleus (LGN) and on to the primary visual cortex (V1) before arriving at the middle temporal visual area (MT/V5). Alternative subcortical pathways that bypass V1 are thought to convey unconscious visual information. One flows from the retina to the pulvinar (PUL) and on to medial temporal visual area (MT); while the other directly connects the LGN to MT. Evidence for these pathways comes from non-human primates and modest-sized studies in humans with brain lesions. Thus, the aim of the current study was to reconstruct these pathways in a large sample of neurotypical individuals and to determine the degree to which these pathways are myelinated, suggesting information flow is rapid. We used the publicly available 7T (N = 98; 'discovery') and 3T (N = 381; 'validation') diffusion magnetic resonance imaging datasets from the Human Connectome Project to reconstruct the PUL-MT (including all subcompartments of the PUL) and LGN-MT pathways. We found more fibre tracts with greater density in the left hemisphere. Although the left PUL-MT path was denser, the bilateral LGN-MT tracts were more heavily myelinated, suggesting faster signal transduction. We suggest that this apparent discrepancy may be due to 'adaptive myelination' caused by more frequent use of the LGN-MT pathway that leads to greater myelination and faster overall signal transmission.

Rapid compensatory plasticity revealed by dynamic correlated activity in monkeys in vivo

To produce adaptive behavior, neural networks must balance between plasticity and stability. Computational work has demonstrated that network stability requires plasticity mechanisms to be counterbalanced by rapid compensatory processes. However, such processes have yet to be experimentally observed. Here we demonstrate that repeated optogenetic activation of excitatory neurons in monkey visual cortex (area V1) induces a population-wide dynamic reduction in the strength of neuronal interactions over the timescale of minutes during the awake state, but not during rest. This new form of rapid plasticity was observed only in the correlation structure, with firing rates remaining stable across trials. A computational network model operating in the balanced regime confirmed experimental findings and revealed that inhibitory plasticity is responsible for the decrease in correlated activity in response to repeated light stimulation. These results provide the first experimental evidence for rapid homeostatic plasticity that primarily operates during wakefulness, which stabilizes neuronal interactions during strong network co-activation.

Specialized structure of neural population codes in parietal cortex outputs

Do cortical neurons that send axonal projections to the same target area form specialized population codes for transmitting information? We used calcium imaging in mouse posterior parietal cortex (PPC), retrograde labeling, and statistical multivariate models to address this question during a delayed match-to-sample task. We found that PPC broadcasts sensory, choice, and locomotion signals widely, but sensory information is enriched in the output to anterior cingulate cortex. Neurons projecting to the same area have elevated pairwise activity correlations. These correlations are structured as information-limiting and information-enhancing interaction networks that collectively enhance information levels. This network structure is unique to sub-populations projecting to the same target and strikingly absent in surrounding neural populations with unidentified projections. Furthermore, this structure is only present when mice make correct, but not incorrect, behavioral choices. Therefore, cortical neurons comprising an output pathway form uniquely structured population codes that enhance information transmission to guide accurate behavior.

Convolutional neural networks for vision neuroscience: significance, developments, and outstanding issues

Convolutional Neural Networks (CNN) are a class of machine learning models predominately used in computer vision tasks and can achieve human-like performance through learning from experience. Their striking similarities to the structural and functional principles of the primate visual system allow for comparisons between these artificial networks and their biological counterparts, enabling exploration of how visual functions and neural representations may emerge in the real brain from a limited set of computational principles. After considering the basic features of CNNs, we discuss the opportunities and challenges of endorsing CNNs as in silico models of the primate visual system. Specifically, we highlight several emerging notions about the anatomical and physiological properties of the visual system that still need to be systematically integrated into current CNN models. These tenets include the implementation of parallel processing pathways from the early stages of retinal input and the reconsideration of several assumptions concerning the serial progression of information flow. We suggest design choices and architectural constraints that could facilitate a closer alignment with biology provide causal evidence of the predictive link between the artificial and biological visual systems. Adopting this principled perspective could potentially lead to new research questions and applications of CNNs beyond modeling object recognition.

The QBIT Theory: Consciousness from Entangled Qubits

How does the brain generate consciousness? The present paper is an attempt to answer this question from the perspective of the QBIT theory. In sum, the theory argues that the brain has a prior belief (P) about the stimulus that has caused a sensory representation (R) to be created in the brain. When the conditional entropy of P given R becomes less than zero, the brain becomes more than certain about (i.e. becomes conscious of) the stimulus. Conditional entropy can become negative (and thus the brain can become more than certain) only if the brain uses entangled quantum information in its computations. The QBIT theory suggests that, at the most fundamental level, consciousness is nothing but a special kind of entangled information.

Revealing the Dynamics of Neural Information Processing with Multivariate Information Decomposition

The varied cognitive abilities and rich adaptive behaviors enabled by the animal nervous system are often described in terms of information processing. This framing raises the issue of how biological neural circuits actually process information, and some of the most fundamental outstanding questions in neuroscience center on understanding the mechanisms of neural information processing. Classical information theory has long been understood to be a natural framework within which information processing can be understood, and recent advances in the field of multivariate information theory offer new insights into the structure of computation in complex systems. In this review, we provide an introduction to the conceptual and practical issues associated with using multivariate information theory to analyze information processing in neural circuits, as well as discussing recent empirical work in this vein. Specifically, we provide an accessible introduction to the partial information decomposition (PID) framework. PID reveals redundant, unique, and synergistic modes by which neurons integrate information from multiple sources. We focus particularly on the synergistic mode, which quantifies the "higher-order" information carried in the patterns of multiple inputs and is not reducible to input from any single source. Recent work in a variety of model systems has revealed that synergistic dynamics are ubiquitous in neural circuitry and show reliable structure-function relationships, emerging disproportionately in neuronal rich clubs, downstream of recurrent connectivity, and in the convergence of correlated activity. We draw on the existing literature on higher-order information dynamics in neuronal networks to illustrate the insights that have been gained by taking an information decomposition perspective on neural activity. Finally, we briefly discuss future promising directions for information decomposition approaches to neuroscience, such as work on behaving animals, multi-target generalizations of PID, and time-resolved local analyses.

The structures and functions of correlations in neural population codes

The collective activity of a population of neurons, beyond the properties of individual cells, is crucial for many brain functions. A fundamental question is how activity correlations between neurons affect how neural populations process information. Over the past 30 years, major progress has been made on how the levels and structures of correlations shape the encoding of information in population codes. Correlations influence population coding through the organization of pairwise-activity correlations with respect to the similarity of tuning of individual neurons, by their stimulus modulation and by the presence of higher-order correlations. Recent work has shown that correlations also profoundly shape other important functions performed by neural populations, including generating codes across multiple timescales and facilitating information transmission to, and readout by, downstream brain areas to guide behaviour. Here, we review this recent work and discuss how the structures of correlations can have opposite effects on the different functions of neural populations, thus creating trade-offs and constraints for the structure-function relationships of population codes. Further, we present ideas on how to combine large-scale simultaneous recordings of neural populations, computational models, analyses of behaviour, optogenetics and anatomy to unravel how the structures of correlations might be optimized to serve multiple functions.

Sequential transmission of task-relevant information in cortical neuronal networks

Cortical processing of task-relevant information enables recognition of behaviorally meaningful sensory events. It is unclear how task-related information is represented within cortical networks by the activity of individual neurons and their functional interactions. Here, we use two-photon imaging to record neuronal activity from the primary auditory cortex of mice during a pure-tone discrimination task. We find that a subset of neurons transiently encode sensory information used to inform behavioral choice. Using Granger causality analysis, we show that these neurons form functional networks in which information transmits sequentially. Network structures differ for target versus non-target tones, encode behavioral choice, and differ between correct versus incorrect behavioral choices. Correct behavioral choices are associated with shorter communication timescales, larger functional correlations, and greater information redundancy. In summary, specialized neurons in primary auditory cortex integrate task-related information and form functional networks whose structures encode both sensory input and behavioral choice.

Francis et al. find that, as mice perform an auditory discrimination task, cortical neurons form functional networks in which task-relevant information transmits sequentially between neurons. Network structures encode behavioral choice, and correct behavioral choices are associated with shorter communication timescales, larger functional correlations, and greater information redundancy between neurons.

Spatial Attention Enhances Crowded Stimulus Encoding Across Modeled Receptive Fields by Increasing Redundancy of Feature Representations

Any visual system, biological or artificial, must make a trade-off between the number of units used to represent the visual environment and the spatial resolution of the sampling array. Humans and some other animals are able to allocate attention to spatial locations to reconfigure the sampling array of receptive fields (RFs), thereby enhancing the spatial resolution of representations without changing the overall number of sampling units. Here, we examine how representations of visual features in a fully convolutional neural network interact and interfere with each other in an eccentricity-dependent RF pooling array and how these interactions are influenced by dynamic changes in spatial resolution across the array. We study these feature interactions within the framework of visual crowding, a well-characterized perceptual phenomenon in which target objects in the visual periphery that are easily identified in isolation are much more difficult to identify when flanked by similar nearby objects. By separately simulating effects of spatial attention on RF size and on the density of the pooling array, we demonstrate that the increase in RF density due to attention is more beneficial than changes in RF size for enhancing target classification for crowded stimuli. Furthermore, by varying target/flanker spacing, as well as the spatial extent of attention, we find that feature redundancy across RFs has more influence on target classification than the fidelity of the feature representations themselves. Based on these findings, we propose a candidate mechanism by which spatial attention relieves visual crowding through enhanced feature redundancy that is mostly due to increased RF density.

Heterogeneous side effects of cortical inactivation in behaving animals

Cortical inactivation represents a key causal manipulation allowing the study of cortical circuits and their impact on behavior. A key assumption in inactivation studies is that the neurons in the target area become silent while the surrounding cortical tissue is only negligibly impacted. However, individual neurons are embedded in complex local circuits composed of excitatory and inhibitory cells with connections extending hundreds of microns. This raises the possibility that silencing one part of the network could induce complex, unpredictable activity changes in neurons outside the targeted inactivation zone. These off-target side effects can potentially complicate interpretations of inactivation manipulations, especially when they are related to changes in behavior. Here, we demonstrate that optogenetic inactivation of glutamatergic neurons in the superficial layers of monkey primary visual cortex (V1) induces robust suppression at the light-targeted site, but destabilizes stimulus responses in the neighboring, untargeted network. We identified four types of stimulus-evoked neuronal responses within a cortical column, ranging from full suppression to facilitation, and a mixture of both. Mixed responses were most prominent in middle and deep cortical layers. These results demonstrate that response modulation driven by lateral network connectivity is diversely implemented throughout a cortical column. Importantly, consistent behavioral changes induced by optogenetic inactivation were only achieved when cumulative network activity was homogeneously suppressed. Therefore, careful consideration of the full range of network changes outside the inactivated cortical region is required, as heterogeneous side effects can confound interpretation of inactivation experiments.

The Geometry of Information Coding in Correlated Neural Populations

Neurons in the brain represent information in their collective activity. The fidelity of this neural population code depends on whether and how variability in the response of one neuron is shared with other neurons. Two decades of studies have investigated the influence of these noise correlations on the properties of neural coding. We provide an overview of the theoretical developments on the topic. Using simple, qualitative, and general arguments, we discuss, categorize, and relate the various published results. We emphasize the relevance of the fine structure of noise correlation, and we present a new approach to the issue. Throughout this review, we emphasize a geometrical picture of how noise correlations impact the neural code.

Pairwise Synchrony and Correlations Depend on the Structure of the Population Code in Visual Cortex

In visual areas of primates, neurons activate in parallel while the animal is engaged in a behavioral task. In this study, we examine the structure of the population code while the animal performs delayed match-to-sample tasks on complex natural images. The macaque monkeys visualized two consecutive stimuli that were either the same or different, while being recorded with laminar arrays across the cortical depth in cortical areas V1 and V4. We decode correct choice behavior from neural populations of simultaneously recorded units. Utilizing decoding weights, we divide neurons into most informative and less informative and show that most informative neurons in V4, but not in V1, are more strongly synchronized, coupled, and correlated than less informative neurons. Because neurons are divided into two coding pools according to their coding preference, in V4, but not in V1, spiking synchrony, coupling, and correlations within the coding pool are stronger than across coding pools.

Inferring neural information flow from spiking data

The brain can be regarded as an information processing system in which neurons store and propagate information about external stimuli and internal processes. Therefore, estimating interactions between neural activity at the cellular scale has significant implications in understanding how neuronal circuits encode and communicate information across brain areas to generate behavior. While the number of simultaneously recorded neurons is growing exponentially, current methods relying only on pairwise statistical dependencies still suffer from a number of conceptual and technical challenges that preclude experimental breakthroughs describing neural information flows. In this review, we examine the evolution of the field over the years, starting from descriptive statistics to model-based and model-free approaches. Then, we discuss in detail the Granger Causality framework, which includes many popular state-of-the-art methods and we highlight some of its limitations from a conceptual and practical estimation perspective. Finally, we discuss directions for future research, including the development of theoretical information flow models and the use of dimensionality reduction techniques to extract relevant interactions from large-scale recording datasets.

Correlated activity favors synergistic processing in local cortical networks in vitro at synaptically relevant timescales

Neural information processing is widely understood to depend on correlations in neuronal activity. However, whether correlation is favorable or not is contentious. Here, we sought to determine how correlated activity and information processing are related in cortical circuits. Using recordings of hundreds of spiking neurons in organotypic cultures of mouse neocortex, we asked whether mutual information between neurons that feed into a common third neuron increased synergistic information processing by the receiving neuron. We found that mutual information and synergistic processing were positively related at synaptic timescales (0.05-14 ms), where mutual information values were low. This effect was mediated by the increase in information transmission-of which synergistic processing is a component-that resulted as mutual information grew. However, at extrasynaptic windows (up to 3,000 ms), where mutual information values were high, the relationship between mutual information and synergistic processing became negative. In this regime, greater mutual information resulted in a disproportionate increase in redundancy relative to information transmission. These results indicate that the emergence of synergistic processing from correlated activity differs according to timescale and correlation regime. In a low-correlation regime, synergistic processing increases with greater correlation, and in a high-correlation regime, synergistic processing decreases with greater correlation.

Synergy in Cortical Networks

A dot in this text can simultaneously stimulate thousands of neurons in primary visual cortex, a response that may seem unnecessarily redundant. Contrary to such a view, in this issue of Neuron, Nigam et al. (2019) demonstrate that these co-activated neurons generate abundant synergistic interactions that help to decode the stimulus.