Multivariate information theory uncovers synergistic subsystems of the human cerebral cortex

Efficient Search Algorithms for Identifying Synergistic Associations in High-Dimensional Datasets

In recent years, there has been a notably increased interest in the study of multivariate interactions and emergent higher-order dependencies. This is particularly evident in the context of identifying synergistic sets, which are defined as combinations of elements whose joint interactions result in the emergence of information that is not present in any individual subset of those elements. The scalability of frameworks such as partial information decomposition (PID) and those based on multivariate extensions of mutual information, such as O-information, is limited by combinational explosion in the number of sets that must be assessed. In order to address these challenges, we propose a novel approach that utilises stochastic search strategies in order to identify synergistic triplets within datasets. Furthermore, the methodology is extensible to larger sets and various synergy measures. By employing stochastic search, our approach circumvents the constraints of exhaustive enumeration, offering a scalable and efficient means to uncover intricate dependencies. The flexibility of our method is illustrated through its application to two epidemiological datasets: The Young Finns Study and the UK Biobank Nuclear Magnetic Resonance (NMR) data. Additionally, we present a heuristic for reducing the number of synergistic sets to analyse in large datasets by excluding sets with overlapping information. We also illustrate the risks of performing a feature selection before assessing synergistic information in the system.

Milking a spherical cow: Toy models in neuroscience

There are many different kinds of models, and they play many different roles in the scientific endeavour. Neuroscience, and biology more generally, has understandably tended to emphasise empirical models that are grounded in data and make specific, experimentally testable predictions. Meanwhile, strongly idealised or 'toy' models have played a central role in the theoretical development of other sciences such as physics. In this paper, we examine the nature of toy models and their prospects in neuroscience.

Competitive interactions shape brain dynamics and computation across species

Adaptive cognition relies on cooperation across anatomically distributed brain circuits. However, specialised neural systems are also in constant competition for limited processing resources. How does the brain's network architecture enable it to balance these cooperative and competitive tendencies? Here we use computational whole-brain modelling to examine the dynamical and computational relevance of cooperative and competitive interactions in the mammalian connectome. Across human, macaque, and mouse we show that the architecture of the models that most faithfully reproduce brain activity, consistently combines modular cooperative interactions with diffuse, long-range competitive interactions. The model with competitive interactions consistently outperforms the cooperative-only model, with excellent fit to both spatial and dynamical properties of the living brain, which were not explicitly optimised but rather emerge spontaneously. Competitive interactions in the effective connectivity produce greater levels of synergistic information and local-global hierarchy, and lead to superior computational capacity when used for neuromorphic computing. Altogether, this work provides a mechanistic link between network architecture, dynamical properties, and computation in the mammalian brain.

Information dynamics of in silico EEG Brain Waves: Insights into oscillations and functions

The relation between electroencephalography (EEG) rhythms, brain functions, and behavioral correlates is well-established. Some physiological mechanisms underlying rhythm generation are understood, enabling the replication of brain rhythms in silico. This offers a pathway to explore connections between neural oscillations and specific neuronal circuits, potentially yielding fundamental insights into the functional properties of brain waves. Information theory frameworks, such as Integrated Information Decomposition (Φ-ID), relate dynamical regimes with informational properties, providing deeper insights into neuronal dynamic functions. Here, we investigate wave emergence in an excitatory/inhibitory (E/I) balanced network of integrate and fire neurons with short-term synaptic plasticity. This model produces a diverse range of EEG-like rhythms, from low δ waves to high-frequency oscillations. Through Φ-ID, we analyze the network's information dynamics and its relation with different emergent rhythms, elucidating the system's suitability for functions such as robust information transfer, storage, and parallel operation. Furthermore, our study helps to identify regimes that may resemble pathological states due to poor informational properties and high randomness. We found, e.g., that in silico β and δ waves are associated with maximum information transfer in inhibitory and excitatory neuron populations, respectively, and that the coexistence of excitatory θ, α, and β waves is associated to information storage. Additionally, we observed that high-frequency oscillations can exhibit either high or poor informational properties, potentially shedding light on ongoing discussions regarding physiological versus pathological high-frequency oscillations. In summary, our study demonstrates that dynamical regimes with similar oscillations may exhibit vastly different information dynamics. Characterizing information dynamics within these regimes serves as a potent tool for gaining insights into the functions of complex neuronal networks. Finally, our findings suggest that the use of information dynamics in both model and experimental data analysis, could help discriminate between oscillations associated with cognitive functions and those linked to neuronal disorders.

Synergy Makes Direct Perception Inefficient

A typical claim in anti-representationalist approaches to cognition such as ecological psychology or radical embodied cognitive science is that ecological information is sufficient for guiding behavior. According to this view, affordances are immediately perceptually available to the agent (in the so-called "ambient energy array"), so sensory data does not require much further inner processing. As a consequence, mental representations are explanatorily idle: perception is immediate and direct. Here we offer one way to formalize this direct-perception claim and identify some important limits to it. We argue that the claim should be read as saying that successful behavior just implies picking out affordance-related information from the ambient energy array. By relying on the Partial Information Decomposition framework, and more concretely on its development of the notion of synergy, we show that in multimodal perception, where various energy arrays carry affordance-related information, the "just pick out affordance-related information" approach is very inefficient, as it is bound to miss all synergistic components. Efficient multimodal information combination requires transmitting sensory-specific (and not affordance-specific) information to wherever it is that the various information streams are combined. The upshot is that some amount of computation is necessary for efficient affordance reconstruction.

High-order brain interactions in ketamine during rest and task: a double-blinded cross-over design using portable EEG on male participants

Ketamine is a dissociative anesthetic that induces a shift in global consciousness states and related brain dynamics. Portable low-density EEG systems could be used to monitor these effects. However, previous evidence is almost null and lacks adequate methods to address global dynamics with a small number of electrodes. This study delves into brain high-order interactions (HOI) to explore the effects of ketamine using portable EEG. In a double-blinded cross-over design, 30 male adults (mean age = 25.57, SD = 3.74) were administered racemic ketamine and compared against saline infusion as a control. Both task-driven (auditory oddball paradigm) and resting-state EEG were recorded. HOI were computed using advanced multivariate information theory tools, allowing us to quantify nonlinear statistical dependencies between all possible electrode combinations. Ketamine induced an increase in redundancy in brain dynamics (copies of the same information that can be retrieved from 3 or more electrodes), most significantly in the alpha frequency band. Redundancy was more evident during resting state, associated with a shift in conscious states towards more dissociative tendencies. Furthermore, in the task-driven context (auditory oddball), the impact of ketamine on redundancy was more significant for predictable (standard stimuli) compared to deviant ones. Finally, associations were observed between ketamine's HOI and experiences of derealization. Ketamine appears to increase redundancy and HOI across psychometric measures, suggesting these effects are correlated with alterations in consciousness towards dissociation. In comparisons with event-related potential (ERP) or standard functional connectivity metrics, HOI represent an innovative method to combine all signal spatial interactions obtained from low-density dry EEG in drug interventions, as it is the only approach that exploits all possible combinations between electrodes. This research emphasizes the potential of complexity measures coupled with portable EEG devices in monitoring shifts in consciousness, especially when paired with low-density configurations, paving the way for better understanding and monitoring of pharmacological-induced changes.

Unravelling consciousness and brain function through the lens of time, space, and information

Disentangling how cognitive functions emerge from the interplay of brain dynamics and network architecture is among the major challenges that neuroscientists face. Pharmacological and pathological perturbations of consciousness provide a lens to investigate these complex challenges. Here, we review how recent advances about consciousness and the brain's functional organisation have been driven by a common denominator: decomposing brain function into fundamental constituents of time, space, and information. Whereas unconsciousness increases structure-function coupling across scales, psychedelics may decouple brain function from structure. Convergent effects also emerge: anaesthetics, psychedelics, and disorders of consciousness can exhibit similar reconfigurations of the brain's unimodal-transmodal functional axis. Decomposition approaches reveal the potential to translate discoveries across species, with computational modelling providing a path towards mechanistic integration.

Identifying Diagnostic Biomarkers for Autism Spectrum Disorder From Higher-order Interactions Using the PED Algorithm

In the field of neuroimaging, more studies of abnormalities in brain regions of the autism spectrum disorder (ASD) usually focused on two brain regions connected, and less on abnormalities of higher-order interactions of brain regions. To explore the complex relationships of brain regions, we used the partial entropy decomposition (PED) algorithm to capture higher-order interactions by computing the higher-order dependencies of all three brain regions (triads). We proposed a method for examining the effect of individual brain regions on triads based on the PED and surrogate tests. The key triads were discovered by analyzing the effects. Further, the hypergraph modularity maximization algorithm revealed the higher-order brain structures, of which the link between right thalamus and left thalamus in ASD was more loose compared with the typical control (TC). Redundant key triad (left cerebellum crus 1 and left precuneus and right inferior occipital gyrus) exhibited a discernible attenuation in interaction in ASD, while the synergistic key triad (right cerebellum crus 1 and left postcentral gyrus and left lingual gyrus) indicated a notable decline. The results of classification model further confirmed the potential of the key triads as diagnostic biomarkers.

Changes in high-order interaction measures of synergy and redundancy during non-ordinary states of consciousness induced by meditation, hypnosis, and auto-induced cognitive trance

High-order interactions are required across brain regions to accomplish specific cognitive functions. These functional interdependencies are reflected by synergistic information that can be obtained by combining the information from all the sources considered and redundant information (i.e., common information provided by all the sources). However, electroencephalogram (EEG) functional connectivity is limited to pairwise interactions thereby precluding the estimation of high-order interactions. In this multicentric study, we used measures of synergistic and redundant information to study in parallel the high-order interactions between five EEG electrodes during three non-ordinary states of consciousness (NSCs): Rajyoga meditation (RM), hypnosis, and auto-induced cognitive trance (AICT). We analyzed EEG data from 22 long-term Rajyoga meditators, nine volunteers undergoing hypnosis, and 21 practitioners of AICT. We here report the within-group changes in synergy and redundancy for each NSC in comparison with their respective baseline. During RM, synergy increased at the whole brain level in the delta and theta bands. Redundancy decreased in frontal, right central, and posterior electrodes in delta, and frontal, central, and posterior electrodes in beta1 and beta2 bands. During hypnosis, synergy decreased in mid-frontal, temporal, and mid-centro-parietal electrodes in the delta band. The decrease was also observed in the beta2 band in the left frontal and right parietal electrodes. During AICT, synergy decreased in delta and theta bands in left-frontal, right-frontocentral, and posterior electrodes. The decrease was also observed at the whole brain level in the alpha band. However, redundancy changes during hypnosis and AICT were not significant. The subjective reports of absorption and dissociation during hypnosis and AICT, as well as the mystical experience questionnaires during AICT, showed no correlation with the high-order measures. The proposed study is the first exploratory attempt to utilize the concepts of synergy and redundancy in NSCs. The differences in synergy and redundancy during different NSCs warrant further studies to relate the extracted measures with the phenomenology of the NSCs.

Evolving higher-order synergies reveals a trade-off between stability and information-integration capacity in complex systems

There has recently been an explosion of interest in how "higher-order" structures emerge in complex systems comprised of many interacting elements (often called "synergistic" information). This "emergent" organization has been found in a variety of natural and artificial systems, although at present, the field lacks a unified understanding of what the consequences of higher-order synergies and redundancies are for systems under study. Typical research treats the presence (or absence) of synergistic information as a dependent variable and report changes in the level of synergy in response to some change in the system. Here, we attempt to flip the script: rather than treating higher-order information as a dependent variable, we use evolutionary optimization to evolve boolean networks with significant higher-order redundancies, synergies, or statistical complexity. We then analyze these evolved populations of networks using established tools for characterizing discrete dynamics: the number of attractors, the average transient length, and the Derrida coefficient. We also assess the capacity of the systems to integrate information. We find that high-synergy systems are unstable and chaotic, but with a high capacity to integrate information. In contrast, evolved redundant systems are extremely stable, but have negligible capacity to integrate information. Finally, the complex systems that balance integration and segregation (known as Tononi-Sporns-Edelman complexity) show features of both chaosticity and stability, with a greater capacity to integrate information than the redundant systems while being more stable than the random and synergistic systems. We conclude that there may be a fundamental trade-off between the robustness of a system's dynamics and its capacity to integrate information (which inherently requires flexibility and sensitivity) and that certain kinds of complexity naturally balance this trade-off.

Modeling the neurocognitive dynamics of language across the lifespan

Healthy aging is associated with a heterogeneous decline across cognitive functions, typically observed between language comprehension and language production (LP). Examining resting-state fMRI and neuropsychological data from 628 healthy adults (age 18-88) from the CamCAN cohort, we performed state-of-the-art graph theoretical analysis to uncover the neural mechanisms underlying this variability. At the cognitive level, our findings suggest that LP is not an isolated function but is modulated throughout the lifespan by the extent of inter-cognitive synergy between semantic and domain-general processes. At the cerebral level, we show that default mode network (DMN) suppression coupled with fronto-parietal network (FPN) integration is the way for the brain to compensate for the effects of dedifferentiation at a minimal cost, efficiently mitigating the age-related decline in LP. Relatedly, reduced DMN suppression in midlife could compromise the ability to manage the cost of FPN integration. This may prompt older adults to adopt a more cost-efficient compensatory strategy that maintains global homeostasis at the expense of LP performances. Taken together, we propose that midlife represents a critical neurocognitive juncture that signifies the onset of LP decline, as older adults gradually lose control over semantic representations. We summarize our findings in a novel synergistic, economical, nonlinear, emergent, cognitive aging model, integrating connectomic and cognitive dimensions within a complex system perspective.

Information decomposition and the informational architecture of the brain

To explain how the brain orchestrates information-processing for cognition, we must understand information itself. Importantly, information is not a monolithic entity. Information decomposition techniques provide a way to split information into its constituent elements: unique, redundant, and synergistic information. We review how disentangling synergistic and redundant interactions is redefining our understanding of integrative brain function and its neural organisation. To explain how the brain navigates the trade-offs between redundancy and synergy, we review converging evidence integrating the structural, molecular, and functional underpinnings of synergy and redundancy; their roles in cognition and computation; and how they might arise over evolution and development. Overall, disentangling synergistic and redundant information provides a guiding principle for understanding the informational architecture of the brain and cognition.

A Measure of Synergy Based on Union Information

The partial information decomposition (PID) framework is concerned with decomposing the information that a set of (two or more) random variables (the sources) has about another variable (the target) into three types of information: unique, redundant, and synergistic. Classical information theory alone does not provide a unique way to decompose information in this manner and additional assumptions have to be made. One often overlooked way to achieve this decomposition is using a so-called measure of union information-which quantifies the information that is present in at least one of the sources-from which a synergy measure stems. In this paper, we introduce a new measure of union information based on adopting a communication channel perspective, compare it with existing measures, and study some of its properties. We also include a comprehensive critical review of characterizations of union information and synergy measures that have been proposed in the literature.

Information synergy maximizes the growth rate of heterogeneous groups

Collective action and group formation are fundamental behaviors among both organisms cooperating to maximize their fitness and people forming socioeconomic organizations. Researchers have extensively explored social interaction structures via game theory and homophilic linkages, such as kin selection and scalar stress, to understand emergent cooperation in complex systems. However, we still lack a general theory capable of predicting how agents benefit from heterogeneous preferences, joint information, or skill complementarities in statistical environments. Here, we derive general statistical dynamics for the origin of cooperation based on the management of resources and pooled information. Specifically, we show how groups that optimally combine complementary agent knowledge about resources in statistical environments maximize their growth rate. We show that these advantages are quantified by the information synergy embedded in the conditional probability of environmental states given agents' signals, such that groups with a greater diversity of signals maximize their collective information. It follows that, when constraints are placed on group formation, agents must intelligently select with whom they cooperate to maximize the synergy available to their own signal. Our results show how the general properties of information underlie the optimal collective formation and dynamics of groups of heterogeneous agents across social and biological phenomena.

Behavioural relevance of redundant and synergistic stimulus information between functionally connected neurons in mouse auditory cortex

Measures of functional connectivity have played a central role in advancing our understanding of how information is transmitted and processed within the brain. Traditionally, these studies have focused on identifying redundant functional connectivity, which involves determining when activity is similar across different sites or neurons. However, recent research has highlighted the importance of also identifying synergistic connectivity-that is, connectivity that gives rise to information not contained in either site or neuron alone. Here, we measured redundant and synergistic functional connectivity between neurons in the mouse primary auditory cortex during a sound discrimination task. Specifically, we measured directed functional connectivity between neurons simultaneously recorded with calcium imaging. We used Granger Causality as a functional connectivity measure. We then used Partial Information Decomposition to quantify the amount of redundant and synergistic information about the presented sound that is carried by functionally connected or functionally unconnected pairs of neurons. We found that functionally connected pairs present proportionally more redundant information and proportionally less synergistic information about sound than unconnected pairs, suggesting that their functional connectivity is primarily redundant. Further, synergy and redundancy coexisted both when mice made correct or incorrect perceptual discriminations. However, redundancy was much higher (both in absolute terms and in proportion to the total information available in neuron pairs) in correct behavioural choices compared to incorrect ones, whereas synergy was higher in absolute terms but lower in relative terms in correct than in incorrect behavioural choices. Moreover, the proportion of redundancy reliably predicted perceptual discriminations, with the proportion of synergy adding no extra predictive power. These results suggest a crucial contribution of redundancy to correct perceptual discriminations, possibly due to the advantage it offers for information propagation, and also suggest a role of synergy in enhancing information level during correct discriminations.

Statistical Approaches to Identify Pairwise and High-Order Brain Functional Connectivity Signatures on a Single-Subject Basis

Keeping up with the shift towards personalized neuroscience essentially requires the derivation of meaningful insights from individual brain signal recordings by analyzing the descriptive indexes of physio-pathological states through statistical methods that prioritize subject-specific differences under varying experimental conditions. Within this framework, the current study presents a methodology for assessing the value of the single-subject fingerprints of brain functional connectivity, assessed both by standard pairwise and novel high-order measures. Functional connectivity networks, which investigate the inter-relationships between pairs of brain regions, have long been a valuable tool for modeling the brain as a complex system. However, their usefulness is limited by their inability to detect high-order dependencies beyond pairwise correlations. In this study, by leveraging multivariate information theory, we confirm recent evidence suggesting that the brain contains a plethora of high-order, synergistic subsystems that would go unnoticed using a pairwise graph structure. The significance and variations across different conditions of functional pairwise and high-order interactions (HOIs) between groups of brain signals are statistically verified on an individual level through the utilization of surrogate and bootstrap data analyses. The approach is illustrated on the single-subject recordings of resting-state functional magnetic resonance imaging (rest-fMRI) signals acquired using a pediatric patient with hepatic encephalopathy associated with a portosystemic shunt and undergoing liver vascular shunt correction. Our results show that (i) the proposed single-subject analysis may have remarkable clinical relevance for subject-specific investigations and treatment planning, and (ii) the possibility of investigating brain connectivity and its post-treatment functional developments at a high-order level may be essential to fully capture the complexity and modalities of the recovery.

Partial entropy decomposition reveals higher-order information structures in human brain activity

The standard approach to modeling the human brain as a complex system is with a network, where the basic unit of interaction is a pairwise link between two brain regions. While powerful, this approach is limited by the inability to assess higher-order interactions involving three or more elements directly. In this work, we explore a method for capturing higher-order dependencies in multivariate data: the partial entropy decomposition (PED). Our approach decomposes the joint entropy of the whole system into a set of nonnegative atoms that describe the redundant, unique, and synergistic interactions that compose the system's structure. PED gives insight into the mathematics of functional connectivity and its limitation. When applied to resting-state fMRI data, we find robust evidence of higher-order synergies that are largely invisible to standard functional connectivity analyses. Our approach can also be localized in time, allowing a frame-by-frame analysis of how the distributions of redundancies and synergies change over the course of a recording. We find that different ensembles of regions can transiently change from being redundancy-dominated to synergy-dominated and that the temporal pattern is structured in time. These results provide strong evidence that there exists a large space of unexplored structures in human brain data that have been largely missed by a focus on bivariate network connectivity models. This synergistic structure is dynamic in time and likely will illuminate interesting links between brain and behavior. Beyond brain-specific application, the PED provides a very general approach for understanding higher-order structures in a variety of complex systems.

Neural complexity through a nonextensive statistical-mechanical approach of human electroencephalograms

The brain is a complex system whose understanding enables potentially deeper approaches to mental phenomena. Dynamics of wide classes of complex systems have been satisfactorily described within q-statistics, a current generalization of Boltzmann-Gibbs (BG) statistics. Here, we study human electroencephalograms of typical human adults (EEG), very specifically their inter-occurrence times across an arbitrarily chosen threshold of the signal (observed, for instance, at the midparietal location in scalp). The distributions of these inter-occurrence times differ from those usually emerging within BG statistical mechanics. They are instead well approached within the q-statistical theory, based on non-additive entropies characterized by the index q. The present method points towards a suitable tool for quantitatively accessing brain complexity, thus potentially opening useful studies of the properties of both typical and altered brain physiology.