Brain networks engaged in audiovisual integration during speech perception revealed by persistent homology-based network filtration

Visuo-auditory stimuli with semantic, temporal and spatial congruence for a P300-based BCI: An exploratory test with an ALS patient in a completely locked-in state

BACKGROUND

Brain-computer interfaces (BCIs) are a promising tool for communication with completely locked-in state (CLIS) patients. Despite the great efforts already made by the BCI research community, the cases of success are still very few, very exploratory, limited in time, and based on simple 'yes/no' paradigms.

NEW METHOD

A P300-based BCI is proposed comparing two conditions, one corresponding to purely spatial auditory stimuli (AU-S) and the other corresponding to hybrid visual and spatial auditory stimuli (HVA-S). In the HVA-S condition, there is a semantic, temporal, and spatial congruence between visual and auditory stimuli. The stimuli comprise a lexicon of 7 written and spoken words. Spatial sounds are generated through the head-related transfer function. Given the good results obtained with 10 able-bodied participants, we investigated whether a patient entering CLIS could use the proposed BCI.

RESULTS

The able-bodied group achieved 71.3 % and 90.5 % online classification accuracy for the auditory and hybrid BCIs respectively, while the patient achieved 30 % and chance level accuracies, for the same conditions. Notwithstanding, the patient's event-related potentials (ERPs) showed statistical discrimination between target and non-target events in different time windows.

COMPARISON WITH EXISTING METHODS

The results of the control group compare favorably with the state-of-the-art, considering a 7-class BCI controlled visual-covertly and with auditory stimuli. The integration of visual and auditory stimuli has not been tested before with CLIS patients.

CONCLUSIONS

The semantic, temporal, and spatial congruence of the stimuli increased the performance of the control group, but not of the CLIS patient, which can be due to impaired attention and cognitive function. The patient's unique ERP patterns make interpretation difficult, requiring further tests/paradigms to decouple patients' responses at different levels (reflexive, perceptual, cognitive). The ERPs discrimination found indicates that a simplification of the proposed approaches may be feasible.

From mathematics to medicine: A practical primer on topological data analysis (TDA) and the development of related analytic tools for the functional discovery of latent structure in fMRI data

fMRI is the preeminent method for collecting signals from the human brain in vivo, for using these signals in the service of functional discovery, and relating these discoveries to anatomical structure. Numerous computational and mathematical techniques have been deployed to extract information from the fMRI signal. Yet, the application of Topological Data Analyses (TDA) remain limited to certain sub-areas such as connectomics (that is, with summarized versions of fMRI data). While connectomics is a natural and important area of application of TDA, applications of TDA in the service of extracting structure from the (non-summarized) fMRI data itself are heretofore nonexistent. “Structure” within fMRI data is determined by dynamic fluctuations in spatially distributed signals over time, and TDA is well positioned to help researchers better characterize mass dynamics of the signal by rigorously capturing shape within it. To accurately motivate this idea, we a) survey an established method in TDA (“persistent homology”) to reveal and describe how complex structures can be extracted from data sets generally, and b) describe how persistent homology can be applied specifically to fMRI data. We provide explanations for some of the mathematical underpinnings of TDA (with expository figures), building ideas in the following sequence: a) fMRI researchers can and should use TDA to extract structure from their data; b) this extraction serves an important role in the endeavor of functional discovery, and c) TDA approaches can complement other established approaches toward fMRI analyses (for which we provide examples). We also provide detailed applications of TDA to fMRI data collected using established paradigms, and offer our software pipeline for readers interested in emulating our methods. This working overview is both an inter-disciplinary synthesis of ideas (to draw researchers in TDA and fMRI toward each other) and a detailed description of methods that can motivate collaborative research.

Evaluating the phase dynamics of coupled oscillators via time-variant topological features

By characterizing the phase dynamics in coupled oscillators, we gain insights into the fundamental phenomena of complex systems. The collective dynamics in oscillatory systems are often described by order parameters, which are insufficient for identifying more specific behaviors. To improve this situation, we propose a topological approach that constructs the quantitative features describing the phase evolution of oscillators. Here, the phase data are mapped into a high-dimensional space at each time, and the topological features describing the shape of the data are subsequently extracted from the mapped points. These features are extended to time-variant topological features by adding the evolution time as an extra dimension in the topological feature space. The time-variant features provide crucial insights into the evolution of phase dynamics. Combining these features with the kernel method, we characterize the multiclustered synchronized dynamics during the early evolution stages. Finally, we demonstrate that our method can qualitatively explain chimera states. The experimental results confirmed the superiority of our method over those based on order parameters, especially when the available data are limited to the early-stage dynamics.

Meta-Analyses Support a Taxonomic Model for Representations of Different Categories of Audio-Visual Interaction Events in the Human Brain

Our ability to perceive meaningful action events involving objects, people, and other animate agents is characterized in part by an interplay of visual and auditory sensory processing and their cross-modal interactions. However, this multisensory ability can be altered or dysfunctional in some hearing and sighted individuals, and in some clinical populations. The present meta-analysis sought to test current hypotheses regarding neurobiological architectures that may mediate audio-visual multisensory processing. Reported coordinates from 82 neuroimaging studies (137 experiments) that revealed some form of audio-visual interaction in discrete brain regions were compiled, converted to a common coordinate space, and then organized along specific categorical dimensions to generate activation likelihood estimate (ALE) brain maps and various contrasts of those derived maps. The results revealed brain regions (cortical "hubs") preferentially involved in multisensory processing along different stimulus category dimensions, including 1) living versus nonliving audio-visual events, 2) audio-visual events involving vocalizations versus actions by living sources, 3) emotionally valent events, and 4) dynamic-visual versus static-visual audio-visual stimuli. These meta-analysis results are discussed in the context of neurocomputational theories of semantic knowledge representations and perception, and the brain volumes of interest are available for download to facilitate data interpretation for future neuroimaging studies.

Development of the Mechanisms Underlying Audiovisual Speech Perception Benefit

The natural environments in which infants and children learn speech and language are noisy and multimodal. Adults rely on the multimodal nature of speech to compensate for noisy environments during speech communication. Multiple mechanisms underlie mature audiovisual benefit to speech perception, including reduced uncertainty as to when auditory speech will occur, use of correlations between the amplitude envelope of auditory and visual signals in fluent speech, and use of visual phonetic knowledge for lexical access. This paper reviews evidence regarding infants' and children's use of temporal and phonetic mechanisms in audiovisual speech perception benefit. The ability to use temporal cues for audiovisual speech perception benefit emerges in infancy. Although infants are sensitive to the correspondence between auditory and visual phonetic cues, the ability to use this correspondence for audiovisual benefit may not emerge until age four. A more cohesive account of the development of audiovisual speech perception may follow from a more thorough understanding of the development of sensitivity to and use of various temporal and phonetic cues.

Multi-dimensional persistent feature analysis identifies connectivity patterns of resting-state brain networks in Alzheimer's disease

OBJECTIVE

The characterization of functional brain network is crucial to understanding the neural mechanisms associated with Alzheimer's disease (AD) and mild cognitive impairment (MCI). Some studies have shown that graph theoretical analysis could reveal changes of the disease-related brain networks by thresholding edge weights. But the choice of threshold depends on ambiguous cognitive conditions, which leads to the lack of interpretability. Recently, persistent homology (PH) was proposed to record the persistence of topological features of networks across every possible thresholds, reporting a higher sensitivity than graph theoretical features in detecting network-level biomarkers of AD. However, most research on PH focused on 0-dimensional features (persistence of connected components) reflecting the intrinsic topology of the brain network, rather than 1-dimensional features (persistence of cycles) with an interesting neurobiological communication pattern. Our aim is to explore the multi-dimensional persistent features of brain networks in the AD and MCI patients, and further to capture valuable brain connectivity patterns.

APPROACH

We characterized the change rate of the connected component numbers across graph filtration using the functional derivative curves, and examined the persistence landscapes that vectorize the persistence of cycle structures. After that, the multi-dimensional persistent features were validated in disease identification using a K-nearest neighbor algorithm. Furthermore, a connectivity pattern mining framework was designed to capture the disease-specific brain structures.

MAIN RESULTS

We found that the multi-dimensional persistent features can identify statistical group differences, quantify subject-level distances, and yield disease-specific connectivity patterns. Relatively high classification accuracies were received when compared with graph theoretical features.

SIGNIFICANCE

This work represents a conceptual bridge linking complex brain network analysis and computational topology. Our results can be beneficial for providing a complementary objective opinion to the clinical diagnosis of neurodegenerative diseases.

Maturational delay and asymmetric information flow of brain connectivity in SHR model of ADHD revealed by topological analysis of metabolic networks

Attention-deficit hyperactivity disorder (ADHD) is a complex brain development disorder characterized by hyperactivity/impulsivity and inattention. A major hypothesis of ADHD is a lag of maturation, which is supported mainly by anatomical studies evaluating cortical thickness. Here, we analyzed changes of topological characteristics of whole-brain metabolic connectivity in twelve SHR rats selected as ADHD-model rats by confirming behavior abnormalities using the marble burying test, open field test, and delay discounting task and 12 Wistar Kyoto rats as the control group, across development from 4 weeks old (childhood) and 6 weeks old (entry of puberty). A topological approach based on graph filtrations revealed a lag in the strengthening of limbic-cortical/subcortical connections in ADHD-model rats. This in turn related to impaired modularization of memory and reward-motivation associated regions. Using mathematical network analysis techniques such as single linkage hierarchical clustering and volume entropy, we observed left-lateralized connectivity in the ADHD-model rats at 6 weeks old. Our findings supported the maturational delay of metabolic connectivity in the SHR model of ADHD, and also suggested the possibility of impaired and compensative reconfiguration of information flow over the brain network.

Topological exploration of artificial neuronal network dynamics

One of the paramount challenges in neuroscience is to understand the dynamics of individual neurons and how they give rise to network dynamics when interconnected. Historically, researchers have resorted to graph theory, statistics, and statistical mechanics to describe the spatiotemporal structure of such network dynamics. Our novel approach employs tools from algebraic topology to characterize the global properties of network structure and dynamics. We propose a method based on persistent homology to automatically classify network dynamics using topological features of spaces built from various spike train distances. We investigate the efficacy of our method by simulating activity in three small artificial neural networks with different sets of parameters, giving rise to dynamics that can be classified into four regimes. We then compute three measures of spike train similarity and use persistent homology to extract topological features that are fundamentally different from those used in traditional methods. Our results show that a machine learning classifier trained on these features can accurately predict the regime of the network it was trained on and also generalize to other networks that were not presented during training. Moreover, we demonstrate that using features extracted from multiple spike train distances systematically improves the performance of our method.

Clinical Personal Connectomics Using Hybrid PET/MRI

Brain connectivity can now be studied with topological analysis using persistent homology. It overcame the arbitrariness of thresholding to make binary graphs for comparison between disease and normal control groups. Resting-state fMRI can yield personal interregional brain connectivity based on perfusion signal on MRI on individual subject bases and FDG PET produces the topography of glucose metabolism. Assuming metabolism perfusion coupling and disregarding the slight difference of representing time of metabolism (before image acquisition) and representing time of perfusion (during image acquisition), topography of brain metabolism on FDG PET and topologically analyzed brain connectivity on resting-state fMRI might be related to yield personal connectomics of individual subjects and even individual patients. The work of association of FDG PET/resting-state fMRI is yet to be warranted; however, the statistics behind the group comparison of connectivity on FDG PET or resting-state MRI was already developed. Before going further into the connectomics construction using directed weighted brain graphs of FDG PET or resting-state fMRI, I detailed in this review the plausibility of using hybrid PET/MRI to enable the interpretation of personal connectomics which can lead to the clinical use of brain connectivity in the near future.

Disrupted Resting State Network of Fibromyalgia in Theta frequency

Fibromyalgia (FM), chronic widespread pain, exhibits spontaneous pain without external stimuli and is associated with altered brain activities during resting state. To understand the topological features of brain network in FM, we employed persistent homology which is a multiple scale network modeling framework not requiring thresholding. Spontaneous magnetoencephalography (MEG) activity was recorded in 19 healthy controls (HCs) and 18 FM patients. Barcode, single linkage dendrogram and single linkage matrix were generated based on the proposed modeling framework. In theta band, the slope of decrease in the number of connected components in barcodes showed steeper in HC, suggesting FM patients had decreased global connectivity. FM patients had reduced connectivity within default mode network, between middle/inferior temporal gyrus and visual cortex. The longer pain duration was correlated with reduced connectivity between inferior temporal gyrus and visual cortex. Our findings demonstrated that the aberrant resting state network could be associated with dysfunction of sensory processing in chronic pain. The spontaneous nature of FM pain may accrue to disruption of resting state network.

Combination of Interaural Level and Time Difference in Azimuthal Sound Localization in Owls

A function of the auditory system is to accurately determine the location of a sound source. The main cues for sound location are interaural time (ITD) and level (ILD) differences. Humans use both ITD and ILD to determine the azimuth. Thus far, the conception of sound localization in barn owls was that their facial ruff and asymmetrical ears generate a two-dimensional grid of ITD for azimuth and ILD for elevation. We show that barn owls also use ILD for azimuthal sound localization when ITDs are ambiguous. For high-frequency narrowband sounds, midbrain neurons can signal multiple locations, leading to the perception of an auditory illusion called a phantom source. Owls respond to such an illusory percept by orienting toward it instead of the true source. Acoustical measurements close to the eardrum reveal a small ILD component that changes with azimuth, suggesting that ITD and ILD information could be combined to eliminate the illusion. Our behavioral data confirm that perception was robust against ambiguities if ITD and ILD information was combined. Electrophysiological recordings of ILD sensitivity in the owl’s midbrain support the behavioral findings indicating that rival brain hemispheres drive the decision to orient to either true or phantom sources. Thus, the basis for disambiguation, and reliable detection of sound source azimuth, relies on similar cues across species as similar response to combinations of ILD and narrowband ITD has been observed in humans.

Gating of memory encoding of time-delayed cross-frequency MEG networks revealed by graph filtration based on persistent homology

To explain gating of memory encoding, magnetoencephalography (MEG) was analyzed over multi-regional network of negative correlations between alpha band power during cue (cue-alpha) and gamma band power during item presentation (item-gamma) in Remember (R) and No-remember (NR) condition. Persistent homology with graph filtration on alpha-gamma correlation disclosed topological invariants to explain memory gating. Instruction compliance (R-hits minus NR-hits) was significantly related to negative coupling between the left superior occipital (cue-alpha) and the left dorsolateral superior frontal gyri (item-gamma) on permutation test, where the coupling was stronger in R than NR. In good memory performers (R-hits minus false alarm), the coupling was stronger in R than NR between the right posterior cingulate (cue-alpha) and the left fusiform gyri (item-gamma). Gating of memory encoding was dictated by inter-regional negative alpha-gamma coupling. Our graph filtration over MEG network revealed these inter-regional time-delayed cross-frequency connectivity serve gating of memory encoding.

Maturation of metabolic connectivity of the adolescent rat brain

Neuroimaging has been used to examine developmental changes of the brain. While PET studies revealed maturation-related changes, maturation of metabolic connectivity of the brain is not yet understood. Here, we show that rat brain metabolism is reconfigured to achieve long-distance connections with higher energy efficiency during maturation. Metabolism increased in anterior cerebrum and decreased in thalamus and cerebellum during maturation. When functional covariance patterns of PET images were examined, metabolic networks including default mode network (DMN) were extracted. Connectivity increased between the anterior and posterior parts of DMN and sensory-motor cortices during maturation. Energy efficiency, a ratio of connectivity strength to metabolism of a region, increased in medial prefrontal and retrosplenial cortices. Our data revealed that metabolic networks mature to increase metabolic connections and establish its efficiency between large-scale spatial components from childhood to early adulthood. Neurodevelopmental diseases might be understood by abnormal reconfiguration of metabolic connectivity and efficiency.

DOI:http://dx.doi.org/10.7554/eLife.11571.001

The brain consumes a great deal of a sugar called glucose, which is delivered to the brain through blood vessels. Active regions of the brain need more glucose, and so the brain has a metabolic network that controls when and where glucose is metabolized. Yet precisely how this metabolic network changes during brain development is not yet understood.

Choi et al. have now monitored the patterns of glucose metabolism in the brains of awake rats as they matured from 'childhood' to early adulthood. The experiments involved injecting the rats with radioactive glucose, and then using a technique called positron emission tomography (commonly known as 'PET scan') to monitor the metabolism of these radioactive sugar molecules in the animals’ brains.

Choi et al. showed that the patterns of glucose consumption in the brain shift drastically as the rats mature. Importantly, the findings showed that these shifts in glucose metabolism seem to support the activity of long distance connections that develop as the brain matures. The findings also showed that the increased long distance connections were energy efficient. The results suggest that these metabolic changes are likely a way of maintaining high-energy efficiency that is crucial for the brain to perform normally.

Finally, in addition to revealing the changes involved in normal brain development, these findings may have implications in neurological and psychiatric disorders in which the brain fails to achieve efficient metabolic networks as it matures.

DOI:http://dx.doi.org/10.7554/eLife.11571.002

Amplitude-modulated stimuli reveal auditory-visual interactions in brain activity and brain connectivity

The temporal congruence between auditory and visual signals coming from the same source can be a powerful means by which the brain integrates information from different senses. To investigate how the brain uses temporal information to integrate auditory and visual information from continuous yet unfamiliar stimuli, we used amplitude-modulated tones and size-modulated shapes with which we could manipulate the temporal congruence between the sensory signals. These signals were independently modulated at a slow or a fast rate. Participants were presented with auditory-only, visual-only, or auditory-visual (AV) trials in the fMRI scanner. On AV trials, the auditory and visual signal could have the same (AV congruent) or different modulation rates (AV incongruent). Using psychophysiological interaction analyses, we found that auditory regions showed increased functional connectivity predominantly with frontal regions for AV incongruent relative to AV congruent stimuli. We further found that superior temporal regions, shown previously to integrate auditory and visual signals, showed increased connectivity with frontal and parietal regions for the same contrast. Our findings provide evidence that both activity in a network of brain regions and their connectivity are important for AV integration, and help to bridge the gap between transient and familiar AV stimuli used in previous studies.