Identifying musical pieces from fMRI data using encoding and decoding models

Dimerization activates the Inversin complex in C. elegans

Genetic, colocalization, and biochemical studies suggest that the ankyrin repeat-containing proteins Inversin (INVS) and ANKS6 function with the NEK8 kinase to control tissue patterning and maintain organ physiology. It is unknown whether these three proteins assemble into a static "Inversin complex" or one that adopts multiple bioactive forms. Through characterization of hyperactive alleles in C. elegans , we discovered that the Inversin complex is activated by dimerization. Genome engineering of an RFP tag onto the nematode homologs of INVS (MLT-4) and NEK8 (NEKL-2) induced a gain-of-function, cyst-like phenotype that was suppressed by monomerization of the fluorescent tag. Stimulated dimerization of MLT-4 or NEKL-2 using optogenetics was sufficient to recapitulate the phenotype of a constitutively active Inversin complex. Further, dimerization of NEKL-2 bypassed a lethal MLT-4 mutant, demonstrating that the dimeric form is required for function. We propose that dynamic switching between at least two functionally distinct states-an active dimer and an inactive monomer-gates the output of the Inversin complex.

Lifespan analysis of repeat expression reveals age-dependent upregulation of HERV-K in the neurotypical human brain

DNA repetitive sequences (or repeats) comprise over 50% of the human genome and have a crucial regulatory role, specifically regulating transcription machinery. The human brain is the tissue with the highest detectable repeat expression and dysregulations on the repeat activity are related to several neurological and neurodegenerative disorders, as repeat-derived products can stimulate a pro-inflammatory response. Even so, it is unclear how repeat expression acts on the aging neurotypical brain. Here, we leverage a large postmortem transcriptome cohort spanning the human lifespan to assess global repeat expression in the neurotypical brain. We identified 21,696 differentially expressed repeats (DERs) that varied across seven age bins (Prenatal; 0-15; 16-29; 30-39; 40-49; 50-59; 60+) across the caudate nucleus (n=271), dorsolateral prefrontal cortex (n=304), and hippocampus (n=310). Interestingly, we found that long interspersed nuclear elements and long terminal repeats (LTRs) DERs were the most abundant repeat families when comparing infants to early adolescence (0-15) with older adults (60+). Of these differentially regulated LTRs, we identified 17 shared across all brain regions, including increased expression of HERV-K-int in older adult brains (60+). Co-expression analysis from each of the three brain regions also showed repeats from the HERV subfamily were intramodular hubs in its subnetworks. While we do not observe a strong global relationship between repeat expression and age, we identified HERV-K as a repeat signature associated with the aging neurotypical brain. Our study is the first global assessment of repeat expression in the neurotypical brain.

The binding of Mint/X11 PDZ domains to Ca V 2 calcium channels predates bilaterian animals

PDZ domain mediated interactions with voltage-gated calcium (Ca V ) channel C-termini play important roles in localizing and compartmentalizing membrane Ca 2+ signaling. The first such interaction discovered was between the neuronal multi-domain protein Mint-1, and the presynaptc calcium channel Ca V 2.2 in mammals. Although the physiological significance of this interaction is unclear, its occurrence in vertebrates and bilaterian invertebrates suggests important and conserved functions. In this study, we explore the evolutionary origins of Mint and its interaction with Ca V 2 channels. Phylogenetic and structural in silico analyses revealed that Mint is an animal-specific gene, like Ca V 2 channels, which bears a highly divergent N-terminus but strongly conserved C-terminus comprised of a phosphotyrosine binding domain, two tandem PDZ domains (PDZ-1 and PDZ-2), and a C-terminal auto-inhibitory element that binds and inhibits PDZ-1. Also deeply conserved are other Mint interacting proteins, namely amyloid precursor and related proteins, presenilins, neurexin, as well as CASK and Veli which form a tripartite complex with Mint in bilaterians. Through yeast 2-hybrid and bacterial 2-hybrid experiments, we show that Mint and Ca V 2 channels from cnidarians and placozoans interact in vitro , and in situ hybridization revealed co-expression of corresponding transcripts in dissociated neurons from the cnidarian Nematostella vectensis . Unexpectedly, the Mint orthologue from the ctenophore Hormiphora californiensis was able to strongly bind the divergent C-terminal ligands of cnidarian and placozoan Ca V 2 channels, despite neither the ctenophore Mint, nor the placozoan and cnidarian orthologues, binding the ctenophore Ca V 2 channel C-terminus. Altogether, our analyses provide a model for the emergence of this interaction in early animals first via adoption of a PDZ ligand by Ca V 2 channels, followed by sequence changes in the ligand that caused a modality switch for binding to Mint.

Music can be reconstructed from human auditory cortex activity using nonlinear decoding models

Music is core to human experience, yet the precise neural dynamics underlying music perception remain unknown. We analyzed a unique intracranial electroencephalography (iEEG) dataset of 29 patients who listened to a Pink Floyd song and applied a stimulus reconstruction approach previously used in the speech domain. We successfully reconstructed a recognizable song from direct neural recordings and quantified the impact of different factors on decoding accuracy. Combining encoding and decoding analyses, we found a right-hemisphere dominance for music perception with a primary role of the superior temporal gyrus (STG), evidenced a new STG subregion tuned to musical rhythm, and defined an anterior-posterior STG organization exhibiting sustained and onset responses to musical elements. Our findings show the feasibility of applying predictive modeling on short datasets acquired in single patients, paving the way for adding musical elements to brain-computer interface (BCI) applications.

What multiplexing means for the interpretation of functional MRI data

Despite technology advances that have enabled routine acquisition of functional MRI data with sub-millimeter resolution, the inferences that cognitive neuroscientists must make to link fMRI data to behavior are complicated. Thus, a single dataset subjected to different analyses can be interpreted in different ways. This article presents two optical analogies that can be useful for framing fMRI analyses in a way that allows for multiple interpretations of fMRI data to be valid simultaneously without undermining each other. The first is reflection: when an object is reflected in a mirrored surface, it appears as if the reflected object is sharing space with the mirrored object, but of course it is not. This analogy can be a good guide for interpreting the fMRI signal, since even at sub-millimeter resolutions the signal is determined by a mixture of local and long-range neural computations. The second is refraction. If we view an object through a multi-faceted prism or gemstone, our view will change-sometimes dramatically-depending on our viewing angle. In the same way, interpretation of fMRI data (inference of underlying neuronal activity) can and should be different depending on the analysis approach. Rather than representing a weakness of the methodology, or the superiority of one approach over the other (for example, simple regression analysis versus multi-voxel pattern analysis), this is an expected consequence of how information is multiplexed in the neural networks of the brain: multiple streams of information are simultaneously present in each location. The fact that any one analysis typically shows only one view of the data also puts some parentheses around fMRI practitioners' constant search for ground truth against which to compare their data. By holding our interpretations lightly and understanding that many interpretations of the data can all be true at the same time, we do a better job of preparing ourselves to appreciate, and eventually understand, the complexity of the brain and the behavior it produces.

Neural decoding of music from the EEG

Neural decoding models can be used to decode neural representations of visual, acoustic, or semantic information. Recent studies have demonstrated neural decoders that are able to decode accoustic information from a variety of neural signal types including electrocortiography (ECoG) and the electroencephalogram (EEG). In this study we explore how functional magnetic resonance imaging (fMRI) can be combined with EEG to develop an accoustic decoder. Specifically, we first used a joint EEG-fMRI paradigm to record brain activity while participants listened to music. We then used fMRI-informed EEG source localisation and a bi-directional long-term short term deep learning network to first extract neural information from the EEG related to music listening and then to decode and reconstruct the individual pieces of music an individual was listening to. We further validated our decoding model by evaluating its performance on a separate dataset of EEG-only recordings. We were able to reconstruct music, via our fMRI-informed EEG source analysis approach, with a mean rank accuracy of 71.8% ([Formula: see text], [Formula: see text]). Using only EEG data, without participant specific fMRI-informed source analysis, we were able to identify the music a participant was listening to with a mean rank accuracy of 59.2% ([Formula: see text], [Formula: see text]). This demonstrates that our decoding model may use fMRI-informed source analysis to aid EEG based decoding and reconstruction of acoustic information from brain activity and makes a step towards building EEG-based neural decoders for other complex information domains such as other acoustic, visual, or semantic information.

On the encoding of natural music in computational models and human brains

This article discusses recent developments and advances in the neuroscience of music to understand the nature of musical emotion. In particular, it highlights how system identification techniques and computational models of music have advanced our understanding of how the human brain processes the textures and structures of music and how the processed information evokes emotions. Musical models relate physical properties of stimuli to internal representations called features, and predictive models relate features to neural or behavioral responses and test their predictions against independent unseen data. The new frameworks do not require orthogonalized stimuli in controlled experiments to establish reproducible knowledge, which has opened up a new wave of naturalistic neuroscience. The current review focuses on how this trend has transformed the domain of the neuroscience of music.

Neural decoding of semantic concepts: A systematic literature review

Objective Semantic concepts are coherent entities within our minds. They underpin our thought processes and are a part of the basis for our understanding of the world. Modern neuroscience research is increasingly exploring how individual semantic concepts are encoded within our brains and a number of studies are beginning to reveal key patterns of neural activity that underpin specific concepts. Building upon this basic understanding of the process of semantic neural encoding, neural engineers are beginning to explore tools and methods for semantic decoding: identifying which semantic concepts an individual is focused on at a given moment in time from recordings of their neural activity. In this paper we review the current literature on semantic neural decoding. Approach We conducted this review according to the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) guidelines. Specifically, we assess the eligibility of published peer-reviewed reports via a search of PubMed and Google Scholar. We identify a total of 74 studies in which semantic neural decoding is used to attempt to identify individual semantic concepts from neural activity. Results Our review reveals how modern neuroscientific tools have been developed to allow decoding of individual concepts from a range of neuroimaging modalities. We discuss specific neuroimaging methods, experimental designs, and machine learning pipelines that are employed to aid the decoding of semantic concepts. We quantify the efficacy of semantic decoders by measuring information transfer rates. We also discuss current challenges presented by this research area and present some possible solutions. Finally, we discuss some possible emerging and speculative future directions for this research area. Significance Semantic decoding is a rapidly growing area of research. However, despite its increasingly widespread popularity and use in neuroscientific research this is the first literature review focusing on this topic across neuroimaging modalities and with a focus on quantifying the efficacy of semantic decoders.

Distillation of Regional Activity Reveals Hidden Content of Neural Information in Visual Processing

Multivoxel pattern analysis (MVPA) has become a standard tool for decoding mental states from brain activity patterns. Recent studies have demonstrated that MVPA can be applied to decode activity patterns of a certain region from those of the other regions. By applying a similar region-to-region decoding technique, we examined whether the information represented in the visual areas can be explained by those represented in the other visual areas. We first predicted the brain activity patterns of an area on the visual pathway from the others, then subtracted the predicted patterns from their originals. Subsequently, the visual features were derived from these residuals. During the visual perception task, the elimination of the top-down signals enhanced the simple visual features represented in the early visual cortices. By contrast, the elimination of the bottom-up signals enhanced the complex visual features represented in the higher visual cortices. The directions of such modulation effects varied across visual perception/imagery tasks, indicating that the information flow across the visual cortices is dynamically altered, reflecting the contents of visual processing. These results demonstrated that the distillation approach is a useful tool to estimate the hidden content of information conveyed across brain regions.

Neocortical substrates of feelings evoked with music in the ACC, insula, and somatosensory cortex

Neurobiological models of emotion focus traditionally on limbic/paralimbic regions as neural substrates of emotion generation, and insular cortex (in conjunction with isocortical anterior cingulate cortex, ACC) as the neural substrate of feelings. An emerging view, however, highlights the importance of isocortical regions beyond insula and ACC for the subjective feeling of emotions. We used music to evoke feelings of joy and fear, and multivariate pattern analysis (MVPA) to decode representations of feeling states in functional magnetic resonance (fMRI) data of n = 24 participants. Most of the brain regions providing information about feeling representations were neocortical regions. These included, in addition to granular insula and cingulate cortex, primary and secondary somatosensory cortex, premotor cortex, frontal operculum, and auditory cortex. The multivoxel activity patterns corresponding to feeling representations emerged within a few seconds, gained in strength with increasing stimulus duration, and replicated results of a hypothesis-generating decoding analysis from an independent experiment. Our results indicate that several neocortical regions (including insula, cingulate, somatosensory and premotor cortices) are important for the generation and modulation of feeling states. We propose that secondary somatosensory cortex, which covers the parietal operculum and encroaches on the posterior insula, is of particular importance for the encoding of emotion percepts, i.e., preverbal representations of subjective feeling.

Correspondence of categorical and feature-based representations of music in the human brain

INTRODUCTION

Humans tend to categorize auditory stimuli into discrete classes, such as animal species, language, musical instrument, and music genre. Of these, music genre is a frequently used dimension of human music preference and is determined based on the categorization of complex auditory stimuli. Neuroimaging studies have reported that the superior temporal gyrus (STG) is involved in response to general music-related features. However, there is considerable uncertainty over how discrete music categories are represented in the brain and which acoustic features are more suited for explaining such representations.

METHODS

We used a total of 540 music clips to examine comprehensive cortical representations and the functional organization of music genre categories. For this purpose, we applied a voxel-wise modeling approach to music-evoked brain activity measured using functional magnetic resonance imaging. In addition, we introduced a novel technique for feature-brain similarity analysis and assessed how discrete music categories are represented based on the cortical response pattern to acoustic features.

RESULTS

Our findings indicated distinct cortical organizations for different music genres in the bilateral STG, and they revealed representational relationships between different music genres. On comparing different acoustic feature models, we found that these representations of music genres could be explained largely by a biologically plausible spectro-temporal modulation-transfer function model.

CONCLUSION

Our findings have elucidated the quantitative representation of music genres in the human cortex, indicating the possibility of modeling this categorization of complex auditory stimuli based on brain activity.

Decoding visual information from high-density diffuse optical tomography neuroimaging data

Background:

Neural decoding could be useful in many ways, from serving as a neuroscience research tool to providing a means of augmented communication for patients with neurological conditions. However, applications of decoding are currently constrained by the limitations of traditional neuroimaging modalities. Electrocorticography requires invasive neurosurgery, magnetic resonance imaging (MRI) is too cumbersome for uses like daily communication, and alternatives like functional near-infrared spectroscopy (fNIRS) offer poor image quality. High-density diffuse optical tomography (HD-DOT) is an emerging modality that uses denser optode arrays than fNIRS to combine logistical advantages of optical neuroimaging with enhanced image quality. Despite the resulting promise of HD-DOT for facilitating field applications of neuroimaging, decoding of brain activity as measured by HD-DOT has yet to be evaluated.

Objective:

To assess the feasibility and performance of decoding with HD-DOT in visual cortex.

Methods and Results:

To establish the feasibility of decoding at the single-trial level with HD-DOT, a template matching strategy was used to decode visual stimulus position. A receiver operating characteristic (ROC) analysis was used to quantify the sensitivity, specificity, and reproducibility of binary visual decoding. Mean areas under the curve (AUCs) greater than 0.97 across 10 imaging sessions in a highly sampled participant were observed. ROC analyses of decoding across 5 participants established both reproducibility in multiple individuals and the feasibility of inter-individual decoding (mean AUCs > 0.7), although decoding performance varied between individuals. Phase-encoded checkerboard stimuli were used to assess more complex, non-binary decoding with HD-DOT. Across 3 highly sampled participants, the phase of a 60° wide checkerboard wedge rotating 10° per second through 360° was decoded with a within-participant error of 25.8±24.7°. Decoding between participants was also feasible based on permutation-based significance testing.

Conclusions:

Visual stimulus information can be decoded accurately, reproducibly, and across a range of detail (for both binary and non-binary outcomes) at the single-trial level (without needing to block-average test data) using HD-DOT data. These results lay the foundation for future studies of more complex decoding with HD-DOT and applications in clinical populations.

Quantitative models reveal the organization of diverse cognitive functions in the brain

Our daily life is realized by the complex orchestrations of diverse brain functions, including perception, decision-making, and action. The essential goal of cognitive neuroscience is to reveal the complete representations underlying these functions. Recent studies have characterised perceptual experiences using encoding models. However, few attempts have been made to build a quantitative model describing the cortical organization of multiple active, cognitive processes. Here, we measure brain activity using fMRI, while subjects perform 103 cognitive tasks, and examine cortical representations with two voxel-wise encoding models. A sparse task-type model reveals a hierarchical organization of cognitive tasks, together with their representation in cognitive space and cortical mapping. A cognitive factor model utilizing continuous, metadata-based intermediate features predicts brain activity and decodes tasks, even under novel conditions. Collectively, our results show the usability of quantitative models of cognitive processes, thus providing a framework for the comprehensive cortical organization of human cognition.

Speech Recognition via fNIRS Based Brain Signals

In this paper, we present the first evidence that perceived speech can be identified from the listeners' brain signals measured via functional-near infrared spectroscopy (fNIRS)—a non-invasive, portable, and wearable neuroimaging technique suitable for ecologically valid settings. In this study, participants listened audio clips containing English stories while prefrontal and parietal cortices were monitored with fNIRS. Machine learning was applied to train predictive models using fNIRS data from a subject pool to predict which part of a story was listened by a new subject not in the pool based on the brain's hemodynamic response as measured by fNIRS. fNIRS signals can vary considerably from subject to subject due to the different head size, head shape, and spatial locations of brain functional regions. To overcome this difficulty, a generalized canonical correlation analysis (GCCA) was adopted to extract latent variables that are shared among the listeners before applying principal component analysis (PCA) for dimension reduction and applying logistic regression for classification. A 74.7% average accuracy has been achieved for differentiating between two 50 s. long story segments and a 43.6% average accuracy has been achieved for differentiating four 25 s. long story segments. These results suggest the potential of an fNIRS based-approach for building a speech decoding brain-computer-interface for developing a new type of neural prosthetic system.