Relating the Structure of Noise Correlations in Macaque Primary Visual Cortex to Decoder Performance

Probing the Structure and Functional Properties of the Dropout-Induced Correlated Variability in Convolutional Neural Networks

Computational neuroscience studies have shown that the structure of neural variability to an unchanged stimulus affects the amount of information encoded. Some artificial deep neural networks, such as those with Monte Carlo dropout layers, also have variable responses when the input is fixed. However, the structure of the trial-by-trial neural covariance in neural networks with dropout has not been studied, and its role in decoding accuracy is unknown. We studied the above questions in a convolutional neural network model with dropout in both the training and testing phases. We found that trial-by-trial correlation between neurons (i.e., noise correlation) is positive and low dimensional. Neurons that are close in a feature map have larger noise correlation. These properties are surprisingly similar to the findings in the visual cortex. We further analyzed the alignment of the main axes of the covariance matrix. We found that different images share a common trial-by-trial noise covariance subspace, and they are aligned with the global signal covariance. This evidence that the noise covariance is aligned with signal covariance suggests that noise covariance in dropout neural networks reduces network accuracy, which we further verified directly with a trial-shuffling procedure commonly used in neuroscience. These findings highlight a previously overlooked aspect of dropout layers that can affect network performance. Such dropout networks could also potentially be a computational model of neural variability.

Cognition is entangled with metabolism: relevance for resting-state EEG-fMRI

The brain is a living organ with distinct metabolic constraints. However, these constraints are typically considered as secondary or supportive of information processing which is primarily performed by neurons. The default operational definition of neural information processing is that (1) it is ultimately encoded as a change in individual neuronal firing rate as this correlates with the presentation of a peripheral stimulus, motor action or cognitive task. Two additional assumptions are associated with this default interpretation: (2) that the incessant background firing activity against which changes in activity are measured plays no role in assigning significance to the extrinsically evoked change in neural firing, and (3) that the metabolic energy that sustains this background activity and which correlates with differences in neuronal firing rate is merely a response to an evoked change in neuronal activity. These assumptions underlie the design, implementation, and interpretation of neuroimaging studies, particularly fMRI, which relies on changes in blood oxygen as an indirect measure of neural activity. In this article we reconsider all three of these assumptions in light of recent evidence. We suggest that by combining EEG with fMRI, new experimental work can reconcile emerging controversies in neurovascular coupling and the significance of ongoing, background activity during resting-state paradigms. A new conceptual framework for neuroimaging paradigms is developed to investigate how ongoing neural activity is "entangled" with metabolism. That is, in addition to being recruited to support locally evoked neuronal activity (the traditional hemodynamic response), changes in metabolic support may be independently "invoked" by non-local brain regions, yielding flexible neurovascular coupling dynamics that inform the cognitive context. This framework demonstrates how multimodal neuroimaging is necessary to probe the neurometabolic foundations of cognition, with implications for the study of neuropsychiatric disorders.

Altered Response Dynamics and Increased Population Correlation to Tonal Stimuli Embedded in Noise in Aging Auditory Cortex

Age-related hearing loss (presbycusis) is a chronic health condition that affects one-third of the world population. One hallmark of presbycusis is a difficulty hearing in noisy environments. Presbycusis can be separated into two components: alterations of peripheral mechanotransduction of sound in the cochlea and central alterations of auditory processing areas of the brain. Although the effects of the aging cochlea in hearing loss have been well studied, the role of the aging brain in hearing loss is less well understood. Therefore, to examine how age-related central processing changes affect hearing in noisy environments, we used a mouse model (Thy1-GCaMP6s X CBA) that has excellent peripheral hearing in old age. We used in vivo two-photon Ca2+ imaging to measure the responses of neuronal populations in auditory cortex (ACtx) of adult (2-6 months, nine male, six female, 4180 neurons) and aging mice (15-17 months, six male, three female, 1055 neurons) while listening to tones in noisy backgrounds. We found that ACtx neurons in aging mice showed larger responses to tones and have less suppressed responses consistent with reduced inhibition. Aging neurons also showed less sensitivity to temporal changes. Population analysis showed that neurons in aging mice showed higher pairwise activity correlations and showed a reduced diversity in responses to sound stimuli. Using neural decoding techniques, we show a loss of information in neuronal populations in the aging brain. Thus, aging not only affects the responses of single neurons but also affects how these neurons jointly represent stimuli.SIGNIFICANCE STATEMENT Aging results in hearing deficits particularly under challenging listening conditions. We show that auditory cortex contains distinct subpopulations of excitatory neurons that preferentially encode different stimulus features and that aging selectively reduces certain subpopulations. We also show that aging increases correlated activity between neurons and thereby reduces the response diversity in auditory cortex. The loss of population response diversity leads to a decrease of stimulus information and deficits in sound encoding, especially in noisy backgrounds. Future work determining the identities of circuits affected by aging could provide new targets for therapeutic strategies.

Temporally precise population coding of dynamic sounds by auditory cortex

Fluctuations in the amplitude envelope of complex sounds provide critical cues for hearing, particularly for speech and animal vocalizations. Responses to amplitude modulation (AM) in the ascending auditory pathway have chiefly been described for single neurons. How neural populations might collectively encode and represent information about AM remains poorly characterized, even in primary auditory cortex (A1). We modeled population responses to AM based on data recorded from A1 neurons in awake squirrel monkeys and evaluated how accurately single trial responses to modulation frequencies from 4 to 512 Hz could be decoded as functions of population size, composition, and correlation structure. We found that a population-based decoding model that simulated convergent, equally weighted inputs was highly accurate and remarkably robust to the inclusion of neurons that were individually poor decoders. By contrast, average rate codes based on convergence performed poorly; effective decoding using average rates was only possible when the responses of individual neurons were segregated, as in classical population decoding models using labeled lines. The relative effectiveness of dynamic rate coding in auditory cortex was explained by shared modulation phase preferences among cortical neurons, despite heterogeneity in rate-based modulation frequency tuning. Our results indicate significant population-based synchrony in primary auditory cortex and suggest that robust population coding of the sound envelope information present in animal vocalizations and speech can be reliably achieved even with indiscriminate pooling of cortical responses. These findings highlight the importance of firing rate dynamics in population-based sensory coding.NEW & NOTEWORTHY Fundamental questions remain about population coding in primary auditory cortex (A1). In particular, issues of spike timing in models of neural populations have been largely ignored. We find that spike-timing in response to sound envelope fluctuations is highly similar across neuron populations in A1. This property of shared envelope phase preference allows for a simple population model involving unweighted convergence of neuronal responses to classify amplitude modulation frequencies with high accuracy.

Synergistic population coding of natural communication stimuli by hindbrain electrosensory neurons

Understanding how neural populations encode natural stimuli with complex spatiotemporal structure to give rise to perception remains a central problem in neuroscience. Here we investigated population coding of natural communication stimuli by hindbrain neurons within the electrosensory system of weakly electric fish Apteronotus leptorhynchus. Overall, we found that simultaneously recorded neural activities were correlated: signal but not noise correlations were variable depending on the stimulus waveform as well as the distance between neurons. Combining the neural activities using an equal-weight sum gave rise to discrimination performance between different stimulus waveforms that was limited by redundancy introduced by noise correlations. However, using an evolutionary algorithm to assign different weights to individual neurons before combining their activities (i.e., a weighted sum) gave rise to increased discrimination performance by revealing synergistic interactions between neural activities. Our results thus demonstrate that correlations between the neural activities of hindbrain electrosensory neurons can enhance information about the structure of natural communication stimuli that allow for reliable discrimination between different waveforms by downstream brain areas.

Population Coding of Natural Electrosensory Stimuli by Midbrain Neurons

Natural stimuli display spatiotemporal characteristics that typically vary over orders of magnitude, and their encoding by sensory neurons remains poorly understood. We investigated population coding of highly heterogeneous natural electrocommunication stimuli in Apteronotus leptorhynchus of either sex. Neuronal activities were positively correlated with one another in the absence of stimulation, and correlation magnitude decayed with increasing distance between recording sites. Under stimulation, we found that correlations between trial-averaged neuronal responses (i.e., signal correlations) were positive and higher in magnitude for neurons located close to another, but that correlations between the trial-to-trial variability (i.e., noise correlations) were independent of physical distance. Overall, signal and noise correlations were independent of stimulus waveform as well as of one another. To investigate how neuronal populations encoded natural electrocommunication stimuli, we considered a nonlinear decoder for which the activities were combined. Decoding performance was best for a timescale of 6 ms, indicating that midbrain neurons transmit information via precise spike timing. A simple summation of neuronal activities (equally weighted sum) revealed that noise correlations limited decoding performance by introducing redundancy. Using an evolution algorithm to optimize performance when considering instead unequally weighted sums of neuronal activities revealed much greater performance values, indicating that midbrain neuron populations transmit information that reliably enable discrimination between different stimulus waveforms. Interestingly, we found that different weight combinations gave rise to similar discriminability, suggesting robustness. Our results have important implications for understanding how natural stimuli are integrated by downstream brain areas to give rise to behavioral responses.SIGNIFICANCE STATEMENT We show that midbrain electrosensory neurons display correlations between their activities and that these can significantly impact performance of decoders. While noise correlations limited discrimination performance by introducing redundancy, considering unequally weighted sums of neuronal activities gave rise to much improved performance and mitigated the deleterious effects of noise correlations. Further analysis revealed that increased discriminability was achieved by making trial-averaged responses more separable, as well as by reducing trial-to-trial variability by eliminating noise correlations. We further found that multiple combinations of weights could give rise to similar discrimination performances, which suggests that such combinatorial codes could be achieved in the brain. We conclude that the activities of midbrain neuronal populations can be used to reliably discriminate between highly heterogeneous stimulus waveforms.

Scaling of sensory information in large neural populations shows signatures of information-limiting correlations

How is information distributed across large neuronal populations within a given brain area? Information may be distributed roughly evenly across neuronal populations, so that total information scales linearly with the number of recorded neurons. Alternatively, the neural code might be highly redundant, meaning that total information saturates. Here we investigate how sensory information about the direction of a moving visual stimulus is distributed across hundreds of simultaneously recorded neurons in mouse primary visual cortex. We show that information scales sublinearly due to correlated noise in these populations. We compartmentalized noise correlations into information-limiting and nonlimiting components, then extrapolate to predict how information grows with even larger neural populations. We predict that tens of thousands of neurons encode 95% of the information about visual stimulus direction, much less than the number of neurons in primary visual cortex. These findings suggest that the brain uses a widely distributed, but nonetheless redundant code that supports recovering most sensory information from smaller subpopulations.