The Representation of Prediction Error in Auditory Cortex

Auditory learning of recurrent tone sequences is present in the newborn's brain

The seemingly effortless ability of our auditory system to rapidly detect new events in a dynamic environment is crucial for survival. Whether the underlying brain processes are innate is unknown. To answer this question, electroencephalography was recorded while regularly patterned (REG) versus random (RAND) tone sequences were presented to sleeping neonates. Regular relative to random sequences elicited differential neural responses after only a single repetition of the pattern indicating the existence of an innate capacity of the auditory system to detect auditory sequential regularities. We show that the newborn auditory system accumulates evidence only somewhat longer than the minimum amount determined by the ideal Bayesian observer model (the prediction from a variable-order Markov chain model) before detecting a repeating pattern. Thus, newborns can quickly form representations for regular features of the sound input, preparing the way for learning the contingencies of the environment.

Hierarchical temporal prediction captures motion processing along the visual pathway

Visual neurons respond selectively to features that become increasingly complex from the eyes to the cortex. Retinal neurons prefer flashing spots of light, primary visual cortical (V1) neurons prefer moving bars, and those in higher cortical areas favor complex features like moving textures. Previously, we showed that V1 simple cell tuning can be accounted for by a basic model implementing temporal prediction - representing features that predict future sensory input from past input (Singer et al., 2018). Here, we show that hierarchical application of temporal prediction can capture how tuning properties change across at least two levels of the visual system. This suggests that the brain does not efficiently represent all incoming information; instead, it selectively represents sensory inputs that help in predicting the future. When applied hierarchically, temporal prediction extracts time-varying features that depend on increasingly high-level statistics of the sensory input.

Surprise and novelty in the brain

Notions of surprise and novelty have been used in various experimental and theoretical studies across multiple brain areas and species. However, 'surprise' and 'novelty' refer to different quantities in different studies, which raises concerns about whether these studies indeed relate to the same functionalities and mechanisms in the brain. Here, we address these concerns through a systematic investigation of how different aspects of surprise and novelty relate to different brain functions and physiological signals. We review recent classifications of definitions proposed for surprise and novelty along with links to experimental observations. We show that computational modeling and quantifiable definitions enable novel interpretations of previous findings and form a foundation for future theoretical and experimental studies.

Repetition plasticity in primary auditory cortex occurs across long timescales for spectrotemporally randomized pure-tones

Repetition plasticity is a ubiquitous property of sensory systems in which repetitive sensation causes either a decrease ("repetition suppression", i.e. "adaptation") or increase ("repetition enhancement", i.e. "facilitation") in the amplitude of neural responses. Timescales of repetition plasticity for sensory neurons typically span milliseconds to tens of seconds, with longer durations for cortical vs subcortical regions. Here, we used 2-photon (2P) imaging to study repetition plasticity in mouse primary auditory cortex (A1) layer 2/3 (L2/3) during the presentation of spectrotemporally randomized pure-tone frequencies. Our study revealed subpopulations of neurons with repetition plasticity for equiprobable frequencies spaced minutes apart over a 20-minute period. We found both repetition suppression and enhancement in individual neurons and on average across populations. Each neuron tended to show repetition plasticity for 1-2 pure-tone frequencies near the neuron's best frequency. Moreover, we found correlated changes in neural response amplitude and latency across stimulus repetitions. Together, our results highlight cortical specialization for pattern recognition over long timescales in complex acoustic sequences.

Adaptation in auditory processing

Adaptation is an essential feature of auditory neurons, which reduces their responses to unchanging and recurring sounds and allows their response properties to be matched to the constantly changing statistics of sounds that reach the ears. As a consequence, processing in the auditory system highlights novel or unpredictable sounds and produces an efficient representation of the vast range of sounds that animals can perceive by continually adjusting the sensitivity and, to a lesser extent, the tuning properties of neurons to the most commonly encountered stimulus values. Together with attentional modulation, adaptation to sound statistics also helps to generate neural representations of sound that are tolerant to background noise and therefore plays a vital role in auditory scene analysis. In this review, we consider the diverse forms of adaptation that are found in the auditory system in terms of the processing levels at which they arise, the underlying neural mechanisms, and their impact on neural coding and perception. We also ask what the dynamics of adaptation, which can occur over multiple timescales, reveal about the statistical properties of the environment. Finally, we examine how adaptation to sound statistics is influenced by learning and experience and changes as a result of aging and hearing loss.

Learning-Induced Odor Modulation of Neuronal Activity in Auditory Cortex

Sensory cortices, even of primary regions, are not purely unisensory. Rather, cortical neurons in sensory cortex show various forms of multisensory interactions. While some multisensory interactions naturally co-occur, the combination of others will co-occur through experience. In real life, learning and experience will result in conjunction with seemingly disparate sensory information that ultimately becomes behaviorally relevant, impacting perception, cognition, and action. Here we describe a novel auditory discrimination task in mice, designed to manipulate the expectation of upcoming trials using olfactory cues. We show that, after learning, female mice display a transient period of several days during which they exploit odor-mediated expectations for making correct decisions. Using two-photon calcium imaging of single neurons in auditory cortex (ACx) during behavior, we found that the behavioral effects of odor-mediated expectations are accompanied by an odor-induced modulation of neuronal activity. Further, we find that these effects are manifested differentially, based on the response preference of individual cells. A significant portion of effects, but not all, are consistent with a predictive coding framework. Our data show that learning novel odor-sound associations evoke changes in ACx. We suggest that behaviorally relevant multisensory environments mediate contextual effects as early as ACx.SIGNIFICANCE STATEMENT Natural environments are composed of multisensory objects. It remains unclear whether and how animals learn the regularities of congruent multisensory associations and how these may impact behavior and neural activity. We tested how learned odor-sound associations affected single-neuron responses in auditory cortex. We introduce a novel auditory discrimination task for mice in which odors set different contexts of expectation to upcoming trials. We show that, although the task can be solved purely by sounds, odor-mediated expectation impacts performance. We further show that odors cause a modulation of neuronal activity in auditory cortex, which is correlated with behavior. These results suggest that learning prompts an interaction of odor and sound information as early as sensory cortex.

Zipfian Distributions in Child-Directed Speech

Across languages, word frequency and rank follow a power law relation, forming a distribution known as the Zipfian distribution. There is growing experimental evidence that this well-studied phenomenon may be beneficial for language learning. However, most investigations of word distributions in natural language have focused on adult-to-adult speech: Zipf's law has not been thoroughly evaluated in child-directed speech (CDS) across languages. If Zipfian distributions facilitate learning, they should also be found in CDS. At the same time, several unique properties of CDS may result in a less skewed distribution. Here, we examine the frequency distribution of words in CDS in three studies. We first show that CDS is Zipfian across 15 languages from seven language families. We then show that CDS is Zipfian from early on (six-months) and across development for five languages with sufficient longitudinal data. Finally, we show that the distribution holds across different parts of speech: Nouns, verbs, adjectives and prepositions follow a Zipfian distribution. Together, the results show that the input children hear is skewed in a particular way from early on, providing necessary (but not sufficient) support for the postulated learning advantage of such skew. They highlight the need to study skewed learning environments experimentally.

Predictive encoding of pure tones and FM-sweeps in the human auditory cortex

Expectations substantially influence perception, but the neural mechanisms underlying this influence are not fully understood. A prominent view is that sensory neurons encode prediction error with respect to expectations on upcoming sensory input. Although the encoding of prediction error has been previously demonstrated in the human auditory cortex (AC), previous studies often induced expectations using stimulus repetition, potentially confounding prediction error with neural habituation. These studies also measured AC as a single population, failing to consider possible predictive specializations of different AC fields. Moreover, the few studies that considered prediction error to stimuli other than pure tones yielded conflicting results. Here, we used functional magnetic resonance imaging (fMRI) to systematically investigate prediction error to subjective expectations in auditory cortical fields Te1.0, Te1.1, Te1.2, and Te3, and two types of stimuli: pure tones and frequency modulated (FM) sweeps. Our results show that prediction error is elicited with respect to the participants' expectations independently of stimulus repetition and similarly expressed across auditory fields. Moreover, despite the radically different strategies underlying the decoding of pure tones and FM-sweeps, both stimulus modalities were encoded as prediction error in most fields of AC. Altogether, our results provide unequivocal evidence that predictive coding is the general encoding mechanism in AC.

Distinct interacting cortical networks for stimulus-response and repetition-suppression

Non-invasive studies consider the initial neural stimulus response (SR) and repetition suppression (RS) - the decreased response to repeated sensory stimuli - as engaging the same neurons. That is, RS is a suppression of the SR. We challenge this conjecture using electrocorticographic (ECoG) recordings with high spatial resolution in ten patients listening to task-irrelevant trains of auditory stimuli. SR and RS were indexed by high-frequency activity (HFA) across temporal, parietal, and frontal cortices. HFASR and HFARS were temporally and spatially distinct, with HFARS emerging later than HFASR and showing only a limited spatial intersection with HFASR: most HFASR sites did not demonstrate HFARS, and HFARS was found where no HFASR could be recorded. β activity was enhanced in HFARS compared to HFASR cortical sites. θ activity was enhanced in HFASR compared to HFARS sites. Furthermore, HFASR sites propagated information to HFARS sites via transient θ:β phase-phase coupling. In contrast to predictive coding (PC) accounts our results indicate that HFASR and HFARS are functionally linked but have minimal spatial overlap. HFASR might enable stable and rapid perception of environmental stimuli across extended temporal intervals. In contrast HFARS might support efficient generation of an internal model based on stimulus history.

Efficient Temporal Coding in the Early Visual System: Existing Evidence and Future Directions

While it is universally accepted that the brain makes predictions, there is little agreement about how this is accomplished and under which conditions. Accurate prediction requires neural circuits to learn and store spatiotemporal patterns observed in the natural environment, but it is not obvious how such information should be stored, or encoded. Information theory provides a mathematical formalism that can be used to measure the efficiency and utility of different coding schemes for data transfer and storage. This theory shows that codes become efficient when they remove predictable, redundant spatial and temporal information. Efficient coding has been used to understand retinal computations and may also be relevant to understanding more complicated temporal processing in visual cortex. However, the literature on efficient coding in cortex is varied and can be confusing since the same terms are used to mean different things in different experimental and theoretical contexts. In this work, we attempt to provide a clear summary of the theoretical relationship between efficient coding and temporal prediction, and review evidence that efficient coding principles explain computations in the retina. We then apply the same framework to computations occurring in early visuocortical areas, arguing that data from rodents is largely consistent with the predictions of this model. Finally, we review and respond to criticisms of efficient coding and suggest ways that this theory might be used to design future experiments, with particular focus on understanding the extent to which neural circuits make predictions from efficient representations of environmental statistics.

Spatiotemporal Signatures of Surprise Captured by Magnetoencephalography

Surprise and social influence are linked through several neuropsychological mechanisms. By garnering attention, causing arousal, and motivating engagement, surprise provides a context for effective or durable social influence. Attention to a surprising event motivates the formation of an explanation or updating of models, while high arousal experiences due to surprise promote memory formation. They both encourage engagement with the surprising event through efforts aimed at understanding the situation. By affecting the behavior of the individual or a social group via setting an attractive engagement context, surprise plays an important role in shaping personal and social change. Surprise is an outcome of the brain’s function in constantly anticipating the future of sensory inputs based on past experiences. When new sensory data is different from the brain’s predictions shaped by recent trends, distinct neural signals are generated to report this surprise. As a quantitative approach to modeling the generation of brain surprise, input stimuli containing surprising elements are employed in experiments such as oddball tasks during which brain activity is recorded. Although surprise has been well characterized in many studies, an information-theoretical model to describe and predict the surprise level of an external stimulus in the recorded MEG data has not been reported to date, and setting forth such a model is the main objective of this paper. Through mining trial-by-trial MEG data in an oddball task according to theoretical definitions of surprise, the proposed surprise decoding model employs the entire epoch of the brain response to a stimulus to measure surprise and assesses which collection of temporal/spatial components in the recorded data can provide optimal power for describing the brain’s surprise. We considered three different theoretical formulations for surprise assuming the brain acts as an ideal observer that calculates transition probabilities to estimate the generative distribution of the input. We found that middle temporal components and the right and left fronto-central regions offer the strongest power for decoding surprise. Our findings provide a practical and rigorous method for measuring the brain’s surprise, which can be employed in conjunction with behavioral data to evaluate the interactive and social effects of surprising events.

Context-Dependent Inhibitory Control of Stimulus-Specific Adaptation

Stimulus-specific adaptation (SSA) is the reduction in responses to frequent stimuli (standards) that does not generalize to rare stimuli (deviants). We investigated the contribution of inhibition in auditory cortex to SSA using two-photon targeted cell-attached recordings and optogenetic manipulations in male mice. We characterized the responses of parvalbumin (PV)-, somatostatin (SST)-, and vasoactive intestinal polypeptide (VIP)-expressing interneurons of layer 2/3, and of serotonin receptor 5HT3a-expressing interneurons of layer 1. All populations showed early-onset SSA. Unexpectedly, the PV, SST, and VIP populations exhibited a substantial late component of evoked activity, often stronger for standard than for deviant stimuli. Optogenetic suppression of PV neurons facilitated pyramidal neuron responses substantially more (approximately ×10) for deviants than for standards. VIP suppression decreased responses of putative PV neurons, specifically for standard but not for deviant stimuli. Thus, the inhibitory network does not generate cortical SSA, but powerfully controls its expression by differentially affecting the responses to deviants and to standards.SIGNIFICANCE STATEMENT Stimulus-specific adaptation (SSA) reflects the growing complexity of auditory processing along the ascending auditory system. In the presence of SSA, neuronal responses depend not only on the stimulus itself but also on the history of stimulation. Strong SSA in the fast, ascending auditory pathway first occurs in cortex. Here we studied the role of the cortical inhibitory network in shaping SSA, showing that while cortical inhibition does not generate SSA, it powerfully controls its expression. We deduce that the cortical network contributes in crucial ways to the properties of SSA.

Simultaneous mnemonic and predictive representations in the auditory cortex

Recent studies have shown that stimulus history can be decoded via the use of broadband sensory impulses to reactivate mnemonic representations.^1-4. However, memories of previous stimuli can also be used to form sensory predictions about upcoming stimuli.^5,6 Predictive mechanisms allow the brain to create a probable model of the outside world, which can be updated when errors are detected between the model predictions and external inputs. ^7-10 Direct recordings in the auditory cortex of awake mice established neural mechanisms for how encoding mechanisms might handle working memory and predictive processes without "overwriting" recent sensory events in instances where predictive mechanisms are triggered by oddballs within a sequence.¹¹ However, it remains unclear whether mnemonic and predictive information can be decoded from cortical activity simultaneously during passive, implicit sequence processing, even in anesthetized models. Here, we recorded neural activity elicited by repeated stimulus sequences using electrocorticography (ECoG) in the auditory cortex of anesthetized rats, where events within the sequence (referred to henceforth as "vowels," for simplicity) were occasionally replaced with a broadband noise burst or omitted entirely. We show that both stimulus history and predicted stimuli can be decoded from neural responses to broadband impulses, at overlapping latencies but based on independent and uncorrelated data features. We also demonstrate that predictive representations are dynamically updated over the course of stimulation.

Constructing the hierarchy of predictive auditory sequences in the marmoset brain

Our brains constantly generate predictions of sensory input that are compared with actual inputs, propagate the prediction-errors through a hierarchy of brain regions, and subsequently update the internal predictions of the world. However, the essential feature of predictive coding, the notion of hierarchical depth and its neural mechanisms, remains largely unexplored. Here, we investigated the hierarchical depth of predictive auditory processing by combining functional magnetic resonance imaging (fMRI) and high-density whole-brain electrocorticography (ECoG) in marmoset monkeys during an auditory local-global paradigm in which the temporal regularities of the stimuli were designed at two hierarchical levels. The prediction-errors and prediction updates were examined as neural responses to auditory mismatches and omissions. Using fMRI, we identified a hierarchical gradient along the auditory pathway: midbrain and sensory regions represented local, shorter-time-scale predictive processing followed by associative auditory regions, whereas anterior temporal and prefrontal areas represented global, longer-time-scale sequence processing. The complementary ECoG recordings confirmed the activations at cortical surface areas and further differentiated the signals of prediction-error and update, which were transmitted via putative bottom-up γ and top-down β oscillations, respectively. Furthermore, omission responses caused by absence of input, reflecting solely the two levels of prediction signals that are unique to the hierarchical predictive coding framework, demonstrated the hierarchical top-down process of predictions in the auditory, temporal, and prefrontal areas. Thus, our findings support the hierarchical predictive coding framework, and outline how neural networks and spatiotemporal dynamics are used to represent and arrange a hierarchical structure of auditory sequences in the marmoset brain.

Broadband Dynamics Rather than Frequency-Specific Rhythms Underlie Prediction Error in the Primate Auditory Cortex

Detection of statistical irregularities, measured as a prediction error response, is fundamental to the perceptual monitoring of the environment. We studied whether prediction error response is associated with neural oscillations or asynchronous broadband activity. Electrocorticography was conducted in three male monkeys, who passively listened to the auditory roving oddball stimuli. Local field potentials (LFPs) recorded over the auditory cortex underwent spectral principal component analysis, which decoupled broadband and rhythmic components of the LFP signal. We found that the broadband component captured the prediction error response, whereas none of the rhythmic components were associated with statistical irregularities of sounds. The broadband component displayed more stochastic, asymmetrical multifractal properties than the rhythmic components, which revealed more self-similar dynamics. We thus conclude that the prediction error response is captured by neuronal populations generating asynchronous broadband activity, defined by irregular dynamic states, which, unlike oscillatory rhythms, appear to enable the neural representation of auditory prediction error response.SIGNIFICANCE STATEMENT This study aimed to examine the contribution of oscillatory and asynchronous components of auditory local field potentials in the generation of prediction error responses to sensory irregularities, as this has not been directly addressed in the previous studies. Here, we show that mismatch negativity-an auditory prediction error response-is driven by the asynchronous broadband component of potentials recorded in the auditory cortex. This finding highlights the importance of nonoscillatory neural processes in the predictive monitoring of the environment. At a more general level, the study demonstrates that stochastic neural processes, which are often disregarded as neural noise, do have a functional role in the processing of sensory information.

Encoding of acquired sound-sequence salience by auditory cortical offset responses

Behaviorally relevant sounds are often composed of distinct acoustic units organized into specific temporal sequences. The meaning of such sound sequences can therefore be fully recognized only when they have terminated. However, the neural mechanisms underlying the perception of sound sequences remain unclear. Here, we use two-photon calcium imaging in the auditory cortex of behaving mice to test the hypothesis that neural responses to termination of sound sequences ("Off-responses") encode their acoustic history and behavioral salience. We find that auditory cortical Off-responses encode preceding sound sequences and that learning to associate a sound sequence with a reward induces enhancement of Off-responses relative to responses during the sound sequence ("On-responses"). Furthermore, learning enhances network-level discriminability of sound sequences by Off-responses. Last, learning-induced plasticity of Off-responses but not On-responses lasts to the next day. These findings identify auditory cortical Off-responses as a key neural signature of acquired sound-sequence salience.

Shorter Contextual Timescale Rather Than Memory Deficit in Aging

Many aspects of cognitive ability and brain function that change as we age look like deficits on account of measurable differences in comparison to younger adult groups. One such difference occurs in auditory sensory responses that index perceptual learning. Meta-analytic findings show reliable age-related differences in auditory responses to repetitive patterns of sound and to rare violations of those patterns, variously attributed to deficits in auditory sensory memory and inhibition. Here, we determine whether proposed deficits would render older adults less prone to primacy effects, robustly observed in young adults, which present as a tendency for first learning to have a disproportionate influence over later perceptual inference. The results confirm this reduced sensitivity to primacy effects but do not support impairment in auditory sensory memory as the origin of this difference. Instead, the aging brain produces data consistent with shorter timescales of contextual reference. In conclusion, age-related differences observed previously for perceptual inference appear highly context-specific necessitating reconsideration of whether and to what function the notion of deficit should be attributed, and even whether the notion of deficit is appropriate at all.

Inverted central auditory hierarchies for encoding local intervals and global temporal patterns

In sensory systems, representational features of increasing complexity emerge at successive stages of processing. In the mammalian auditory pathway, the clearest change from brainstem to cortex is defined by what is lost, not by what is gained, in that high-fidelity temporal coding becomes increasingly restricted to slower acoustic modulation rates.^1,2 Here, we explore the idea that sluggish temporal processing is more than just an inability for fast processing, but instead reflects an emergent specialization for encoding sound features that unfold on very slow timescales.^3,4 We performed simultaneous single unit ensemble recordings from three hierarchical stages of auditory processing in awake mice - the inferior colliculus (IC), medial geniculate body of the thalamus (MGB) and primary auditory cortex (A1). As expected, temporal coding of brief local intervals (0.001 - 0.1 s) separating consecutive noise bursts was robust in the IC and declined across MGB and A1. By contrast, slowly developing (∼1 s period) global rhythmic patterns of inter-burst interval sequences strongly modulated A1 spiking, were weakly captured by MGB neurons, and not at all by IC neurons. Shifts in stimulus regularity were not represented by changes in A1 spike rates, but rather in how the spikes were arranged in time. These findings show that low-level auditory neurons with fast timescales encode isolated sound features but not the longer gestalt, while the extended timescales in higher-level areas can facilitate sensitivity to slower contextual changes in the sensory environment.

Neural surprise in somatosensory Bayesian learning

Tracking statistical regularities of the environment is important for shaping human behavior and perception. Evidence suggests that the brain learns environmental dependencies using Bayesian principles. However, much remains unknown about the employed algorithms, for somesthesis in particular. Here, we describe the cortical dynamics of the somatosensory learning system to investigate both the form of the generative model as well as its neural surprise signatures. Specifically, we recorded EEG data from 40 participants subjected to a somatosensory roving-stimulus paradigm and performed single-trial modeling across peri-stimulus time in both sensor and source space. Our Bayesian model selection procedure indicates that evoked potentials are best described by a non-hierarchical learning model that tracks transitions between observations using leaky integration. From around 70ms post-stimulus onset, secondary somatosensory cortices are found to represent confidence-corrected surprise as a measure of model inadequacy. Indications of Bayesian surprise encoding, reflecting model updating, are found in primary somatosensory cortex from around 140ms. This dissociation is compatible with the idea that early surprise signals may control subsequent model update rates. In sum, our findings support the hypothesis that early somatosensory processing reflects Bayesian perceptual learning and contribute to an understanding of its underlying mechanisms.

Our environment features statistical regularities, such as a drop of rain predicting imminent rainfall. Despite the importance for behavior and survival, much remains unknown about how these dependencies are learned, particularly for somatosensation. As surprise signalling about novel observations indicates a mismatch between one’s beliefs and the world, it has been hypothesized that surprise computation plays an important role in perceptual learning. By analyzing EEG data from human participants receiving sequences of tactile stimulation, we compare different formulations of surprise and investigate the employed underlying learning model. Our results indicate that the brain estimates transitions between observations. Furthermore, we identified different signatures of surprise computation and thereby provide a dissociation of the neural correlates of belief inadequacy and belief updating. Specifically, early surprise responses from around 70ms were found to signal the need for changes to the model, with encoding of its subsequent updating occurring from around 140ms. These results provide insights into how somatosensory surprise signals may contribute to the learning of environmental statistics.

Stimulus uncertainty affects perception in human echolocation: Timing, level, and spectrum

The human brain may use recent sensory experience to create sensory templates that are then compared to incoming sensory input, that is, "knowing what to listen for." This can lead to greater perceptual sensitivity, as long as the relevant properties of the target stimulus can be reliably estimated from past sensory experiences. Echolocation is an auditory skill probably best understood in bats, but humans can also echolocate. Here we investigated for the first time whether echolocation in humans involves the use of sensory templates derived from recent sensory experiences. Our results showed that when there was certainty in the acoustic properties of the echo relative to the emission, either in temporal onset, spectral content or level, people detected the echo more accurately than when there was uncertainty. In addition, we found that people were more accurate when the emission's spectral content was certain but, surprisingly, not when either its level or temporal onset was certain. Importantly, the lack of an effect of temporal onset of the emission is counter to that found previously for tasks using nonecholocation sounds, suggesting that the underlying mechanisms might be different for echolocation and nonecholocation sounds. Importantly, the effects of stimulus certainty were no different for people with and without experience in echolocation, suggesting that stimulus-specific sensory templates can be used in a skill that people have never used before. From an applied perspective our results suggest that echolocation instruction should encourage users to make clicks that are similar to one another in their spectral content. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Surprise response as a probe for compressed memory states

The limited capacity of recent memory inevitably leads to partial memory of past stimuli. There is also evidence that behavioral and neural responses to novel or rare stimuli are dependent on one’s memory of past stimuli. Thus, these responses may serve as a probe of different individuals’ remembering and forgetting characteristics. Here, we utilize two lossy compression models of stimulus sequences that inherently involve forgetting, which in addition to being a necessity under many conditions, also has theoretical and behavioral advantages. One model is based on a simple stimulus counter and the other on the Information Bottleneck (IB) framework which suggests a more general, theoretically justifiable principle for biological and cognitive phenomena. These models are applied to analyze a novelty-detection event-related potential commonly known as the P300. The trial-by-trial variations of the P300 response, recorded in an auditory oddball paradigm, were subjected to each model to extract two stimulus-compression parameters for each subject: memory length and representation accuracy. These parameters were then utilized to estimate the subjects’ recent memory capacity limit under the task conditions. The results, along with recently published findings on single neurons and the IB model, underscore how a lossy compression framework can be utilized to account for trial-by-trial variability of neural responses at different spatial scales and in different individuals, while at the same time providing estimates of individual memory characteristics at different levels of representation using a theoretically-based parsimonious model.

Surprise responses reflect expectations based on preceding stimuli representations, and hence can be used to infer the characteristics of memory utilized for a task. We suggest a quantitative method for extracting an individual estimate of effective memory capacity dedicated for a task based on the correspondence between a theoretical surprise model and electrophysiological single-trial surprise responses. We demonstrate this method on EEG responses recorded while participants were performing a simple auditory task; we show the correspondence between the theoretical and physiological surprise, and calculate an estimate of the utilized memory. The generality of this framework allows it to be applied to different EEG features that reflect different modes and levels of the processing hierarchy, as well as other physiological measures of surprise responses. Future studies may use this framework to construct a handy diagnostic tool for a quantitative, individualized characterization of memory-related disorders.

Mice tune out not in: violation of prediction drives auditory saliency

Successful navigation in complex acoustic scenes requires focusing on relevant sounds while ignoring irrelevant distractors. It has been argued that the ability to track stimulus statistics and generate predictions supports the choice of what to attend and what to ignore. However, the role of these predictions about future auditory events in drafting decisions remains elusive. While most psychophysical studies in humans indicate that expected stimuli are more easily detected, most work studying physiological auditory processing in animals highlights the detection of unexpected, surprising stimuli. Here, we tested whether in the mouse, high target probability results in enhanced detectability or whether detection is biased towards low-probability deviants using an auditory detection task. We implemented a probabilistic choice model to investigate whether a possible dependence on stimulus statistics arises from short-term serial correlations or from integration over longer periods. Our results demonstrate that target detectability in mice decreases with increasing probability, contrary to humans. We suggest that mice indeed track probability over a timescale of at least several minutes but do not use this information in the same way as humans do: instead of maximizing reward by focusing on high-probability targets, the saliency of a target is determined by surprise.

Musical anhedonia and rewards of music listening: current advances and a proposed model

Music frequently elicits intense emotional responses, a phenomenon that has been scrutinized from multiple disciplines that span the sciences and arts. While most people enjoy music and find it rewarding, there is substantial individual variability in the experience and degree of music-induced reward. Here, we review current work on the neural substrates of hedonic responses to music. In particular, we focus the present review on specific musical anhedonia, a selective lack of pleasure from music. Based on evidence from neuroimaging, neuropsychology, and brain stimulation studies, we derive a neuroanatomical model of the experience of pleasure during music listening. Our model posits that hedonic responses to music are the result of connectivity between structures involved in auditory perception as a predictive process, and those involved in the brain's dopaminergic reward system. We conclude with open questions and implications of this model for future research on why humans appreciate music.

Automatic and feature-specific prediction-related neural activity in the human auditory system

Prior experience enables the formation of expectations of upcoming sensory events. However, in the auditory modality, it is not known whether prediction-related neural signals carry feature-specific information. Here, using magnetoencephalography (MEG), we examined whether predictions of future auditory stimuli carry tonotopic specific information. Participants passively listened to sound sequences of four carrier frequencies (tones) with a fixed presentation rate, ensuring strong temporal expectations of when the next stimulus would occur. Expectation of which frequency would occur was parametrically modulated across the sequences, and sounds were occasionally omitted. We show that increasing the regularity of the sequence boosts carrier-frequency-specific neural activity patterns during both the anticipatory and omission periods, indicating that prediction-related neural activity is indeed feature-specific. Our results illustrate that even without bottom-up input, auditory predictions can activate tonotopically specific templates.

After listening to a predictable sequence of sounds, we can anticipate and predict the next sound in the sequence. Here, the authors show that during expectation of a sound, the brain generates neural activity matching that which is produced by actually hearing the same sound.

Trial-by-trial surprise-decoding model for visual and auditory binary oddball tasks

Having to survive in a continuously changing environment has driven the human brain to actively predict the future state of its surroundings. Oddball tasks are specific types of experiments in which this nature of the human brain is studied. Detailed mathematical models have been constructed to explain the brain's perception in these tasks. These models consider a subject as an ideal observer who abstracts a hypothesis from the previous stimuli, and estimates its hyper-parameters - in order to make the next prediction. The corresponding prediction error is assumed to manifest the subjective surprise of the brain. While the approach of earlier works to this problem has been to suggest an encoding model, we investigated the reverse model: if the stimuli's surprise is assumed as the cause of the observer's surprise, it must be possible to decode the surprise of each stimulus, for every single subject, given only their neural responses, i.e. to tell how unexpected a specific stimulus has been for them. Employing machine learning tools, we developed a surprise decoding model for binary oddball tasks. We constructed our model using the ideal observer proposed by Meyniel et al. in 2016, and applied it to three datasets, one with visual, one with auditory, and one with both visual and auditory stimuli. We demonstrated that our decoding model performs very well for both of the sensory modalities with or without the presence of the subject's motor response.

Methods for computing the maximum performance of computational models of fMRI responses

Computational neuroimaging methods aim to predict brain responses (measured e.g. with functional magnetic resonance imaging [fMRI]) on the basis of stimulus features obtained through computational models. The accuracy of such prediction is used as an indicator of how well the model describes the computations underlying the brain function that is being considered. However, the prediction accuracy is bounded by the proportion of the variance of the brain response which is related to the measurement noise and not to the stimuli (or cognitive functions). This bound to the performance of a computational model has been referred to as the noise ceiling. In previous fMRI applications two methods have been proposed to estimate the noise ceiling based on either a split-half procedure or Monte Carlo simulations. These methods make different assumptions over the nature of the effects underlying the data, and, importantly, their relation has not been clarified yet. Here, we derive an analytical form for the noise ceiling that does not require computationally expensive simulations or a splitting procedure that reduce the amount of data. The validity of this analytical definition is proved in simulations, we show that the analytical solution results in the same estimate of the noise ceiling as the Monte Carlo method. Considering different simulated noise structure, we evaluate different estimators of the variance of the responses and their impact on the estimation of the noise ceiling. We furthermore evaluate the interplay between regularization (often used to estimate model fits to the data when the number of computational features in the model is large) and model complexity on the performance with respect to the noise ceiling. Our results indicate that when considering the variance of the responses across runs, computing the noise ceiling analytically results in similar estimates as the split half estimator and approaches the true noise ceiling under a variety of simulated noise scenarios. Finally, the methods are tested on real fMRI data acquired at 7 Tesla.

The Information Bottleneck and Geometric Clustering

The information bottleneck (IB) approach to clustering takes a joint distribution [Formula: see text] and maps the data [Formula: see text] to cluster labels [Formula: see text], which retain maximal information about [Formula: see text] (Tishby, Pereira, & Bialek, 1999 ). This objective results in an algorithm that clusters data points based on the similarity of their conditional distributions [Formula: see text]. This is in contrast to classic geometric clustering algorithms such as [Formula: see text]-means and gaussian mixture models (GMMs), which take a set of observed data points [Formula: see text] and cluster them based on their geometric (typically Euclidean) distance from one another. Here, we show how to use the deterministic information bottleneck (DIB) (Strouse & Schwab, 2017 ), a variant of IB, to perform geometric clustering by choosing cluster labels that preserve information about data point location on a smoothed data set. We also introduce a novel intuitive method to choose the number of clusters via kinks in the information curve. We apply this approach to a variety of simple clustering problems, showing that DIB with our model selection procedure recovers the generative cluster labels. We also show that, in particular limits of our model parameters, clustering with DIB and IB is equivalent to [Formula: see text]-means and EM fitting of a GMM with hard and soft assignments, respectively. Thus, clustering with (D)IB generalizes and provides an information-theoretic perspective on these classic algorithms.

Brain signatures of a multiscale process of sequence learning in humans

Extracting the temporal structure of sequences of events is crucial for perception, decision-making, and language processing. Here, we investigate the mechanisms by which the brain acquires knowledge of sequences and the possibility that successive brain responses reflect the progressive extraction of sequence statistics at different timescales. We measured brain activity using magnetoencephalography in humans exposed to auditory sequences with various statistical regularities, and we modeled this activity as theoretical surprise levels using several learning models. Successive brain waves related to different types of statistical inferences. Early post-stimulus brain waves denoted a sensitivity to a simple statistic, the frequency of items estimated over a long timescale (habituation). Mid-latency and late brain waves conformed qualitatively and quantitatively to the computational properties of a more complex inference: the learning of recent transition probabilities. Our findings thus support the existence of multiple computational systems for sequence processing involving statistical inferences at multiple scales.

Interactive effects of linguistic abstraction and stimulus statistics in the online modulation of neural speech encoding

Speech processing is highly modulated by context. Prior studies examining frequency-following responses (FFRs), an electrophysiological 'neurophonic' potential that faithfully reflects phase-locked activity from neural ensembles within the auditory network, have demonstrated that stimulus context modulates the integrity of speech encoding. The extent to which context-dependent encoding reflects general auditory properties or interactivities between statistical and higher-level linguistic processes remains unexplored. Our study examined whether speech encoding, as reflected by FFRs, is modulated by abstract phonological relationships between a stimulus and surrounding contexts. FFRs were elicited to a Mandarin rising-tone syllable (/ji-TR/, 'second') randomly presented with other syllables in three contexts from 17 native listeners. In a contrastive context, /ji-TR/ occurred with meaning-contrastive high-level-tone syllables (/ji-H/, 'one'). In an allotone context, TR occurred with dipping-tone syllables /ji-D/, a non-meaning-contrastive variant of /ji-TR/. In a repetitive context, the same /ji-TR/ occurred with other speech tokens of /ji-TR/. Consistent with prior work, neural tracking of /ji-TR/ pitch contour was more faithful in the repetitive condition wherein /ji-TR/ occurred more predictably (p = 1) than in the contrastive condition (p = 0.34). Crucially, in the allotone context, neural tracking of /ji-TR/ was more accurate relative to the contrastive context, despite both having an identical transitional probability (p = 0.34). Mechanistically, the non-meaning-contrastive relationship may have augmented the probability to /ji-TR/ occurrence in the allotone context. Results indicate online interactions between bottom-up and top-down mechanisms, which facilitate speech perception. Such interactivities may predictively fine-tune incoming speech encoding using linguistic and statistical information from prior context.

Enhanced deviant responses in patterned relative to random sound sequences

The brain draws on knowledge of statistical structure in the environment to facilitate detection of new events. Understanding the nature of this representation is a key challenge in sensory neuroscience. Specifically, it is unknown whether real-time perception of rapidly-unfolding sensory signals is driven by a coarse or detailed representation of the proximal stimulus history. We recorded electroencephalography brain responses to frequency outliers in regularly-patterned (REG) versus random (RAND) tone-pip sequences which were generated anew on each trial. REG and RAND sequences were matched in frequency content and span, only differing in the specific order of the tone-pips. Stimuli were very rapid, limiting conscious reasoning in favour of automatic processing of regularity. Listeners were naïve and performed an incidental visual task. Outliers within REG evoked a larger response than matched outliers in RAND. These effects arose rapidly (within 80 msec) and were underpinned by distinct sources from those classically associated with frequency-based deviance detection. These findings are consistent with the notion that the brain continually maintains a detailed representation of ongoing sensory input and that this representation shapes the processing of incoming information. Predominantly auditory-cortical sources code for frequency deviance whilst frontal sources are associated with tracking more complex sequence structure.

Recent advances in understanding the auditory cortex

Our ability to make sense of the auditory world results from neural processing that begins in the ear, goes through multiple subcortical areas, and continues in the cortex. The specific contribution of the auditory cortex to this chain of processing is far from understood. Although many of the properties of neurons in the auditory cortex resemble those of subcortical neurons, they show somewhat more complex selectivity for sound features, which is likely to be important for the analysis of natural sounds, such as speech, in real-life listening conditions. Furthermore, recent work has shown that auditory cortical processing is highly context-dependent, integrates auditory inputs with other sensory and motor signals, depends on experience, and is shaped by cognitive demands, such as attention. Thus, in addition to being the locus for more complex sound selectivity, the auditory cortex is increasingly understood to be an integral part of the network of brain regions responsible for prediction, auditory perceptual decision-making, and learning. In this review, we focus on three key areas that are contributing to this understanding: the sound features that are preferentially represented by cortical neurons, the spatial organization of those preferences, and the cognitive roles of the auditory cortex.

How Do Expectations Shape Perception?

Perception and perceptual decision-making are strongly facilitated by prior knowledge about the probabilistic structure of the world. While the computational benefits of using prior expectation in perception are clear, there are myriad ways in which this computation can be realized. We review here recent advances in our understanding of the neural sources and targets of expectations in perception. Furthermore, we discuss Bayesian theories of perception that prescribe how an agent should integrate prior knowledge and sensory information, and investigate how current and future empirical data can inform and constrain computational frameworks that implement such probabilistic integration in perception.

Adaptive coding for dynamic sensory inference

Behavior relies on the ability of sensory systems to infer properties of the environment from incoming stimuli. The accuracy of inference depends on the fidelity with which behaviorally relevant properties of stimuli are encoded in neural responses. High-fidelity encodings can be metabolically costly, but low-fidelity encodings can cause errors in inference. Here, we discuss general principles that underlie the tradeoff between encoding cost and inference error. We then derive adaptive encoding schemes that dynamically navigate this tradeoff. These optimal encodings tend to increase the fidelity of the neural representation following a change in the stimulus distribution, and reduce fidelity for stimuli that originate from a known distribution. We predict dynamical signatures of such encoding schemes and demonstrate how known phenomena, such as burst coding and firing rate adaptation, can be understood as hallmarks of optimal coding for accurate inference.

Sensory cortex is optimized for prediction of future input

Neurons in sensory cortex are tuned to diverse features in natural scenes. But what determines which features neurons become selective to? Here we explore the idea that neuronal selectivity is optimized to represent features in the recent sensory past that best predict immediate future inputs. We tested this hypothesis using simple feedforward neural networks, which were trained to predict the next few moments of video or audio in clips of natural scenes. The networks developed receptive fields that closely matched those of real cortical neurons in different mammalian species, including the oriented spatial tuning of primary visual cortex, the frequency selectivity of primary auditory cortex and, most notably, their temporal tuning properties. Furthermore, the better a network predicted future inputs the more closely its receptive fields resembled those in the brain. This suggests that sensory processing is optimized to extract those features with the most capacity to predict future input.

A large part of our brain is devoted to processing the sensory inputs that we receive from the world. This allows us to tell, for example, whether we are looking at a cat or a dog, and if we are hearing a bark or a meow. Neurons in the sensory cortex respond to these stimuli by generating spikes of activity. Within each sensory area, neurons respond best to stimuli with precise properties: those in the primary visual cortex prefer edge-like structures that move in a certain direction at a given speed, while neurons in the primary auditory cortex favour sounds that change in loudness over a particular range of frequencies.

Singer et al. sought to understand why neurons respond to the particular features of stimuli that they do. Why do visual neurons react more to moving edges than to, say, rotating hexagons? And why do auditory neurons respond more to certain changing sounds than to, say, constant tones? One leading idea is that the brain tries to use as few spikes as possible to represent real-world stimuli. Known as sparse coding, this principle can account for much of the behaviour of sensory neurons.

Another possibility is that sensory areas respond the way they do because it enables them to best predict future sensory input. To test this idea, Singer et al. used a computer to simulate a network of neurons and trained this network to predict the next few frames of video clips using the previous few frames. When the network had learned this task, Singer et al. examined the neurons’ preferred stimuli. Like neurons in primary visual cortex, the simulated neurons typically responded most to edges that moved over time.

The same network was also trained in a similar way, but this time using sound. As for neurons in primary auditory cortex, the simulated neurons preferred sounds that changed in loudness at particular frequencies. Notably, for both vision and audition, the simulated neurons favoured recent inputs over those further into the past. In this way and others, they were more similar to real neurons than simulated neurons that used sparse coding.

Both artificial networks trained to foretell sensory input and the brain therefore favour the same types of stimuli: the ones that are good at helping to grasp future information. This suggests that the brain represents the sensory world so as to be able to best predict the future.

Knowing how the brain handles information from our senses may help to understand disorders associated with sensory processing, such as dyslexia and tinnitus. It may also inspire approaches for training machines to process sensory inputs, improving artificial intelligence.

Evolutionarily conserved neural signatures involved in sequencing predictions and their relevance for language

Predicting the occurrence of future events from prior ones is vital for animal perception and cognition. Although how such sequence learning (a form of relational knowledge) relates to particular operations in language remains controversial, recent evidence shows that sequence learning is disrupted in frontal lobe damage associated with aphasia. Also, neural sequencing predictions at different temporal scales resemble those involved in language operations occurring at similar scales. Furthermore, comparative work in humans and monkeys highlights evolutionarily conserved frontal substrates and predictive oscillatory signatures in the temporal lobe processing learned sequences of speech signals. Altogether this evidence supports a relational knowledge hypothesis of language evolution, proposing that language processes in humans are functionally integrated with an ancestral neural system for predictive sequence learning.

Predictive coding in auditory perception: challenges and unresolved questions

Predictive coding is arguably the currently dominant theoretical framework for the study of perception. It has been employed to explain important auditory perceptual phenomena, and it has inspired theoretical, experimental and computational modelling efforts aimed at describing how the auditory system parses the complex sound input into meaningful units (auditory scene analysis). These efforts have uncovered some vital questions, addressing which could help to further specify predictive coding and clarify some of its basic assumptions. The goal of the current review is to motivate these questions and show how unresolved issues in explaining some auditory phenomena lead to general questions of the theoretical framework. We focus on experimental and computational modelling issues related to sequential grouping in auditory scene analysis (auditory pattern detection and bistable perception), as we believe that this is the research topic where predictive coding has the highest potential for advancing our understanding. In addition to specific questions, our analysis led us to identify three more general questions that require further clarification: (1) What exactly is meant by prediction in predictive coding? (2) What governs which generative models make the predictions? and (3) What (if it exists) is the correlate of perceptual experience within the predictive coding framework?

Efficient encoding of motion is mediated by gap junctions in the fly visual system

Understanding the computational implications of specific synaptic connectivity patterns is a fundamental goal in neuroscience. In particular, the computational role of ubiquitous electrical synapses operating via gap junctions remains elusive. In the fly visual system, the cells in the vertical-system network, which play a key role in visual processing, primarily connect to each other via axonal gap junctions. This network therefore provides a unique opportunity to explore the functional role of gap junctions in sensory information processing. Our information theoretical analysis of a realistic VS network model shows that within 10 ms following the onset of the visual input, the presence of axonal gap junctions enables the VS system to efficiently encode the axis of rotation, θ, of the fly’s ego motion. This encoding efficiency, measured in bits, is near-optimal with respect to the physical limits of performance determined by the statistical structure of the visual input itself. The VS network is known to be connected to downstream pathways via a subset of triplets of the vertical system cells; we found that because of the axonal gap junctions, the efficiency of this subpopulation in encoding θ is superior to that of the whole vertical system network and is robust to a wide range of signal to noise ratios. We further demonstrate that this efficient encoding of motion by this subpopulation is necessary for the fly's visually guided behavior, such as banked turns in evasive maneuvers. Because gap junctions are formed among the axons of the vertical system cells, they only impact the system’s readout, while maintaining the dendritic input intact, suggesting that the computational principles implemented by neural circuitries may be much richer than previously appreciated based on point neuron models. Our study provides new insights as to how specific network connectivity leads to efficient encoding of sensory stimuli.

Understanding sensory stimuli from the environment and deciding how best to respond to it behaviorally is essential for survival. What makes organisms efficient in encoding these sensory stimuli? This study provides a novel view on this unresolved issue using the visual system of the fly. We show that a specific synaptic connectivity manifested via gap junctions (GJs) among axons in the Vertical System (VS) network leads to particularly high encoding efficiency of the axis of rotation of the fly’s ego motion. Due to these GJs, triplets of VS neurons (the VS5-6-7 triplet), which connect to a downstream motor system, encode motion stimuli at an efficiency close to the physical limit; this efficient encoding is necessary for evasive maneuvers that are critical for the fly to escape predators. We then suggest why GJs in the VS network enable such high encoding efficiency.

Familiar But Unexpected: Effects of Sound Context Statistics on Auditory Responses in the Songbird Forebrain

Rapid discrimination of salient acoustic signals in the noisy natural environment may depend, not only on specific stimulus features, but also on previous experience that generates expectations about upcoming events. We studied the neural correlates of expectation in the songbird forebrain by using natural vocalizations as stimuli and manipulating the category and familiarity of context sounds. In our paradigm, we recorded bilaterally from auditory neurons in awake adult male zebra finches with multiple microelectrodes during repeated playback of a conspecific song, followed by further playback of this test song in different interleaved sequences with other conspecific or heterospecific songs. Significant enhancement in the auditory response to the test song was seen when its acoustic features differed from the statistical distribution of context song features, but not when it shared the same distribution. Enhancement was also seen when the time of occurrence of the test song was uncertain. These results show that auditory forebrain responses in awake animals in the passive hearing state are modulated dynamically by previous auditory experience and imply that the auditory system can identify the category of a sound based on the global features of the acoustic context. Furthermore, this probability-dependent enhancement in responses to surprising stimuli is independent of stimulus-specific adaptation, which tracks familiarity, suggesting that the two processes could coexist in auditory processing. These findings establish the songbird as a model system for studying these phenomena and contribute to our understanding of statistical learning and the origin of human ERP phenomena to unexpected stimuli.SIGNIFICANCE STATEMENT Traditional auditory neurophysiology has mapped acoustic features of sounds to the response properties of neurons; however, growing evidence suggests that neurons can also encode the probability of sounds. We recorded responses of songbird auditory neurons in a novel paradigm that presented a familiar test stimulus in a sequence with similar or dissimilar sounds. The responses encode, not only stimulus familiarity, but also the expectation for a class of sounds based on the recent statistics of varying sounds in the acoustic context. Our approach thus provides a model system that uses a controlled stimulus paradigm to understand the mechanisms by which top-down processes (expectation and memory) and bottom-up processes (based on stimulus features) interact in sensory coding.

Congenital deafness affects deep layers in primary and secondary auditory cortex

Congenital deafness leads to functional deficits in the auditory cortex for which early cochlear implantation can effectively compensate. Most of these deficits have been demonstrated functionally. Furthermore, the majority of previous studies on deafness have involved the primary auditory cortex; knowledge of higher-order areas is limited to effects of cross-modal reorganization. In this study, we compared the cortical cytoarchitecture of four cortical areas in adult hearing and congenitally deaf cats (CDCs): the primary auditory field A1, two secondary auditory fields, namely the dorsal zone and second auditory field (A2); and a reference visual association field (area 7) in the same section stained either using Nissl or SMI-32 antibodies. The general cytoarchitectonic pattern and the area-specific characteristics in the auditory cortex remained unchanged in animals with congenital deafness. Whereas area 7 did not differ between the groups investigated, all auditory fields were slightly thinner in CDCs, this being caused by reduced thickness of layers IV-VI. The study documents that, while the cytoarchitectonic patterns are in general independent of sensory experience, reduced layer thickness is observed in both primary and higher-order auditory fields in layer IV and infragranular layers. The study demonstrates differences in effects of congenital deafness between supragranular and other cortical layers, but similar dystrophic effects in all investigated auditory fields.

Great Expectations: Is there Evidence for Predictive Coding in Auditory Cortex?

Predictive coding is possibly one of the most influential, comprehensive, and controversial theories of neural function. While proponents praise its explanatory potential, critics object that key tenets of the theory are untested or even untestable. The present article critically examines existing evidence for predictive coding in the auditory modality. Specifically, we identify five key assumptions of the theory and evaluate each in the light of animal, human and modeling studies of auditory pattern processing. For the first two assumptions - that neural responses are shaped by expectations and that these expectations are hierarchically organized - animal and human studies provide compelling evidence. The anticipatory, predictive nature of these expectations also enjoys empirical support, especially from studies on unexpected stimulus omission. However, for the existence of separate error and prediction neurons, a key assumption of the theory, evidence is lacking. More work exists on the proposed oscillatory signatures of predictive coding, and on the relation between attention and precision. However, results on these latter two assumptions are mixed or contradictory. Looking to the future, more collaboration between human and animal studies, aided by model-based analyses will be needed to test specific assumptions and implementations of predictive coding - and, as such, help determine whether this popular grand theory can fulfill its expectations.

Characterization of Predictive Behavior of a Retina by Mutual Information

Probing a bullfrog retina with spatially uniform light pulses of correlated stochastic intervals, we calculate the mutual information between the spiking output at the ganglion cells measured with multi-electrode array (MEA) and the interval of the stimulus at a time shift later. The time-integrated information from the output about the future stimulus is maximized when the mean interval of the stimulus is within the dynamic range of the well-established anticipative phenomena of omitted-stimulus responses for the retina. The peak position of the mutual information as a function of the time shift is typically negative considering the processing delay of the retina. However, the peak position can become positive for long enough correlation time of the stimulus when the pulse intervals are generated by a Hidden Markovian model (HMM). This is indicative of a predictive behavior of the retina which is possible only when the hidden variable of the HMM can be recovered from the history of the stimulus for a prediction of its future. We verify that stochastic intervals of the same mean, variance, and correlation time do not result in the same predictive behavior of the retina when they are generated by an Ornstein-Uhlenbeck (OU) process, which is strictly Markovian.

The Cumulative Effects of Predictability on Synaptic Gain in the Auditory Processing Stream

Stimulus predictability can lead to substantial modulations of brain activity, such as shifts in sustained magnetic field amplitude, measured with magnetoencephalography (MEG). Here, we provide a mechanistic explanation of these effects using MEG data acquired from healthy human volunteers (N = 13, 7 female). In a source-level analysis of induced responses, we established the effects of orthogonal predictability manipulations of rapid tone-pip sequences (namely, sequence regularity and alphabet size) along the auditory processing stream. In auditory cortex, regular sequences with smaller alphabets induced greater gamma activity. Furthermore, sequence regularity shifted induced activity in frontal regions toward higher frequencies. To model these effects in terms of the underlying neurophysiology, we used dynamic causal modeling for cross-spectral density and estimated slow fluctuations in neural (postsynaptic) gain. Using the model-based parameters, we accurately explain the sensor-level sustained field amplitude, demonstrating that slow changes in synaptic efficacy, combined with sustained sensory input, can result in profound and sustained effects on neural responses to predictable sensory streams.

SIGNIFICANCE STATEMENT Brain activity can be strongly modulated by the predictability of stimuli it is currently processing. An example of such a modulation is a shift in sustained magnetic field amplitude, measured with magnetoencephalography. Here, we provide a mechanistic explanation of these effects. First, we establish the oscillatory neural correlates of independent predictability manipulations in hierarchically distinct areas of the auditory processing stream. Next, we use a biophysically realistic computational model to explain these effects in terms of the underlying neurophysiology. Finally, using the model-based parameters describing neural gain modulation, we can explain the previously unexplained effects observed at the sensor level. This demonstrates that slow modulations of synaptic gain can result in profound and sustained effects on neural activity.

Stimulus-specific adaptation in a recurrent network model of primary auditory cortex

Stimulus-specific adaptation (SSA) occurs when neurons decrease their responses to frequently-presented (standard) stimuli but not, or not as much, to other, rare (deviant) stimuli. SSA is present in all mammalian species in which it has been tested as well as in birds. SSA confers short-term memory to neuronal responses, and may lie upstream of the generation of mismatch negativity (MMN), an important human event-related potential. Previously published models of SSA mostly rely on synaptic depression of the feedforward, thalamocortical input. Here we study SSA in a recurrent neural network model of primary auditory cortex. When the recurrent, intracortical synapses display synaptic depression, the network generates population spikes (PSs). SSA occurs in this network when deviants elicit a PS but standards do not, and we demarcate the regions in parameter space that allow SSA. While SSA based on PSs does not require feedforward depression, we identify feedforward depression as a mechanism for expanding the range of parameters that support SSA. We provide predictions for experiments that could help differentiate between SSA due to synaptic depression of feedforward connections and SSA due to synaptic depression of recurrent connections. Similar to experimental data, the magnitude of SSA in the model depends on the frequency difference between deviant and standard, probability of the deviant, inter-stimulus interval and input amplitude. In contrast to models based on feedforward depression, our model shows true deviance sensitivity as found in experiments.