Simple and complex acoustic regularities are encoded at different levels of the auditory hierarchy

Systemic neurophysiological signals of auditory predictive coding

Predictive coding framework posits that our brain continuously monitors changes in the environment and updates its predictive models, minimizing prediction errors to efficiently adapt to environmental demands. However, the underlying neurophysiological mechanisms of these predictive phenomena remain unclear. The present study aimed to explore the systemic neurophysiological correlates of predictive coding processes during passive and active auditory processing. Electroencephalography (EEG), functional near-infrared spectroscopy (fNIRS), and autonomic nervous system (ANS) measures were analyzed using an auditory pattern-based novelty oddball paradigm. A sample of 32 healthy subjects was recruited. The results showed shared slow evoked potentials between passive and active conditions that could be interpreted as automatic predictive processes of anticipation and updating, independent of conscious attentional effort. A dissociated topography of the cortical hemodynamic activity and distinctive evoked potentials upon auditory pattern violation were also found between both conditions, whereas only conscious perception leading to imperative responses was accompanied by phasic ANS responses. These results suggest a systemic-level hierarchical reallocation of predictive coding neural resources as a function of contextual demands in the face of sensory stimulation. Principal component analysis permitted to associate the variability of some of the recorded signals.

Auditory representations for long lasting sounds: Insights from event-related brain potentials and neural oscillations

The basic features of short sounds, such as frequency and intensity including their temporal dynamics, are integrated in a unitary representation. Knowledge on how our brain processes long lasting sounds is scarce. We review research utilizing the Mismatch Negativity event-related potential and neural oscillatory activity for studying representations for long lasting simple versus complex sounds such as sinusoidal tones versus speech. There is evidence for a temporal constraint in the formation of auditory representations: Auditory edges like sound onsets within long lasting sounds open a temporal window of about 350 ms in which the sounds' dynamics are integrated into a representation, while information beyond that window contributes less to that representation. This integration window segments the auditory input into short chunks. We argue that the representations established in adjacent integration windows can be concatenated into an auditory representation of a long sound, thus, overcoming the temporal constraint.

Attention modulates repetition effects in a context of low periodicity

Stimulus repetition can result in a reduction in neural responses (i.e., repetition suppression) in neuroimaging studies. Predictive coding models of perception postulate that this phenomenon largely reflects the top-down attenuation of prediction errors. Electroencephalography research further demonstrated that repetition effects consist of sequentially ordered attention-independent and attention-dependent components in a context of high periodicity. However, the statistical structure of our auditory environment is richer than that of a fixed pattern. It remains unclear if the attentional modulation of repetition effects can be generalised to a setting which better represents the nature of our auditory environment. Here we used electroencephalography to investigate whether the attention-independent and attention-dependent components of repetition effects previously described in the auditory modality remain in a context of low periodicity where temporary disruption might be absent/present. Participants were presented with repetition trains of various lengths, with/without temporary disruptions. We found attention-independent and attention-dependent repetition effects on, respectively, the P2 and P3a event-related potential components. This pattern of results is in line with previous research, confirming that the attenuation of prediction errors upon stimulus repetition is first registered regardless of attentional state before further attenuation of attended but not unattended prediction errors takes place. However, unlike previous reports, these effects manifested on later components. This divergence from previous studies is discussed in terms of the possible contribution of contextual factors.

Mismatch-negativity (MMN) in animal models: Homology of human MMN?

Mismatch negativity (MMN) has long been considered to be one of the deviance-detecting neural characteristics. Animal models exhibit similar neural activities, called MMN-like responses; however, there has been considerable debate on whether MMN-like responses are homologous to MMN in humans. Herein, we reviewed several studies that compared the electrophysiological, pharmacological, and functional properties of MMN-like responses and adaptation-exhibiting middle-latency responses (MLRs) in animals with those in humans. Accumulating evidence suggests that there are clear differences between MMN-like responses and MLRs, in particular that MMN-like responses can be distinguished from mere effects of adaptation, i.e., stimulus-specific adaptation. Finally, we discuss a new direction for research on MMN-like responses by introducing our recent work, which demonstrated that MMN-like responses represent empirical salience of deviant stimuli, suggesting a new functional role of MMN beyond simple deviance detection.

The Neuronal Basis of Predictive Coding Along the Auditory Pathway: From the Subcortical Roots to Cortical Deviance Detection

In this review, we attempt to integrate the empirical evidence regarding stimulus-specific adaptation (SSA) and mismatch negativity (MMN) under a predictive coding perspective (also known as Bayesian or hierarchical-inference model). We propose a renewed methodology for SSA study, which enables a further decomposition of deviance detection into repetition suppression and prediction error, thanks to the use of two controls previously introduced in MMN research: the many-standards and the cascade sequences. Focusing on data obtained with cellular recordings, we explain how deviance detection and prediction error are generated throughout hierarchical levels of processing, following two vectors of increasing computational complexity and abstraction along the auditory neuraxis: from subcortical toward cortical stations and from lemniscal toward nonlemniscal divisions. Then, we delve into the particular characteristics and contributions of subcortical and cortical structures to this generative mechanism of hierarchical inference, analyzing what is known about the role of neuromodulation and local microcircuitry in the emergence of mismatch signals. Finally, we describe how SSA and MMN are occurring at similar time frame and cortical locations, and both are affected by the manipulation of N-methyl- D-aspartate receptors. We conclude that there is enough empirical evidence to consider SSA and MMN, respectively, as the microscopic and macroscopic manifestations of the same physiological mechanism of deviance detection in the auditory cortex. Hence, the development of a common theoretical framework for SSA and MMN is all the more recommendable for future studies. In this regard, we suggest a shared nomenclature based on the predictive coding interpretation of deviance detection.

The effects of aging on early stages of the auditory deviance detection system

OBJECTIVE

The aging effects on auditory change detection have been studied using the Mismatch Negativity (MMN) potential. However, recent studies have found earlier correlates of deviance detection at the level of the middle-latency response (MLR) and the effects of aging on this deviant-related response have not yet been clarified. The purpose of this study was to examine the effects of aging on both levels of the auditory deviance detection system.

METHODS

MMN and MLR responses were recorded in 33 young and 29 older adults from 32 scalp electrodes during frequency oddball and swapped-oddball conditions.

RESULTS

In the young group, modulation of MLR and a clear MMN response were observed, whereas in the aged group, no evidence of deviance detection was found at the level of MLR and the MMN amplitude was significantly diminished.

CONCLUSIONS

Based on the obtained results, aging affects both levels of the auditory deviance detection system which seems to be a result of deficits in regularity encoding along the auditory hierarchy.

SIGNIFICANCE

The current findings suggest that age-related physiological changes result in deficits in regularity encoding, starting from early stages of processing. This might eventually affect stream segregation and induce difficulties in understanding speech in complex environments.

How neuroscience can inform the study of individual differences in cognitive abilities

Theories of human mental abilities should be consistent with what is known in neuroscience. Currently, tests of human mental abilities are modeled by cognitive constructs such as attention, working memory, and speed of information processing. These constructs are in turn related to a single general ability. However, brains are very complex systems and whether most of the variability between the operations of different brains can be ascribed to a single factor is questionable. Research in neuroscience suggests that psychological processes such as perception, attention, decision, and executive control are emergent properties of interacting distributed networks. The modules that make up these networks use similar computational processes that involve multiple forms of neural plasticity, each having different time constants. Accordingly, these networks might best be characterized in terms of the information they process rather than in terms of abstract psychological processes such as working memory and executive control.

Mismatch negativity to pitch pattern deviants in schizophrenia

Simple mismatch negativity (MMN) to infrequent pitch deviants is impaired in individuals with long-term schizophrenia (Sz). The complex MMN elicited by pattern deviance often manifes is cut from here]->ts later after deviant onset than simple MMN and can ascertain deficits in abstracting relationships between stimuli. Sz exhibit reduced complex MMN, but so far this has only been measured when deviance detection relies on a grouping rule. We measured MMN to deviants in pitch-based rules to see whether MMN is also abnormal in Sz under these conditions. Three experiments were conducted. Twenty-seven Sz and 28 healthy matched controls (HC) participated in Experiments 1 and 2, and 24 Sz and 26 HC participated in Experiment 3. Experiment 1 was a standard pitch MMN task, and Sz showed the expected MMN reduction (~ 115 ms) in the simple pitch deviant compared to HC. Experiment 2 comprised standard groups of six tones that ascended in pitch, and deviant groups where the last tone descended in pitch. Complex MMN was late (~ 510 ms) and significantly blunted in Sz. Experiment 3 comprised standard groups of 12 tones (six tones ascending in pitch followed by six tones descending in pitch, like a scale), and deviant groups containing two repetitions of six ascending tones (the scale restarted midstream). Complex MMN was also late (~ 460 ms) and significantly blunted in Sz. These results identify a late pitch pattern deviance-related MMN that is deficient in schizophrenia. This suggests specific deficits in later more complex deviance detection in schizophrenia for abstract patterns.

Comparison of Auditory Middle-Latency Responses From Two Deconvolution Methods at 40 Hz

GOAL

Auditory middle-latency responses (MLRs) are reported to be particularly susceptible to stimulation rate. Deconvolution methods are necessary to unwrap the overlapping responses at a high rate under the linear superposition assumption. This study aims to investigate and compare the MLR characteristics at high and conventional stimulation rates.

METHODS

The characteristics were examined in healthy adults by using two closely related deconvolution paradigms, namely continuous-loop averaging deconvolution and multirate steady-state averaging deconvolution at a mean rate of 40 Hz, and a conventional low rate of 5 Hz.

RESULTS

The morphology and stability of the MLRs can benefit from a high-rate stimulation. It appears that stimulation sequencing strategies of deconvolution methods exerted divergent rate effects on MLR characteristics, which might be associated with different adaptation mechanisms.

CONCLUSION

MLRs obtained by two deconvolution methods and the conventional reference feature differently from one another.

SIGNIFICANCE

These findings have critical implications in our current understanding of the rate effects on MLR characteristics which may inspire further studies to explore the characteristics of evoked responses at high rates and deconvolution paradigms.

Implicit learning of predictable sound sequences modulates human brain responses at different levels of the auditory hierarchy

Deviant stimuli, violating regularities in a sensory environment, elicit the mismatch negativity (MMN), largely described in the Event-Related Potential literature. While it is widely accepted that the MMN reflects more than basic change detection, a comprehensive description of mental processes modulating this response is still lacking. Within the framework of predictive coding, deviance processing is part of an inference process where prediction errors (the mismatch between incoming sensations and predictions established through experience) are minimized. In this view, the MMN is a measure of prediction error, which yields specific expectations regarding its modulations by various experimental factors. In particular, it predicts that the MMN should decrease as the occurrence of a deviance becomes more predictable. We conducted a passive oddball EEG study and manipulated the predictability of sound sequences by means of different temporal structures. Importantly, our design allows comparing mismatch responses elicited by predictable and unpredictable violations of a simple repetition rule and therefore departs from previous studies that investigate violations of different time-scale regularities. We observed a decrease of the MMN with predictability and interestingly, a similar effect at earlier latencies, within 70 ms after deviance onset. Following these pre-attentive responses, a reduced P3a was measured in the case of predictable deviants. We conclude that early and late deviance responses reflect prediction errors, triggering belief updating within the auditory hierarchy. Beside, in this passive study, such perceptual inference appears to be modulated by higher-level implicit learning of sequence statistical structures. Our findings argue for a hierarchical model of auditory processing where predictive coding enables implicit extraction of environmental regularities.

Deviance-Related Responses along the Auditory Hierarchy: Combined FFR, MLR and MMN Evidence

The mismatch negativity (MMN) provides a correlate of automatic auditory discrimination in human auditory cortex that is elicited in response to violation of any acoustic regularity. Recently, deviance-related responses were found at much earlier cortical processing stages as reflected by the middle latency response (MLR) of the auditory evoked potential, and even at the level of the auditory brainstem as reflected by the frequency following response (FFR). However, no study has reported deviance-related responses in the FFR, MLR and long latency response (LLR) concurrently in a single recording protocol. Amplitude-modulated (AM) sounds were presented to healthy human participants in a frequency oddball paradigm to investigate deviance-related responses along the auditory hierarchy in the ranges of FFR, MLR and LLR. AM frequency deviants modulated the FFR, the Na and Nb components of the MLR, and the LLR eliciting the MMN. These findings demonstrate that it is possible to elicit deviance-related responses at three different levels (FFR, MLR and LLR) in one single recording protocol, highlight the involvement of the whole auditory hierarchy in deviance detection and have implications for cognitive and clinical auditory neuroscience. Moreover, the present protocol provides a new research tool into clinical neuroscience so that the functional integrity of the auditory novelty system can now be tested as a whole in a range of clinical populations where the MMN was previously shown to be defective.

Echoic memory: investigation of its temporal resolution by auditory offset cortical responses

Previous studies showed that the amplitude and latency of the auditory offset cortical response depended on the history of the sound, which implicated the involvement of echoic memory in shaping a response. When a brief sound was repeated, the latency of the offset response depended precisely on the frequency of the repeat, indicating that the brain recognized the timing of the offset by using information on the repeat frequency stored in memory. In the present study, we investigated the temporal resolution of sensory storage by measuring auditory offset responses with magnetoencephalography (MEG). The offset of a train of clicks for 1 s elicited a clear magnetic response at approximately 60 ms (Off-P50m). The latency of Off-P50m depended on the inter-stimulus interval (ISI) of the click train, which was the longest at 40 ms (25 Hz) and became shorter with shorter ISIs (2.5∼20 ms). The correlation coefficient r² for the peak latency and ISI was as high as 0.99, which suggested that sensory storage for the stimulation frequency accurately determined the Off-P50m latency. Statistical analysis revealed that the latency of all pairs, except for that between 200 and 400 Hz, was significantly different, indicating the very high temporal resolution of sensory storage at approximately 5 ms.

Encoding of nested levels of acoustic regularity in hierarchically organized areas of the human auditory cortex

Our auditory system is able to encode acoustic regularity of growing levels of complexity to model and predict incoming events. Recent evidence suggests that early indices of deviance detection in the time range of the middle-latency responses (MLR) precede the mismatch negativity (MMN), a well-established error response associated with deviance detection. While studies suggest that only the MMN, but not early deviance-related MLR, underlie complex regularity levels, it is not clear whether these two mechanisms interplay during scene analysis by encoding nested levels of acoustic regularity, and whether neuronal sources underlying local and global deviations are hierarchically organized. We registered magnetoencephalographic evoked fields to rapidly presented four-tone local sequences containing a frequency change. Temporally integrated local events, in turn, defined global regularities, which were infrequently violated by a tone repetition. A global magnetic mismatch negativity (MMNm) was obtained at 140-220 ms when breaking the global regularity, but no deviance-related effects were shown in early latencies. Conversely, Nbm (45-55 ms) and Pbm (60-75 ms) deflections of the MLR, and an earlier MMNm response at 120-160 ms, responded to local violations. Distinct neuronal generators in the auditory cortex underlay the processing of local and global regularity violations, suggesting that nested levels of complexity of auditory object representations are represented in separated cortical areas. Our results suggest that the different processing stages and anatomical areas involved in the encoding of auditory representations, and the subsequent detection of its violations, are hierarchically organized in the human auditory cortex.

Neuronal adaptation, novelty detection and regularity encoding in audition

The ability to detect unexpected stimuli in the acoustic environment and determine their behavioral relevance to plan an appropriate reaction is critical for survival. This perspective article brings together several viewpoints and discusses current advances in understanding the mechanisms the auditory system implements to extract relevant information from incoming inputs and to identify unexpected events. This extraordinary sensitivity relies on the capacity to codify acoustic regularities, and is based on encoding properties that are present as early as the auditory midbrain. We review state-of-the-art studies on the processing of stimulus changes using non-invasive methods to record the summed electrical potentials in humans, and those that examine single-neuron responses in animal models. Human data will be based on mismatch negativity (MMN) and enhanced middle latency responses (MLR). Animal data will be based on the activity of single neurons at the cortical and subcortical levels, relating selective responses to novel stimuli to the MMN and to stimulus-specific neural adaptation (SSA). Theoretical models of the neural mechanisms that could create SSA and novelty responses will also be discussed.

Timing matters: the processing of pitch relations

The human central auditory system can automatically extract abstract regularities from a variant auditory input. To this end, temporarily separated events need to be related. This study tested whether the timing between events, falling either within or outside the temporal window of integration (~350 ms), impacts the extraction of abstract feature relations. We utilized tone pairs for which tones within but not across pairs revealed a constant pitch relation (e.g., pitch of second tone of a pair higher than pitch of first tone, while absolute pitch values varied across pairs). We measured the mismatch negativity (MMN; the brain's error signal to auditory regularity violations) to second tones that rarely violated the pitch relation (e.g., pitch of second tone lower). A Short condition in which tone duration (90 ms) and stimulus onset asynchrony between the tones of a pair were short (110 ms) was compared to two conditions, where this onset asynchrony was long (510 ms). In the Long Gap condition, the tone durations were identical to Short (90 ms), but the silent interval was prolonged by 400 ms. In Long Tone, the duration of the first tone was prolonged by 400 ms, while the silent interval was comparable to Short (20 ms). Results show a frontocentral MMN of comparable amplitude in all conditions. Thus, abstract pitch relations can be extracted even when the within-pair timing exceeds the integration period. Source analyses indicate MMN generators in the supratemporal cortex. Interestingly, they were located more anterior in Long Gap than in Short and Long Tone. Moreover, frontal generator activity was found for Long Gap and Long Tone. Thus, the way in which the system automatically registers irregular abstract pitch relations depends on the timing of the events to be linked. Pending that the current MMN data mirror established abstract rule representations coding the regular pitch relation, neural processes building these templates vary with timing.

Predictability effects in auditory scene analysis: a review

Many sound sources emit signals in a predictable manner. The idea that predictability can be exploited to support the segregation of one source's signal emissions from the overlapping signals of other sources has been expressed for a long time. Yet experimental evidence for a strong role of predictability within auditory scene analysis (ASA) has been scarce. Recently, there has been an upsurge in experimental and theoretical work on this topic resulting from fundamental changes in our perspective on how the brain extracts predictability from series of sensory events. Based on effortless predictive processing in the auditory system, it becomes more plausible that predictability would be available as a cue for sound source decomposition. In the present contribution, empirical evidence for such a role of predictability in ASA will be reviewed. It will be shown that predictability affects ASA both when it is present in the sound source of interest (perceptual foreground) and when it is present in other sound sources that the listener wishes to ignore (perceptual background). First evidence pointing toward age-related impairments in the latter capacity will be addressed. Moreover, it will be illustrated how effects of predictability can be shown by means of objective listening tests as well as by subjective report procedures, with the latter approach typically exploiting the multi-stable nature of auditory perception. Critical aspects of study design will be delineated to ensure that predictability effects can be unambiguously interpreted. Possible mechanisms for a functional role of predictability within ASA will be discussed, and an analogy with the old-plus-new heuristic for grouping simultaneous acoustic signals will be suggested.

Adaptation in the auditory system: an overview

The early stages of the auditory system need to preserve the timing information of sounds in order to extract the basic features of acoustic stimuli. At the same time, different processes of neuronal adaptation occur at several levels to further process the auditory information. For instance, auditory nerve fiber responses already experience adaptation of their firing rates, a type of response that can be found in many other auditory nuclei and may be useful for emphasizing the onset of the stimuli. However, it is at higher levels in the auditory hierarchy where more sophisticated types of neuronal processing take place. For example, stimulus-specific adaptation, where neurons show adaptation to frequent, repetitive stimuli, but maintain their responsiveness to stimuli with different physical characteristics, thus representing a distinct kind of processing that may play a role in change and deviance detection. In the auditory cortex, adaptation takes more elaborate forms, and contributes to the processing of complex sequences, auditory scene analysis and attention. Here we review the multiple types of adaptation that occur in the auditory system, which are part of the pool of resources that the neurons employ to process the auditory scene, and are critical to a proper understanding of the neuronal mechanisms that govern auditory perception.