Brain indices of automatic pattern processing

Auditory Processing of Intonational Rises and Falls in German: Rises Are Special in Attention Orienting

This article investigates the processing of intonational rises and falls when presented unexpectedly in a stream of repetitive auditory stimuli. It examines the neurophysiological correlates (ERPs) of attention to these unexpected stimuli through the use of an oddball paradigm where sequences of repetitive stimuli are occasionally interspersed with a deviant stimulus, allowing for elicitation of an MMN. Whereas previous oddball studies on attention toward unexpected sounds involving pitch rises were conducted on nonlinguistic stimuli, the present study uses as stimuli lexical items in German with naturalistic intonation contours. Results indicate that rising intonation plays a special role in attention orienting at a pre-attentive processing stage, whereas contextual meaning (here a list of items) is essential for activating attentional resources at a conscious processing stage. This is reflected in the activation of distinct brain responses: Rising intonation evokes the largest MMN, whereas falling intonation elicits a less pronounced MMN followed by a P3 (reflecting a conscious processing stage). Subsequently, we also find a complex interplay between the phonological status (i.e., accent/head marking vs. boundary/edge marking) and the direction of pitch change in their contribution to attention orienting: Attention is not oriented necessarily toward a specific position in prosodic structure (head or edge). Rather, we find that the intonation contour itself and the appropriateness of the contour in the linguistic context are the primary cues to two core mechanisms of attention orienting, pre-attentive and conscious orientation respectively, whereas the phonological status of the pitch event plays only a supplementary role.

Markov chains as a proxy for the predictive memory representations underlying mismatch negativity

Events not conforming to a regularity inherent to a sequence of events elicit prediction error signals of the brain such as the Mismatch Negativity (MMN) and impair behavioral task performance. Events conforming to a regularity lead to attenuation of brain activity such as stimulus-specific adaptation (SSA) and behavioral benefits. Such findings are usually explained by theories stating that the information processing system predicts the forthcoming event of the sequence via detected sequential regularities. A mathematical model that is widely used to describe, to analyze and to generate event sequences are Markov chains: They contain a set of possible events and a set of probabilities for transitions between these events (transition matrix) that allow to predict the next event on the basis of the current event and the transition probabilities. The accuracy of such a prediction depends on the distribution of the transition probabilities. We argue that Markov chains also have useful applications when studying cognitive brain functions. The transition matrix can be regarded as a proxy for generative memory representations that the brain uses to predict the next event. We assume that detected regularities in a sequence of events correspond to (a subset of) the entries in the transition matrix. We apply this idea to the Mismatch Negativity (MMN) research and examine three types of MMN paradigms: classical oddball paradigms emphasizing sound probabilities, between-sound regularity paradigms manipulating transition probabilities between adjacent sounds, and action-sound coupling paradigms in which sounds are associated with actions and their intended effects. We show that the Markovian view on MMN yields theoretically relevant insights into the brain processes underlying MMN and stimulates experimental designs to study the brain's processing of event sequences.

Human cortical processing of interaural coherence

Sounds reach the ears as a mixture of energy generated by different sources. Listeners extract cues that distinguish different sources from one another, including how similar sounds arrive at the two ears, the interaural coherence (IAC). Here, we find listeners cannot reliably distinguish two completely interaurally coherent sounds from a single sound with reduced IAC. Pairs of sounds heard toward the front were readily confused with single sounds with high IAC, whereas those heard to the sides were confused with single sounds with low IAC. Sounds that hold supra-ethological spatial cues are perceived as more diffuse than can be accounted for by their IAC, and this is accounted for by a computational model comprising a restricted, and sound-frequency dependent, distribution of auditory-spatial detectors. We observed elevated cortical hemodynamic responses for sounds with low IAC, suggesting that the ambiguity elicited by sounds with low interaural similarity imposes elevated cortical load.

•

Listeners report ambiguous spatial percepts for sounds with reduced interaural coherence

•

Supra-ethological spatial cues impose an additional diffuse quality to sounds

•

A frequency-dependent distribution of spatial detectors explains behavioral results

•

Sounds with low interaural coherence impose elevated cortical load

The effects of sleep on objective measures of gap detection using a time-efficient multi-deviant paradigm

Auditory temporal resolution, measured through gap detection, is critical for the perception of speech. A time-efficient multi-deviant paradigm has previously been developed for gap detection. The purpose of the present study was to determine if this multi-deviant paradigm could be used for gap detection during NREM sleep. ERPs were recorded in 10 young adults while awake and during the first two hours of NREM sleep. A multi-deviant paradigm was employed with six different deviants varying in gap duration, ranging from 2 to 40 ms. During waking, a DRN was observed for the 10, 20, 30 and 40 ms gaps. The DRN was absent during sleep. A P2 was present in NREM for the 20, 30 and 40 ms gaps followed by a P3a to the 30 and 40 ms gaps. An N350 was observed following the 10, 20, 30 and 40 ms gaps. Previous studies have reported significant ERPs to gaps having shorter durations than the present study. The multi-deviant paradigm may not be suitable for the determination of gap threshold during sleep. Nevertheless, it provides an exquisite means to determine perceptibility and the extent of processing of longer duration, supra-threshold gaps during sleep.

Does mismatch negativity have utility for NMDA receptor drug development in depression?

CLINICAL TRIAL REGISTRATION

Rapid antidepressant effects associated with ketamine have shifted the landscape for the development of therapeutics to treat major depressive disorder (MDD) from a monoaminergic to glutamatergic model. Treatment with ketamine, an N-methyl-D-aspartate (NMDA) receptor antagonist, may be effective, but has many non-glutamatergic targets, and clinical and logistical problems are potential challenges. These factors underscore the importance of manipulations of binding mechanics to produce antidepressant effects without concomitant clinical side effects. This will require identification of efficient biomarkers to monitor target engagement. The mismatch negativity (MMN) is a widely used electrophysiological signature linked to the activity of NMDA receptors (NMDAR) in humans and animals and validated in pre-clinical and clinical studies of ketamine. In this review, we explore the flexibility of the MMN and its capabilities for reliable use in drug development for NMDAR antagonists in MDD. We supplement this with findings from our own research with three distinct NMDAR antagonists. The research described illustrates that there are important distinctions between the mechanisms of NMDAR antagonism, which are further crystallized when considering the paradigm used to study the MMN. We conclude that the lack of standardized methodology currently prevents MMN from being ready for common use in drug discovery. This manuscript describes data collected from the following National Institutes of Health (NIH) and Veterans Affairs (VA) studies: AV-101, NCT03583554; lanicemine, NCT03166501; ketamine, NCT02556606.

Non-invasive electroencephalographical (EEG) recording system in awake monkeys

Background

Human clinical studies reported that several electroencephalographical (EEG) parameters can be used as biomarkers of psychiatric disorders. EEGs recorded from non-human primates (monkeys) is useful for understanding of human pathologies of psychiatric disorders and development of new therapeutic agents.

New methods

In this study, we expand a previous non-invasive head holding system with face masks for awake monkeys to be applied to scalp EEG recording. The new design of a head holding system allows to attach scalp EEG electrodes on the positions comparable to human electrode placement and to present auditory stimuli.

Results

With this system, we could record auditory evoked potentials (AEPs) in auditory sensory gating and oddball paradigms, which are often used as biomarkers of psychiatric disorders in animal models and human patients. The recorded AEPs were comparable to previous human clinical data.

Comparison with existing methods

Compared with previous non-invasive head holding systems, top, side (cheek and ears), and rear of the head can be open for attachment of EEG electrodes and auditory stimulation in the present system.

Conclusions

The results suggest that the present system is useful in EEG recording from awake monkeys. Furthermore, this system can be applied to eye-tracking and chronic intra-cerebral recording experiments.

Emergence of prediction error along the human auditory hierarchy

Auditory prediction errors have been extensively associated with the mismatch negativity (MMN), a cortical auditory evoked potential that denotes deviance detection. Yet, many studies lacked the appropriate controls to disentangle sensory adaptation from prediction error. Furthermore, subcortical deviance detection has been shown in humans through recordings of the frequency-following response (FFR), an early auditory evoked potential that reflects the neural tracking of the periodic characteristics of a sound, suggesting the possibility that prediction errors emerge subcortically in the auditory pathway. The present study aimed at investigating the emergence of prediction error along the auditory hierarchy in humans through combined recordings of the FFR and the MMN, tapping at subcortical and cortical levels, respectively, while disentangling prediction error from sensory adaptation with the use of appropriate controls. "Oddball" sequences of pure tones featuring repeated "standard" stimuli (269 Hz; p = 0.8) and rare "deviant" stimuli (p = 0.2; of 289, 329 and 409 Hz delivered in separated blocks to test "frequency separation" effects) were presented to nineteen healthy young participants. "Reversed" oddball sequences (where standard and deviant tones swapped their roles) were presented allowing comparison of responses to same physical stimuli as a function of functional role (i.e., standard, deviant). Critically, control sequences featuring five equiprobable tones (p = 0.2) allowed to dissociate sensory adaptation from prediction error. Results revealed that the MMN amplitude increased as a function of frequency separation yet displayed the same amplitude when retrieved against the control sequences, confirming previous results. FFRs showed repetition enhancement effects across all frequency separations, as supported by larger spectral amplitude to standard than to deviant and control stimuli. This pattern of results provides insights into the hierarchy of the human prediction error system in audition, suggesting that prediction errors in humans emerge at cortical level.

Encoding of deterministic and stochastic auditory rules in the human brain: The mismatch negativity mechanism does not reflect basic probability

Regularities in a sequence of sounds can be automatically encoded in a predictive model by the auditory system. When a sound deviates from the one predicted by the model, a mismatch negativity (MMN) is elicited, which is taken to reflect a prediction error at a particular level of the model hierarchy. Although there are many studies on deterministic regularities, only a few have investigated the brain's ability to encode non-deterministic regularities. We studied a simple stochastic regularity: two tone pitches (standards, each occurring on 45% of trials); this regularity was occasionally violated by another tone pitch (deviant, occurring on 10% of trials). We found MMN when the deviant's pitch was outside those of the standards, but not when it was between them. Importantly, when we alternated the occurrence of the same two standards, making them deterministic, the deviant elicited MMN, even when its pitch was between those of the standards. Thus, although the MMN system is extremely powerful in establishing even quite complex deterministic regularities, it fails with a simple stochastic regularity. We argue that the MMN system does not know basic probability.

Visual Perceptual Load Does Not Affect the Frequency Mismatch Negativity

The mismatch negativity (MMN) has been of particular interest in auditory perception because of its sensitivity to auditory change. It is typically measured in an oddball task and is computed as the difference of deviant minus standard tones. Previous studies suggest that the oddball MMN can be reduced by crossmodal attention to a concurrent, difficult visual task. However, more recent studies did not replicate this effect. Because previous findings seem to be biased, we preregistered the present study and used Bayesian hypothesis testing to measure the strength of evidence for or against an effect of visual task difficulty. We manipulated visual perceptual load (high and low load). In the task, the visual stimuli were identical for both loads to avoid confounding effects from physical differences of the visual stimuli. We also measured the corrected MMN because the oddball MMN may be confounded by physical differences between deviant and standard tones. The corrected MMN is obtained with a separate control condition in which the same tone as the deviant (critical tone) is equiprobable with other tones. The corrected MMN is computed as deviant minus critical tones. Furthermore, we assessed working memory capacity to examine its moderating role. In our large sample (N = 49), the evidential strength in support of no effect of visual load was moderate for the oddball MMN (9.09 > BF01 > 3.57) and anecdotal to moderate for the corrected MMN (4.55 > BF01 > 2.17). Also, working memory capacity did not correlate with the visual load effect on the oddball MMN and the corrected MMN. The present findings support the robustness of the auditory frequency MMN to manipulations of crossmodal, visual attention and suggest that this relationship is not moderated by working memory capacity.

The Neuronal Basis of Predictive Coding Along the Auditory Pathway: From the Subcortical Roots to Cortical Deviance Detection

In this review, we attempt to integrate the empirical evidence regarding stimulus-specific adaptation (SSA) and mismatch negativity (MMN) under a predictive coding perspective (also known as Bayesian or hierarchical-inference model). We propose a renewed methodology for SSA study, which enables a further decomposition of deviance detection into repetition suppression and prediction error, thanks to the use of two controls previously introduced in MMN research: the many-standards and the cascade sequences. Focusing on data obtained with cellular recordings, we explain how deviance detection and prediction error are generated throughout hierarchical levels of processing, following two vectors of increasing computational complexity and abstraction along the auditory neuraxis: from subcortical toward cortical stations and from lemniscal toward nonlemniscal divisions. Then, we delve into the particular characteristics and contributions of subcortical and cortical structures to this generative mechanism of hierarchical inference, analyzing what is known about the role of neuromodulation and local microcircuitry in the emergence of mismatch signals. Finally, we describe how SSA and MMN are occurring at similar time frame and cortical locations, and both are affected by the manipulation of N-methyl- D-aspartate receptors. We conclude that there is enough empirical evidence to consider SSA and MMN, respectively, as the microscopic and macroscopic manifestations of the same physiological mechanism of deviance detection in the auditory cortex. Hence, the development of a common theoretical framework for SSA and MMN is all the more recommendable for future studies. In this regard, we suggest a shared nomenclature based on the predictive coding interpretation of deviance detection.

Mismatch negativity to pitch pattern deviants in schizophrenia

Simple mismatch negativity (MMN) to infrequent pitch deviants is impaired in individuals with long-term schizophrenia (Sz). The complex MMN elicited by pattern deviance often manifes is cut from here]->ts later after deviant onset than simple MMN and can ascertain deficits in abstracting relationships between stimuli. Sz exhibit reduced complex MMN, but so far this has only been measured when deviance detection relies on a grouping rule. We measured MMN to deviants in pitch-based rules to see whether MMN is also abnormal in Sz under these conditions. Three experiments were conducted. Twenty-seven Sz and 28 healthy matched controls (HC) participated in Experiments 1 and 2, and 24 Sz and 26 HC participated in Experiment 3. Experiment 1 was a standard pitch MMN task, and Sz showed the expected MMN reduction (~ 115 ms) in the simple pitch deviant compared to HC. Experiment 2 comprised standard groups of six tones that ascended in pitch, and deviant groups where the last tone descended in pitch. Complex MMN was late (~ 510 ms) and significantly blunted in Sz. Experiment 3 comprised standard groups of 12 tones (six tones ascending in pitch followed by six tones descending in pitch, like a scale), and deviant groups containing two repetitions of six ascending tones (the scale restarted midstream). Complex MMN was also late (~ 460 ms) and significantly blunted in Sz. These results identify a late pitch pattern deviance-related MMN that is deficient in schizophrenia. This suggests specific deficits in later more complex deviance detection in schizophrenia for abstract patterns.

The First Call Note Plays a Crucial Role in Frog Vocal Communication

Vocal Communication plays a crucial role in survival and reproductive success in most amphibian species. Although amphibian communication sounds are often complex consisting of many temporal features, we know little about the biological significance of each temporal component. The present study examined the biological significance of notes of the male advertisement calls of the Emei music frog (Babina daunchina) using the optimized electroencephalogram (EEG) paradigm of mismatch negativity (MMN). Music frog calls generally contain four to six notes separated approximately by 150 millisecond intervals. A standard stimulus (white noise) and five deviant stimuli (five notes from one advertisement call) were played back to each subject while simultaneously recording multi-channel EEG signals. The results showed that the MMN amplitude for the first call note was significantly larger than for that of the others. Moreover, the MMN amplitudes evoked from the left forebrain and midbrain were typically larger than those from the right counterpart. These results are consistent with the ideas that the first call note conveys more information than the others for auditory recognition and that there is left-hemisphere dominance for processing information derived from conspecific calls in frogs.

Detection of event-related potentials in individual subjects using support vector machines

Event-related potentials (ERPs) are tiny electrical brain responses in the human electroencephalogram that are typically not detectable until they are isolated by a process of signal averaging. Owing to the extremely smallsize of ERP components (ranging from less than 1 μV to tens of μV), compared to background brain rhythms, statistical analyses of ERPs are predominantly carried out in groups of subjects. This limitation is a barrier to the translation of ERP-based neuroscience to applications such as medical diagnostics. We show here that support vector machines (SVMs) are a useful method to detect ERP components in individual subjects with a small set of electrodes and a small number of trials for a mismatch negativity (MMN) ERP component. Such a reduced experiment setup is important for clinical applications. One hundred healthy individuals were presented with an auditory pattern containing pattern-violating deviants to evoke the MMN. Two-class SVMs were then trained to classify averaged ERP waveforms in response to the standard tone (tones that match the pattern) and deviant tone stimuli (tones that violate the pattern). The influence of kernel type, number of epochs, electrode selection, and temporal window size in the averaged waveform were explored. When using all electrodes, averages of all available epochs, and a temporal window from 0 to 900-ms post-stimulus, a linear SVM achieved 94.5 % accuracy. Further analyses using SVMs trained with narrower, sliding temporal windows confirmed the sensitivity of the SVM to data in the latency range associated with the MMN.

A rapid event-related potential (ERP) method for point-of-care evaluation of brain function: development of the Halifax Consciousness Scanner

BACKGROUND

Event-related potentials (ERPs) may provide a non-invasive index of brain function for a range of clinical applications. However, as a lab-based technique, ERPs are limited by technical challenges that prevent full integration into clinical settings.

NEW METHOD

To translate ERP capabilities from the lab to clinical applications, we have developed methods like the Halifax Consciousness Scanner (HCS). HCS is essentially a rapid, automated ERP evaluation of brain functional status. The present study describes the ERP components evoked from auditory tones and speech stimuli. ERP results were obtained using a 5-min test in 100 healthy individuals. The HCS sequence was designed to evoke the N100, the mismatch negativity (MMN), P300, the early negative enhancement (ENE), and the N400. These components reflected sensation, perception, attention, memory, and language perception, respectively. Component detection was examined at group and individual levels, and evaluated across both statistical and classification approaches.

RESULTS

All ERP components were robustly detected at the group level. At the individual level, nonparametric statistical analyses showed reduced accuracy relative to support vector (SVM) machine classification, particularly for speech-based ERPs. Optimized SVM results were MMN: 95.6%; P300: 99.0%; ENE: 91.8%; and N400: 92.3%.

CONCLUSIONS

A spectrum of individual-level ERPs can be obtained in a very short time. Machine learning classification improved detection accuracy across a large healthy control sample. Translating ERPs into clinical applications is increasingly possible at the individual level.

The role of temporal structure in the investigation of sensory memory, auditory scene analysis, and speech perception: a healthy-aging perspective

Listening situations with multiple talkers or background noise are common in everyday communication and are particularly demanding for older adults. Here we review current research on auditory perception in aging individuals in order to gain insights into the challenges of listening under noisy conditions. Informationally rich temporal structure in auditory signals--over a range of time scales from milliseconds to seconds--renders temporal processing central to perception in the auditory domain. We discuss the role of temporal structure in auditory processing, in particular from a perspective relevant for hearing in background noise, and focusing on sensory memory, auditory scene analysis, and speech perception. Interestingly, these auditory processes, usually studied in an independent manner, show considerable overlap of processing time scales, even though each has its own 'privileged' temporal regimes. By integrating perspectives on temporal structure processing in these three areas of investigation, we aim to highlight similarities typically not recognized.

Encoding of nested levels of acoustic regularity in hierarchically organized areas of the human auditory cortex

Our auditory system is able to encode acoustic regularity of growing levels of complexity to model and predict incoming events. Recent evidence suggests that early indices of deviance detection in the time range of the middle-latency responses (MLR) precede the mismatch negativity (MMN), a well-established error response associated with deviance detection. While studies suggest that only the MMN, but not early deviance-related MLR, underlie complex regularity levels, it is not clear whether these two mechanisms interplay during scene analysis by encoding nested levels of acoustic regularity, and whether neuronal sources underlying local and global deviations are hierarchically organized. We registered magnetoencephalographic evoked fields to rapidly presented four-tone local sequences containing a frequency change. Temporally integrated local events, in turn, defined global regularities, which were infrequently violated by a tone repetition. A global magnetic mismatch negativity (MMNm) was obtained at 140-220 ms when breaking the global regularity, but no deviance-related effects were shown in early latencies. Conversely, Nbm (45-55 ms) and Pbm (60-75 ms) deflections of the MLR, and an earlier MMNm response at 120-160 ms, responded to local violations. Distinct neuronal generators in the auditory cortex underlay the processing of local and global regularity violations, suggesting that nested levels of complexity of auditory object representations are represented in separated cortical areas. Our results suggest that the different processing stages and anatomical areas involved in the encoding of auditory representations, and the subsequent detection of its violations, are hierarchically organized in the human auditory cortex.

Timing matters: the processing of pitch relations

The human central auditory system can automatically extract abstract regularities from a variant auditory input. To this end, temporarily separated events need to be related. This study tested whether the timing between events, falling either within or outside the temporal window of integration (~350 ms), impacts the extraction of abstract feature relations. We utilized tone pairs for which tones within but not across pairs revealed a constant pitch relation (e.g., pitch of second tone of a pair higher than pitch of first tone, while absolute pitch values varied across pairs). We measured the mismatch negativity (MMN; the brain's error signal to auditory regularity violations) to second tones that rarely violated the pitch relation (e.g., pitch of second tone lower). A Short condition in which tone duration (90 ms) and stimulus onset asynchrony between the tones of a pair were short (110 ms) was compared to two conditions, where this onset asynchrony was long (510 ms). In the Long Gap condition, the tone durations were identical to Short (90 ms), but the silent interval was prolonged by 400 ms. In Long Tone, the duration of the first tone was prolonged by 400 ms, while the silent interval was comparable to Short (20 ms). Results show a frontocentral MMN of comparable amplitude in all conditions. Thus, abstract pitch relations can be extracted even when the within-pair timing exceeds the integration period. Source analyses indicate MMN generators in the supratemporal cortex. Interestingly, they were located more anterior in Long Gap than in Short and Long Tone. Moreover, frontal generator activity was found for Long Gap and Long Tone. Thus, the way in which the system automatically registers irregular abstract pitch relations depends on the timing of the events to be linked. Pending that the current MMN data mirror established abstract rule representations coding the regular pitch relation, neural processes building these templates vary with timing.

Mismatch negativity latency and cognitive function in schizophrenia

Background

The Mismatch Negativity (MMN) is an event-related potential (ERP) sensitive to early auditory deviance detection and has been shown to be reduced in schizophrenia patients. Moreover, MMN amplitude reduction to duration deviant tones was found to be related to functional outcomes particularly, to neuropsychological (working memory and verbal domains) and psychosocial measures. While MMN amplitude is thought to be correlated with deficits of early sensory processing, the functional significance of MMN latency remains unclear so far. The present study focused on the investigation of MMN in relation to neuropsychological function in schizophrenia.

Method

Forty schizophrenia patients and 16 healthy controls underwent a passive oddball paradigm (2400 binaural tones; 88% standards [1 kHz, 80 db, 80 ms], 11% frequency deviants [1.2 kHz], 11% duration deviants [40 ms]) and a neuropsychological test-battery. Patients were assessed with regard to clinical symptoms.

Results

Compared to healthy controls schizophrenia patients showed diminished MMN amplitude and shorter MMN latency to both deviants as well as an impaired neuropsychological test performance. Severity of positive symptoms was related to decreased MMN amplitude to duration deviants. Furthermore, enhanced verbal memory performance was associated with prolonged MMN latency to frequency deviants in patients.

Conclusion

The present study corroborates previous results of a diminished MMN amplitude and its association with positive symptoms in schizophrenia patients. Both, the findings of a shorter latency to duration and frequency deviants and the relationship of the latter with verbal memory in patients, emphasize the relevance of the temporal aspect of early auditory discrimination processing in schizophrenia.

"Change deafness" arising from inter-feature masking within a single auditory object

Our ability to detect prominent changes in complex acoustic scenes depends not only on the ear's sensitivity but also on the capacity of the brain to process competing incoming information. Here, employing a combination of psychophysics and magnetoencephalography (MEG), we investigate listeners' sensitivity in situations when two features belonging to the same auditory object change in close succession. The auditory object under investigation is a sequence of tone pips characterized by a regularly repeating frequency pattern. Signals consisted of an initial, regularly alternating sequence of three short (60 msec) pure tone pips (in the form ABCABC…) followed by a long pure tone with a frequency that is either expected based on the on-going regular pattern ("LONG expected"-i.e., "LONG-expected") or constitutes a pattern violation ("LONG-unexpected"). The change in LONG-expected is manifest as a change in duration (when the long pure tone exceeds the established duration of a tone pip), whereas the change in LONG-unexpected is manifest as a change in both the frequency pattern and a change in the duration. Our results reveal a form of "change deafness," in that although changes in both the frequency pattern and the expected duration appear to be processed effectively by the auditory system-cortical signatures of both changes are evident in the MEG data-listeners often fail to detect changes in the frequency pattern when that change is closely followed by a change in duration. By systematically manipulating the properties of the changing features and measuring behavioral and MEG responses, we demonstrate that feature changes within the same auditory object, which occur close together in time, appear to compete for perceptual resources.

The what, where and how of auditory-object perception

The fundamental perceptual unit in hearing is the 'auditory object'. Similar to visual objects, auditory objects are the computational result of the auditory system's capacity to detect, extract, segregate and group spectrotemporal regularities in the acoustic environment; the multitude of acoustic stimuli around us together form the auditory scene. However, unlike the visual scene, resolving the component objects within the auditory scene crucially depends on their temporal structure. Neural correlates of auditory objects are found throughout the auditory system. However, neural responses do not become correlated with a listener's perceptual reports until the level of the cortex. The roles of different neural structures and the contribution of different cognitive states to the perception of auditory objects are not yet fully understood.

Event-related potential measures of a violation of an expected increase and decrease in intensity

Unexpected physical increases in the intensity of a frequently occurring “standard” auditory stimulus are experienced as obtrusive. This could either be because of a physical change, the increase in intensity of the “deviant” stimulus, or a psychological change, the violation of the expectancy for the occurrence of the lower intensity standard stimulus. Two experiments were run in which event-related potentials (ERPs) were recorded to determine whether “psychological” increments (violation of an expectancy for a lower intensity) would be processed differently than psychological decrements (violation of an expectancy for a higher intensity). Event-related potentials (ERPs) were recorded while subjects were presented with auditory tones that alternated between low and high intensity. The subjects ignored the auditory stimuli while watching a video. Deviants were created by repeating the same stimulus. In the first experiment, pairs of stimuli alternating in intensity, were presented in separate increment (H-L…H-L…H-H…H-L, in which H = 80 dB SPL and L = 60 dB SPL) and decrement conditions (L-H…L-H…L-L… L-H, in which H = 90 dB SPL and L = 80 dB SPL). The paradigm employed in the second experiment consisted of an alternating intensity pattern (H-L-H-L-H-H-H-L) or (H-L-H-L-L-L-H-L). Importantly, the stimulus prior to the deviant (the standard) and the actual deviants in both increment and decrement conditions in both experiments were physically identical (80 dB SPL tones). The repetition of the lower intensity tone therefore acted as a psychological rather than a physical decrement (a higher intensity tone was expected) while the repetition of the higher intensity tone acted as a psychological increment (a lower intensity tone was expected). The psychological increments in both experiments elicited a larger amplitude mismatch negativity (MMN) than the decrements. Thus, regardless of whether an acoustic change signals a physical increase in intensity or violates an expected decrease in intensity, a large MMN will be elicited.

Attention to natural auditory signals

The challenge of understanding how the brain processes natural signals is compounded by the fact that such signals are often tied closely to specific natural behaviors and natural environments. This added complexity is especially true for auditory communication signals that can carry information at multiple hierarchical levels, and often occur in the context of other competing communication signals. Selective attention provides a mechanism to focus processing resources on specific components of auditory signals, and simultaneously suppress responses to unwanted signals or noise. Although selective auditory attention has been well-studied behaviorally, very little is known about how selective auditory attention shapes the processing on natural auditory signals, and how the mechanisms of auditory attention are implemented in single neurons or neural circuits. Here we review the role of selective attention in modulating auditory responses to complex natural stimuli in humans. We then suggest how the current understanding can be applied to the study of selective auditory attention in the context natural signal processing at the level of single neurons and populations in animal models amenable to invasive neuroscience techniques. This article is part of a Special Issue entitled "Communication Sounds and the Brain: New Directions and Perspectives".

Detection of simple and pattern regularity violations occurs at different levels of the auditory hierarchy

Auditory deviance detection in humans is indexed by the mismatch negativity (MMN), a component of the auditory evoked potential (AEP) of the electroencephalogram (EEG) occurring at a latency of 100–250 ms after stimulus onset. However, by using classic oddball paradigms, differential responses to regularity violations of simple auditory features have been found at the level of the middle latency response (MLR) of the AEP occurring within the first 50 ms after stimulus (deviation) onset. These findings suggest the existence of fast deviance detection mechanisms for simple feature changes, but it is not clear whether deviance detection among more complex acoustic regularities could be observed at such early latencies. To test this, we examined the pre-attentive processing of rare stimulus repetitions in a sequence of tones alternating in frequency in both long and middle latency ranges. Additionally, we introduced occasional changes in the interaural time difference (ITD), so that a simple-feature regularity could be examined in the same paradigm. MMN was obtained for both repetition and ITD deviants, occurring at 150 ms and 100 ms after stimulus onset respectively. At the level of the MLR, a difference was observed between standards and ITD deviants at the Na component (20–30 ms after stimulus onset), for 800 Hz tones, but not for repetition deviants. These findings suggest that detection mechanisms for deviants to simple regularities, but not to more complex regularities, are already activated in the MLR range, supporting the view that the auditory deviance detection system is organized in a hierarchical manner.

Arousal modulates auditory attention and awareness: insights from sleep, sedation, and disorders of consciousness

The interplay between attention and consciousness is frequently tested in altered states of consciousness, including transitions between stages of sleep and sedation, and in pathological disorders of consciousness (DoC; the vegetative and minimally conscious states; VS and MCS). One of the most widely used tasks to assess cognitive processing in this context is the auditory oddball paradigm, where an infrequent change in a sequence of sounds elicits, in awake subjects, a characteristic EEG event-related potential called the mismatch negativity, followed by the classic P300 wave. The latter is further separable into the slightly earlier, anterior P3a and the later, posterior P3b, thought to be linked to task-irrelevant "bottom-up" and task-oriented "top-down" attention, respectively. We discuss here the putative dissociations between attention and awareness in DoC, sedation and sleep, bearing in mind the recently emerging evidence from healthy volunteers and patients. These findings highlight the neurophysiological and cognitive parallels (and differences) across these three distinct variations in levels of consciousness, and inform the theoretical framework for interpreting the role of attention therein.

Electromagnetic correlates of musical expertise in processing of tone patterns

Using magnetoencephalography (MEG), we investigated the influence of long term musical training on the processing of partly imagined tone patterns (imagery condition) compared to the same perceived patterns (perceptual condition). The magnetic counterpart of the mismatch negativity (MMNm) was recorded and compared between musicians and non-musicians in order to assess the effect of musical training on the detection of deviants to tone patterns. The results indicated a clear MMNm in the perceptual condition as well as in a simple pitch oddball (control) condition in both groups. However, there was no significant mismatch response in either group in the imagery condition despite above chance behavioral performance in the task of detecting deviant tones. The latency and the laterality of the MMNm in the perceptual condition differed significantly between groups, with an earlier MMNm in musicians, especially in the left hemisphere. In contrast the MMNm amplitudes did not differ significantly between groups. The behavioral results revealed a clear effect of long-term musical training in both experimental conditions. The obtained results represent new evidence that the processing of tone patterns is faster and more strongly lateralized in musically trained subjects, which is consistent with other findings in different paradigms of enhanced auditory neural system functioning due to long-term musical training.

Effects of a violation of an expected increase or decrease in intensity on detection of change within an auditory pattern

An infrequent physical increase in the intensity of an auditory stimulus relative to an already loud frequently occurring "standard" is processed differently than an equally perceptible physical decrease in intensity. This may be because a physical increment results in increased activation in two different systems, a transient and a change detector system (signalling detection of an increase in transient energy and a change from the past, respectively). By contrast, a decrease in intensity results in increased activation in only the change detector system. The major question asked by the present study was whether a psychological (rather than a physical) increment would continue to be processed differently than a psychological decrement when both stimuli activated only the change detector system. Participants were presented with a sequence of 1000 Hz tones that followed a standard rule-based alternating high-low intensity pattern (LHLHLH). They were asked to watch a silent video and thus ignore the auditory stimuli. A rare "deviant" was created by repeating one of the stimuli (e.g., LHLHLLLH. The repetition of the high intensity stimulus thus acted as a relative, psychological increment compared to what the rule would have predicted (the low intensity); the repetition of the low intensity stimulus acted as a relative, psychological decrement compared to what the rule would have predicted (the high intensity). In different conditions, the intensity difference between the low and high intensity tones was either 3, 9 or 27 dB. A large MMN was elicited only when the separation between the low and high intensities was 27dB. Importantly, this MMN peaked significantly earlier and its amplitude was significantly larger following presentation of the psychological increment. Thus, a deviant representing an increment in intensity relative to what would be predicted by the auditory past is processed differently than a deviant representing a decrement, even when activation of the transient detector system is controlled. The psychological increment did not however elicit a later positivity, the P3a, often thought to reflect the interruption of the central executive and a forced switching of attention.

Electrophysiological evidence for age effects on sensory memory processing of tonal patterns

In older adults, difficulties processing complex auditory scenes, such as speech comprehension in noisy environments, might be due to a specific impairment of temporal processing at early, automatic processing stages involving auditory sensory memory (ASM). Even though age effects on auditory temporal processing have been well-documented, there is a paucity of research on how ASM processing of more complex tone-patterns is altered by age. In the current study, age effects on ASM processing of temporal and frequency aspects of two-tone patterns were investigated using a passive listening protocol. The P1 component, the mismatch negativity (MMN) and the P3a component of event-related brain potentials (ERPs) to tone frequency and temporal pattern deviants were recorded in younger and older adults as a measure of auditory event detection, ASM processing, and attention switching, respectively. MMN was elicited with smaller amplitude to both frequency and temporal deviants in older adults. Furthermore, P3a was elicited only in the younger adults. In conclusion, the smaller MMN amplitude indicates that automatic processing of both frequency and temporal aspects of two-tone patterns is impaired in older adults. The failure to initiate an attention switch, suggested by the absence of P3a, indicates that impaired ASM processing of patterns may lead to less distractibility in older adults. Our results suggest age-related changes in ASM processing of patterns that cannot be explained by an inhibitory deficit.

Musicians and the Metric Structure of Words

The present study aimed to examine the influence of musical expertise on the metric and semantic aspects of speech processing. In two attentional conditions (metric and semantic tasks), musicians listened to short sentences ending in trisyllabic words that were semantically and/or metrically congruous or incongruous. Both ERPs and behavioral data were analyzed and the results were compared to previous nonmusicians' data. Regarding the processing of meter, results showed that musical expertise influenced the automatic detection of the syllable temporal structure (P200 effect), the integration of metric structure and its influence on word comprehension (N400 effect), as well as the reanalysis of metric violations (P600 and late positivities effects). By contrast, results showed that musical expertise did not influence the semantic level of processing. These results are discussed in terms of transfer of training effects from music to speech processing.

Saccadic context indicates information processing within visual fixations: evidence from event-related potentials and eye-movements analysis of the distractor effect

Attention, visual information processing, and oculomotor control are integrated functions of closely related brain mechanisms. Recently, it was shown that the processing of visual distractors appearing during a fixation is modulated by the amplitude of its preceding saccade (Pannasch & Velichkovsky, 2009). So far, this was demonstrated only at the behavioral level in terms of saccadic inhibition. The present study investigated distractor-related brain activity with cortical eye fixation-related potentials (EFRPs). Moreover, the following saccade was included as an additional classification criterion. Eye movements and EFRPs were recorded during free visual exploration of paintings. During some of the fixations, a visual distractor was shown as an annulus around the fixation position, 100 ms after the fixation onset. The saccadic context of a fixation was classified by its preceding and following saccade amplitudes with the cut-off criterion set to 4° of visual angle. The prolongation of fixation duration induced by distractors was largest for fixations preceded and followed by short saccades. EFRP data revealed a difference in distractor-related P2 amplitude between the saccadic context conditions, following the same trend as in eye movements. Furthermore, influences of the following saccade amplitude on the latency of the saccadic inhibition and on the N1 amplitude were found. The EFRP results cannot be explained by the influence of saccades per se since this bias was removed by subtracting the baseline from the distractor EFRP. Rather, the data suggest that saccadic context indicates differences in how information is processed within single visual fixations.

Human Visual System Automatically Encodes Sequential Regularities of Discrete Events

For our adaptive behavior in a dynamically changing environment, an essential task of the brain is to automatically encode sequential regularities inherent in the environment into a memory representation. Recent studies in neuroscience have suggested that sequential regularities embedded in discrete sensory events are automatically encoded into a memory representation at the level of the sensory system. This notion is largely supported by evidence from investigations using auditory mismatch negativity (auditory MMN), an event-related brain potential (ERP) correlate of an automatic memory-mismatch process in the auditory sensory system. However, it is still largely unclear whether or not this notion can be generalized to other sensory modalities. The purpose of the present study was to investigate the contribution of the visual sensory system to the automatic encoding of sequential regularities using visual mismatch negativity (visual MMN), an ERP correlate of an automatic memory-mismatch process in the visual sensory system. To this end, we conducted a sequential analysis of visual MMN in an oddball sequence consisting of infrequent deviant and frequent standard stimuli, and tested whether the underlying memory representation of visual MMN generation contains only a sensory memory trace of standard stimuli (trace-mismatch hypothesis) or whether it also contains sequential regularities extracted from the repetitive standard sequence (regularity-violation hypothesis). The results showed that visual MMN was elicited by first deviant (deviant stimuli following at least one standard stimulus), second deviant (deviant stimuli immediately following first deviant), and first standard (standard stimuli immediately following first deviant), but not by second standard (standard stimuli immediately following first standard). These results are consistent with the regularity-violation hypothesis, suggesting that the visual sensory system automatically encodes sequential regularities. In combination with a wide range of auditory MMN studies, the present study highlights the critical role of sensory systems in automatically encoding sequential regularities when modeling the world.