Modeling the auditory scene: predictive regularity representations and perceptual objects

Evoked Response Strength in Primary Auditory Cortex Predicts Performance in a Spectro-Spatial Discrimination Task in Rats

The extent to which the primary auditory cortex (A1) participates in instructing animal behavior remains debated. Although multiple studies have shown A1 activity to correlate with animals' perceptual judgments (Jaramillo and Zador, 2011; Bizley et al., 2013; Rodgers and DeWeese, 2014), others have found no relationship between A1 responses and reported auditory percepts (Lemus et al., 2009; Dong et al., 2011). To address this ambiguity, we performed chronic recordings of evoked local field potentials (eLFPs) in A1 of head-fixed female rats performing a two-alternative forced-choice auditory discrimination task. Rats were presented with two interleaved sequences of pure tones from opposite sides and had to indicate the side from which the higher-frequency target stimulus was played. Animal performance closely correlated (r _rm = 0.68) with the difference between the target and distractor eLFP responses: the more the target response exceeded the distractor response, the better the animals were at identifying the side of the target frequency. Reducing the evoked response of either frequency through stimulus-specific adaptation affected performance in the expected way: target localization accuracy was degraded when the target frequency was adapted and improved when the distractor frequency was adapted. Target frequency eLFPs were stronger on hit trials than on error trials. Our results suggest that the degree to which one stimulus stands out over others within A1 activity may determine its perceptual saliency for the animals and accordingly bias their behavioral choices.SIGNIFICANCE STATEMENT The brain must continuously calibrate the saliency of sensory percepts against their relevance to the current behavioral goal. The inability to ignore irrelevant distractors characterizes a spectrum of human attentional disorders. Meanwhile, the connection between the neural underpinnings of stimulus saliency and sensory decisions remains elusive. Here, we record local field potentials in the primary auditory cortex of rats engaged in auditory discrimination to investigate how the cortical representation of target and distractor stimuli impacts behavior. We find that the amplitude difference between target- and distractor-evoked activity predicts discrimination performance (r _rm = 0.68). Specific adaptation of target or distractor shifts performance either below or above chance, respectively. It appears that recent auditory history profoundly influences stimulus saliency, biasing animals toward diametrically-opposed decisions.

The use of intracranial recordings to decode human language: Challenges and opportunities

Decoding speech from intracranial recordings serves two main purposes: understanding the neural correlates of speech processing and decoding speech features for targeting speech neuroprosthetic devices. Intracranial recordings have high spatial and temporal resolution, and thus offer a unique opportunity to investigate and decode the electrophysiological dynamics underlying speech processing. In this review article, we describe current approaches to decoding different features of speech perception and production - such as spectrotemporal, phonetic, phonotactic, semantic, and articulatory components - using intracranial recordings. A specific section is devoted to the decoding of imagined speech, and potential applications to speech prosthetic devices. We outline the challenges in decoding human language, as well as the opportunities in scientific and neuroengineering applications.

Effects of talker continuity and speech rate on auditory working memory

Speech processing is slower and less accurate when listeners encounter speech from multiple talkers compared to one continuous talker. However, interference from multiple talkers has been investigated only using immediate speech recognition or long-term memory recognition tasks. These tasks reveal opposite effects of speech processing time on speech recognition - while fast processing of multi-talker speech impedes immediate recognition, it also results in more abstract and less talker-specific long-term memories for speech. Here, we investigated whether and how processing multi-talker speech disrupts working memory maintenance, an intermediate stage between perceptual recognition and long-term memory. In a digit sequence recall task, listeners encoded seven-digit sequences and recalled them after a 5-s delay. Sequences were spoken by either a single talker or multiple talkers at one of three presentation rates (0-, 200-, and 500-ms inter-digit intervals). Listeners' recall was slower and less accurate for sequences spoken by multiple talkers than a single talker. Especially for the fastest presentation rate, listeners were less efficient when recalling sequences spoken by multiple talkers. Our results reveal that talker-specificity effects for speech working memory are most prominent when listeners must rapidly encode speech. These results suggest that, like immediate speech recognition, working memory for speech is susceptible to interference from variability across talkers. While many studies ascribe effects of talker variability to the need to calibrate perception to talker-specific acoustics, these results are also consistent with the idea that a sudden change of talkers disrupts attentional focus, interfering with efficient working-memory processing.

Sleep deprivation moderates neural processes associated with passive auditory capture

Sleep loss has a major effect on cognitive tasks that are dependent on the maintenance of active sustained attention. This study examines the effects of sleep deprivation on automatic information processing, more specifically, its effect on processes leading to involuntary auditory attention capture by task-irrelevant auditory events. Two experiments were run. In the first, 13 participants were totally sleep-deprived (TSD); in the second, 16 participants were partially sleep-deprived (PSD), sleeping only four hours. Event-related potentials were recorded while participants discriminated the duration of equiprobable short and long auditory tones. At rare times, a small change to the pitch of these stimuli occurred. This deviant was however irrelevant to the duration detection task. As expected, TSD had a significant effect on the attention-dependent duration detection task; performance was worse and the P3b, associated with ease of detection, was attenuated. PSD had a similar, but reduced effect. Critically, the small pitch deviant resulted in less behavioural distraction following TSD compared to normal sleep.Consistent with this, the P3a, associated with the attention capture process, was significantly reduced following both TSD and PSD. Processes related to the passive switching of attention to potentially critical, but unattended, stimulus events are moderated by sleep deprivation.

From random to regular: neural constraints on the emergence of isochronous rhythm during cultural transmission

A core design feature of human communication systems and expressive behaviours is their temporal organization. The cultural evolutionary origins of this feature remain unclear. Here, we test the hypothesis that regularities in the temporal organization of signalling sequences arise in the course of cultural transmission as adaptations to aspects of cortical function. We conducted two experiments on the transmission of rhythms associated with affective meanings, focusing on one of the most widespread forms of regularity in language and music: isochronicity. In the first experiment, we investigated how isochronous rhythmic regularities emerge and change in multigenerational signalling games, where the receiver (learner) in a game becomes the sender (transmitter) in the next game. We show that signalling sequences tend to become rhythmically more isochronous as they are transmitted across generations. In the second experiment, we combined electroencephalography (EEG) and two-player signalling games over 2 successive days. We show that rhythmic regularization of sequences can be predicted based on the latencies of the mismatch negativity response in a temporal oddball paradigm. These results suggest that forms of isochronicity in communication systems originate in neural constraints on information processing, which may be expressed and amplified in the course of cultural transmission.

The Neuronal Basis of Predictive Coding Along the Auditory Pathway: From the Subcortical Roots to Cortical Deviance Detection

In this review, we attempt to integrate the empirical evidence regarding stimulus-specific adaptation (SSA) and mismatch negativity (MMN) under a predictive coding perspective (also known as Bayesian or hierarchical-inference model). We propose a renewed methodology for SSA study, which enables a further decomposition of deviance detection into repetition suppression and prediction error, thanks to the use of two controls previously introduced in MMN research: the many-standards and the cascade sequences. Focusing on data obtained with cellular recordings, we explain how deviance detection and prediction error are generated throughout hierarchical levels of processing, following two vectors of increasing computational complexity and abstraction along the auditory neuraxis: from subcortical toward cortical stations and from lemniscal toward nonlemniscal divisions. Then, we delve into the particular characteristics and contributions of subcortical and cortical structures to this generative mechanism of hierarchical inference, analyzing what is known about the role of neuromodulation and local microcircuitry in the emergence of mismatch signals. Finally, we describe how SSA and MMN are occurring at similar time frame and cortical locations, and both are affected by the manipulation of N-methyl- D-aspartate receptors. We conclude that there is enough empirical evidence to consider SSA and MMN, respectively, as the microscopic and macroscopic manifestations of the same physiological mechanism of deviance detection in the auditory cortex. Hence, the development of a common theoretical framework for SSA and MMN is all the more recommendable for future studies. In this regard, we suggest a shared nomenclature based on the predictive coding interpretation of deviance detection.

Object-based attention in complex, naturalistic auditory streams

In vision, objects have been described as the 'units' on which non-spatial attention operates in many natural settings. Here, we test the idea of object-based attention in the auditory domain within ecologically valid auditory scenes, composed of two spatially and temporally overlapping sound streams (speech signal vs. environmental soundscapes in Experiment 1 and two speech signals in Experiment 2). Top-down attention was directed to one or the other auditory stream by a non-spatial cue. To test for high-level, object-based attention effects we introduce an auditory repetition detection task in which participants have to detect brief repetitions of auditory objects, ruling out any possible confounds with spatial or feature-based attention. The participants' responses were significantly faster and more accurate in the valid cue condition compared to the invalid cue condition, indicating a robust cue-validity effect of high-level, object-based auditory attention.

A Gestalt inference model for auditory scene segregation

Our current understanding of how the brain segregates auditory scenes into meaningful objects is in line with a Gestaltism framework. These Gestalt principles suggest a theory of how different attributes of the soundscape are extracted then bound together into separate groups that reflect different objects or streams present in the scene. These cues are thought to reflect the underlying statistical structure of natural sounds in a similar way that statistics of natural images are closely linked to the principles that guide figure-ground segregation and object segmentation in vision. In the present study, we leverage inference in stochastic neural networks to learn emergent grouping cues directly from natural soundscapes including speech, music and sounds in nature. The model learns a hierarchy of local and global spectro-temporal attributes reminiscent of simultaneous and sequential Gestalt cues that underlie the organization of auditory scenes. These mappings operate at multiple time scales to analyze an incoming complex scene and are then fused using a Hebbian network that binds together coherent features into perceptually-segregated auditory objects. The proposed architecture successfully emulates a wide range of well established auditory scene segregation phenomena and quantifies the complimentary role of segregation and binding cues in driving auditory scene segregation.

Is auditory distraction by changing-state and deviant sounds underpinned by the same mechanism? Evidence from pupillometry

The mere presence of task-irrelevant auditory stimuli is known to interfere with cognitive functioning. Disruption can be caused by changing auditory distractors (the changing-state effect) or by a sound that deviates from the auditory background (the deviation effect). The unitary account of auditory distraction explains both phenomena in terms of attentional capture whereas the duplex-mechanism account posits that they reflect two fundamentally different forms of distraction in which only the deviation effect is caused by attentional capture. To test these predictions, we exploited a physiological index of attention orienting: the pupillary dilation response (PDR). Participants performed visual serial recall while ignoring sequences of spoken letters. These sequences either comprised repeated or changing letters, and one letter could sometimes be replaced by pink noise (the deviant). Recall was poorer in both changing-state and deviant trials. Interestingly, the PDR was elicited by deviant sounds but not changing-state sounds, while a tonic increase in pupil size was found throughout changing-state trials. This physiological dissociation of the changing-state and the deviation effects suggests they are subtended by distinct mechanisms thereby procuring support for the duplex-mechanism account over the unitary account.

A Model for Statistical Regularity Extraction from Dynamic Sounds

To understand our surroundings, we effortlessly parse our sound environment into sound sources, extracting invariant information-or regularities-over time to build an internal representation of the world around us. Previous experimental work has shown the brain is sensitive to many types of regularities in sound, but theoretical models that capture underlying principles of regularity tracking across diverse sequence structures have been few and far between. Existing efforts often focus on sound patterns rather the stochastic nature of sequences. In the current study, we employ a perceptual model for regularity extraction based on a Bayesian framework that posits the brain collects statistical information over time. We show this model can be used to simulate various results from the literature with stimuli exhibiting a wide range of predictability. This model can provide a useful tool for both interpreting existing experimental results under a unified model and providing predictions for new ones using more complex stimuli.

Reduced late mismatch negativity and auditory sustained potential to rule-based patterns in schizophrenia

Complex rule-based auditory processing is abnormal in individuals with long-term schizophrenia (SZ), as demonstrated by reduced mismatch negativity (MMN) to deviants in rule-based patterns and reduced auditory sustained potential (ASP) that appears when grouping tones together. Together, this suggests deficits later in the auditory processing hierarchy in Sz. Here, MMN and ASP were elicited by deviations from a complex zig-zag pitch pattern that cannot be predicted by simple linear rules. Twenty-seven SZ and 26 matched healthy controls (HC) participated. Frequent groups of patterns contained eight tones that zig-zagged in a two-up one-down pitch-based paradigm. There were two deviant patterns: the final tone was either higher in pitch than expected (creating a jump in pitch) or was repeated. Simple MMN to pitch-deviants among repetitive tones was measured for comparison. Sz exhibited a smaller pitch MMN compared to HC as expected. HC produced a late MMN in response to the repeat and jump-deviant and a larger ASP to the standard group of tones, all of which were significantly blunted in SZ. In Sz, the amplitude of the late complex MMN was related to neuropsychological functioning, whereas ASP was not. ASP and late MMN did not significantly correlate in HC or in Sz, suggesting that they are not dependent on one another and may originate within distinct processing streams. Together, this suggests multiple deficits later in the auditory sensory-perceptual hierarchy in Sz, with impairments evident in both segmentation and deviance detection abilities.

Enhanced deviant responses in patterned relative to random sound sequences

The brain draws on knowledge of statistical structure in the environment to facilitate detection of new events. Understanding the nature of this representation is a key challenge in sensory neuroscience. Specifically, it is unknown whether real-time perception of rapidly-unfolding sensory signals is driven by a coarse or detailed representation of the proximal stimulus history. We recorded electroencephalography brain responses to frequency outliers in regularly-patterned (REG) versus random (RAND) tone-pip sequences which were generated anew on each trial. REG and RAND sequences were matched in frequency content and span, only differing in the specific order of the tone-pips. Stimuli were very rapid, limiting conscious reasoning in favour of automatic processing of regularity. Listeners were naïve and performed an incidental visual task. Outliers within REG evoked a larger response than matched outliers in RAND. These effects arose rapidly (within 80 msec) and were underpinned by distinct sources from those classically associated with frequency-based deviance detection. These findings are consistent with the notion that the brain continually maintains a detailed representation of ongoing sensory input and that this representation shapes the processing of incoming information. Predominantly auditory-cortical sources code for frequency deviance whilst frontal sources are associated with tracking more complex sequence structure.

Neuronal phase consistency tracks dynamic changes in acoustic spectral regularity

The brain parses the auditory environment into distinct sounds by identifying those acoustic features in the environment that have common relationships (e.g., spectral regularities) with one another and then grouping together the neuronal representations of these features. Although there is a large literature that tests how the brain tracks spectral regularities that are predictable, it is not known how the auditory system tracks spectral regularities that are not predictable and that change dynamically over time. Furthermore, the contribution of brain regions downstream of the auditory cortex to the coding of spectral regularity is unknown. Here, we addressed these two issues by recording electrocorticographic activity, while human patients listened to tone-burst sequences with dynamically varying spectral regularities, and identified potential neuronal mechanisms of the analysis of spectral regularities throughout the brain. We found that the degree of oscillatory stimulus phase consistency (PC) in multiple neuronal-frequency bands tracked spectral regularity. In particular, PC in the delta-frequency band seemed to be the best indicator of spectral regularity. We also found that these regularity representations existed in multiple regions throughout cortex. This widespread reliable modulation in PC - both in neuronal-frequency space and in cortical space - suggests that phase-based modulations may be a general mechanism for tracking regularity in the auditory system specifically and other sensory systems more generally. Our findings also support a general role for the delta-frequency band in processing the regularity of auditory stimuli.

Maturation of Speech-Sound ERPs in 5-6-Year-Old Children: A Longitudinal Study

The maturation of 5-6-year-old children's auditory discrimination - indicated by the development of the auditory event-related-potentials (ERPs) - has not been previously studied in longitudinal settings. For the first time, we present here the results based on extensive dataset collected from 75 children. We followed the 5- to 6-year-olds for 20 months and measured their ERPs four times with the same multifeature paradigm with phonemic stimuli. The amplitude of the mismatch negativity (MMN) response increased during this time for vowel, vowel duration and frequency changes. Furthermore, the P3a component started to mature toward adult-like positivity for the vowel, intensity and frequency deviants and the late discriminative negativity (LDN) component decreased with age for vowel and intensity deviants. All the changes in the components seemed to happen during the second follow-up year, when Finnish children are taught letter symbols and other preliminary academic skills before going to school at the age of seven. Therefore, further studies are needed to clarify if these changes in the auditory discrimination are purely age-related or due to increasing linguistic knowledge of the children.

Temporal Processing in Audition: Insights from Music

Music is a curious example of a temporally patterned acoustic stimulus, and a compelling pan-cultural phenomenon. This review strives to bring some insights from decades of music psychology and sensorimotor synchronization (SMS) literature into the mainstream auditory domain, arguing that musical rhythm perception is shaped in important ways by temporal processing mechanisms in the brain. The feature that unites these disparate disciplines is an appreciation of the central importance of timing, sequencing, and anticipation. Perception of musical rhythms relies on an ability to form temporal predictions, a general feature of temporal processing that is equally relevant to auditory scene analysis, pattern detection, and speech perception. By bringing together findings from the music and auditory literature, we hope to inspire researchers to look beyond the conventions of their respective fields and consider the cross-disciplinary implications of studying auditory temporal sequence processing. We begin by highlighting music as an interesting sound stimulus that may provide clues to how temporal patterning in sound drives perception. Next, we review the SMS literature and discuss possible neural substrates for the perception of, and synchronization to, musical beat. We then move away from music to explore the perceptual effects of rhythmic timing in pattern detection, auditory scene analysis, and speech perception. Finally, we review the neurophysiology of general timing processes that may underlie aspects of the perception of rhythmic patterns. We conclude with a brief summary and outlook for future research.

Cascade and no-repetition rules are comparable controls for the auditory frequency mismatch negativity in oddball tasks

The mismatch negativity (MMN) has been widely studied with oddball tasks to index processing of unexpected auditory change. The MMN is computed as the difference of deviant minus standard and is used to capture the pattern violation by the deviant. However, this oddball MMN is confounded because the deviant differs physically from the standard and is presented less often. To improve measurement, the same tone as the deviant is presented in a separate condition. This control tone is equiprobable with other tones and is used to compute a corrected MMN (deviant minus control). Typically, the tones are in random order except that consecutive tones are not identical (no-repetition rule). In contrast, a recent study on frequency MMN presented tones in a regular up-and-down sequence (cascade rule). If the cascade rule is detected more easily than the no-repetition rule, there should be a lower risk of a confounding MMN within the cascade condition. However, in previous research, the cascade and no-repetition conditions differed not only in the regularity of the tone sequence but also in number of tones, frequency range, and proportion of tones. We controlled for these differences to isolate effects of regularity in the tone sequence. Results of our preregistered analyses provided moderate evidence (BF₀₁ >6) that the corrected MMN did not differ between cascade and no-repetition conditions. These findings imply that no-repetition and cascade rules are processed similarly and that the no-repetition condition provides an adequate control in frequency MMN.

Low-Frequency Oscillatory Correlates of Auditory Predictive Processing in Cortical-Subcortical Networks: A MEG-Study

Emerging evidence supports the role of neural oscillations as a mechanism for predictive information processing across large-scale networks. However, the oscillatory signatures underlying auditory mismatch detection and information flow between brain regions remain unclear. To address this issue, we examined the contribution of oscillatory activity at theta/alpha-bands (4-8/8-13 Hz) and assessed directed connectivity in magnetoencephalographic data while 17 human participants were presented with sound sequences containing predictable repetitions and order manipulations that elicited prediction-error responses. We characterized the spectro-temporal properties of neural generators using a minimum-norm approach and assessed directed connectivity using Granger Causality analysis. Mismatching sequences elicited increased theta power and phase-locking in auditory, hippocampal and prefrontal cortices, suggesting that theta-band oscillations underlie prediction-error generation in cortical-subcortical networks. Furthermore, enhanced feedforward theta/alpha-band connectivity was observed in auditory-prefrontal networks during mismatching sequences, while increased feedback connectivity in the alpha-band was observed between hippocampus and auditory regions during predictable sounds. Our findings highlight the involvement of hippocampal theta/alpha-band oscillations towards auditory prediction-error generation and suggest a spectral dissociation between inter-areal feedforward vs. feedback signalling, thus providing novel insights into the oscillatory mechanisms underlying auditory predictive processing.

The Contribution of Primary Auditory Cortex to Auditory Categorization in Behaving Monkeys

The specific contribution of core auditory cortex to auditory perception –such as categorization– remains controversial. To identify a contribution of the primary auditory cortex (A1) to perception, we recorded A1 activity while monkeys reported whether a temporal sequence of tone bursts was heard as having a “small” or “large” frequency difference. We found that A1 had frequency-tuned responses that habituated, independent of frequency content, as this auditory sequence unfolded over time. We also found that A1 firing rate was modulated by the monkeys’ reports of “small” and “large” frequency differences; this modulation correlated with their behavioral performance. These findings are consistent with the hypothesis that A1 contributes to the processes underlying auditory categorization.

Top-down, contextual entrainment of neuronal oscillations in the auditory thalamocortical circuit

Our results indicate that nonhuman primates detect complex repeating acoustic sequences in a continuous auditory stream, which is an important precursor for human speech learning and perception. We demonstrate that oscillatory entrainment, known to support the attentive perception of rhythmic stimulus sequences, can occur for rhythms defined solely by stimulus context rather than physical boundaries. As opposed to acoustically driven entrainment by rhythmic tone sequences demonstrated previously, this form of entrainment relies on the brain’s ability to group auditory inputs based on their statistical regularities. The internally initiated, context-driven modulation of excitability in the medial pulvinar prior to A1 supports the notion of top-down entrainment.

Prior studies have shown that repetitive presentation of acoustic stimuli results in an alignment of ongoing neuronal oscillations to the sequence rhythm via oscillatory entrainment by external cues. Our study aimed to explore the neural correlates of the perceptual parsing and grouping of complex repeating auditory patterns that occur based solely on statistical regularities, or context. Human psychophysical studies suggest that the recognition of novel auditory patterns amid a continuous auditory stimulus sequence occurs automatically halfway through the first repetition. We hypothesized that once repeating patterns were detected by the brain, internal rhythms would become entrained, demarcating the temporal structure of these repetitions despite lacking external cues defining pattern on- or offsets. To examine the neural correlates of pattern perception, neuroelectric activity of primary auditory cortex (A1) and thalamic nuclei was recorded while nonhuman primates passively listened to streams of rapidly presented pure tones and bandpass noise bursts. At arbitrary intervals, random acoustic patterns composed of 11 stimuli were repeated five times without any perturbance of the constant stimulus flow. We found significant delta entrainment by these patterns in the A1, medial geniculate body, and medial pulvinar. In A1 and pulvinar, we observed a statistically significant, pattern structure-aligned modulation of neuronal firing that occurred earliest in the pulvinar, supporting the idea that grouping and detecting complex auditory patterns is a top-down, context-driven process. Besides electrophysiological measures, a pattern-related modulation of pupil diameter verified that, like humans, nonhuman primates consciously detect complex repetitive patterns that lack physical boundaries.

Beta oscillatory power modulation reflects the predictability of pitch change

Humans process highly dynamic auditory information in real time, and regularities in stimuli such as speech and music can aid such processing by allowing sensory predictions for upcoming events. Auditory sequences contain information about both the identity of sounds (what) and their timing (when they occur). Temporal prediction in isochronous sequences is reflected in neural oscillatory power modulation in the beta band (∼20 Hz). Specifically, power decreases (desynchronization) after tone onset and then increases (resynchronization) to reach a maximum around the expected time of the next tone. The current study investigates whether the predictability of the pitch of a tone (what) is also reflected in beta power modulation. We presented two isochronous auditory oddball sequences, each with 20% of tones at a deviant pitch. In one sequence the deviant tones occurred regularly every fifth tone (predictably), but in the other sequence they occurred pseudorandomly (unpredictably). We recorded the electroencephalogram (EEG) while participants listened passively to these sequences. The results showed that auditory beta power desynchronization was larger prior to a predictable than an unpredictable pitch change. A single-trial correlation analysis using linear mixed-effect (LME) models further showed that the deeper the pre-deviant beta desynchronization depth, the smaller the event-related P3a amplitude following the deviant, and this effect only occurred when the pitch change was predictable. Given that P3a is associated with attentional response to prediction error, larger beta desynchronization depth indicates better prediction of an upcoming deviant pitch. Thus, these findings suggest that beta oscillations reflect predictions for what in additional to when during dynamic auditory information processing.

Can't Ignore-Distraction by Task-Irrelevant Sounds in Early and Middle Childhood

Attention control abilities are relevant for learning success. Little is known about the development of audio-visual attention in early childhood. Four groups of children between the ages of 4 and 10 years and adults performed an audio-visual distraction paradigm (N = 106). Multilevel analyses revealed increased reaction times in a visual categorization task when task-irrelevant novel sounds were presented, demonstrating involuntary distraction of attention. This distraction effect decreased with age and significantly differed between age groups. In addition, the two youngest age groups responded with a delay in trials following a distractor trial, indicating delayed reallocation of attention to the task at hand. Results indicate a significant maturation of audio-visual attention control within a few years during early childhood that continues throughout middle childhood.

A feasibility study of predictable and unpredictable surf-like sounds for tinnitus therapy using personal music players

OBJECTIVE

To evaluate the feasibility of predictable or unpredictable amplitude-modulated sounds for tinnitus therapy.

DESIGN

The study consisted of two parts. (1) An adaptation experiment. Loudness level matches and rating scales (10-point) for loudness and distress were obtained at a silent baseline and at the end of three counterbalanced 30-min exposures (silence, predictable and unpredictable). (2) A qualitative 2-week sound therapy feasibility trial. Participants took home a personal music player (PMP).

STUDY SAMPLE

Part 1: 23 individuals with chronic tinnitus and part 2: seven individuals randomly selected from Part 1.

RESULTS

Self-reported tinnitus loudness and annoyance were significantly lower than baseline ratings after acute unpredictable sound exposure. Tinnitus annoyance ratings were also significantly lower than the baseline but the effect was small. The feasibility trial identified that participant preferences for sounds varied. Three participants did not obtain any benefit from either sound. Three participants preferred unpredictable compared to predictable sounds. Some participants had difficulty using the PMP, the average self-report hours of use were low (less <1 h/day).

CONCLUSIONS

Unpredictable surf-like sounds played using a PMP is a feasible tinnitus treatment. Further work is required to improve the acceptance of the sound and ease of PMP use.

Taking Attention Away from the Auditory Modality: Context-dependent Effects on Early Sensory Encoding of Speech

Increasing visual perceptual load can reduce pre-attentive auditory cortical activity to sounds, a reflection of the limited and shared attentional resources for sensory processing across modalities. Here, we demonstrate that modulating visual perceptual load can impact the early sensory encoding of speech sounds, and that the impact of visual load is highly dependent on the predictability of the incoming speech stream. Participants (n = 20, 9 females) performed a visual search task of high (target similar to distractors) and low (target dissimilar to distractors) perceptual load, while early auditory electrophysiological responses were recorded to native speech sounds. Speech sounds were presented either in a 'repetitive context', or a less predictable 'variable context'. Independent of auditory stimulus context, pre-attentive auditory cortical activity was reduced during high visual load, relative to low visual load. We applied a data-driven machine learning approach to decode speech sounds from the early auditory electrophysiological responses. Decoding performance was found to be poorer under conditions of high (relative to low) visual load, when the incoming acoustic stream was predictable. When the auditory stimulus context was less predictable, decoding performance was substantially greater for the high (relative to low) visual load conditions. Our results provide support for shared attentional resources between visual and auditory modalities that substantially influence the early sensory encoding of speech signals in a context-dependent manner.

Neural Signatures of the Processing of Temporal Patterns in Sound

The ability to detect regularities in sound (i.e., recurring structure) is critical for effective perception, enabling, for example, change detection and prediction. Two seemingly unconnected lines of research concern the neural operations involved in processing regularities: one investigates how neural activity synchronizes with temporal regularities (e.g., frequency modulation; FM) in sounds, whereas the other focuses on increases in sustained activity during stimulation with repeating tone-frequency patterns. In three electroencephalography studies with male and female human participants, we investigated whether neural synchronization and sustained neural activity are dissociable, or whether they are functionally interdependent. Experiment I demonstrated that neural activity synchronizes with temporal regularity (FM) in sounds, and that sustained activity increases concomitantly. In Experiment II, phase coherence of FM in sounds was parametrically varied. Although neural synchronization was more sensitive to changes in FM coherence, such changes led to a systematic modulation of both neural synchronization and sustained activity, with magnitude increasing as coherence increased. In Experiment III, participants either performed a duration categorization task on the sounds, or a visual object tracking task to distract attention. Neural synchronization was observed regardless of task, whereas the sustained response was observed only when attention was on the auditory task, not under (visual) distraction. The results suggest that neural synchronization and sustained activity levels are functionally linked: both are sensitive to regularities in sounds. However, neural synchronization might reflect a more sensory-driven response to regularity, compared with sustained activity which may be influenced by attentional, contextual, or other experiential factors.SIGNIFICANCE STATEMENT Optimal perception requires that the auditory system detects regularities in sounds. Synchronized neural activity and increases in sustained neural activity both appear to index the detection of a regularity, but the functional interrelation of these two neural signatures is unknown. In three electroencephalography experiments, we measured both signatures concomitantly while listeners were presented with sounds containing frequency modulations that differed in their regularity. We observed that both neural signatures are sensitive to temporal regularity in sounds, although they functionally decouple when a listener is distracted by a demanding visual task. Our data suggest that neural synchronization reflects a more automatic response to regularity compared with sustained activity, which may be influenced by attentional, contextual, or other experiential factors.

Detecting change in stochastic sound sequences

Our ability to parse our acoustic environment relies on the brain's capacity to extract statistical regularities from surrounding sounds. Previous work in regularity extraction has predominantly focused on the brain's sensitivity to predictable patterns in sound sequences. However, natural sound environments are rarely completely predictable, often containing some level of randomness, yet the brain is able to effectively interpret its surroundings by extracting useful information from stochastic sounds. It has been previously shown that the brain is sensitive to the marginal lower-order statistics of sound sequences (i.e., mean and variance). In this work, we investigate the brain's sensitivity to higher-order statistics describing temporal dependencies between sound events through a series of change detection experiments, where listeners are asked to detect changes in randomness in the pitch of tone sequences. Behavioral data indicate listeners collect statistical estimates to process incoming sounds, and a perceptual model based on Bayesian inference shows a capacity in the brain to track higher-order statistics. Further analysis of individual subjects' behavior indicates an important role of perceptual constraints in listeners' ability to track these sensory statistics with high fidelity. In addition, the inference model facilitates analysis of neural electroencephalography (EEG) responses, anchoring the analysis relative to the statistics of each stochastic stimulus. This reveals both a deviance response and a change-related disruption in phase of the stimulus-locked response that follow the higher-order statistics. These results shed light on the brain's ability to process stochastic sound sequences.

Discontinuity of early and late event-related brain potentials for selective attention in dichotic listening

If a representation of an auditory attention channel was present in the auditory cortices but not in the subcortical structures, it would be predicted that the early event-related brain potential (ERP) would disagree with the late ERP in selective attention effects. To examine this idea, the present study recorded the auditory brain stem response (ABR) as an early ERP and also the negative difference, the processing negativity and the irrelevant positive difference waves as late ERPs during dichotic listening. Each participant experienced two dichotic conditions: (i) 500-Hz standard tones to the left ear and 1000-Hz ones to the right ear (L500/R1000), (ii) 1000-Hz standard tones to the left ear and 500-Hz ones to the right ear (L1000/R500). In a control task, participants performed visual detection and ignored auditory stimuli. Although the negative difference and processing negativity were found to be identical between the two dichotic conditions, the ABR demonstrated a significant difference between relevant and irrelevant tasks only for the L500/R1000 condition. A response preference to lower-frequency tones was found for behavioural measures and late ERPs but not for the ABR. These results suggest difficulty in representing attention channels in the auditory brain stem. In addition, a weak effect of dichotic sound combination in behaviours corresponded only with earlier ERPs.

Processing of interaural phase differences in components of harmonic and mistuned complexes in the inferior colliculus of the Mongolian gerbil

Harmonicity and spatial location provide eminent cues for the perceptual grouping of sounds. In general, harmonicity is a strong grouping cue. In contrast, spatial cues such as interaural phase or time difference provide for strong grouping of stimulus sequences but weak grouping for simultaneously presented sounds. By studying the neuronal basis underlying the interaction of these cues in processing simultaneous sounds using van Rossum spike train distance measures, we aim at explaining the interaction observed in psychophysical experiments. Responses to interaural phase differences imposed on single components of harmonic and mistuned complex tones as well as noise delay functions were recorded as multiunit responses from the inferior colliculus of Mongolian gerbils. Results revealed a better representation of interaural phase differences if imposed on a harmonic rather than a mistuned frequency component of a complex tone. The representation of interaural phase differences was better for long integration-time windows approximately reflecting firing rates rather than short integration-time windows reflecting the temporal pattern of the stimulus-driven response. We found only a weak impact of interaural phase differences if combined with mistuning of a component in a harmonic tone complex.

Predictive coding in auditory perception: challenges and unresolved questions

Predictive coding is arguably the currently dominant theoretical framework for the study of perception. It has been employed to explain important auditory perceptual phenomena, and it has inspired theoretical, experimental and computational modelling efforts aimed at describing how the auditory system parses the complex sound input into meaningful units (auditory scene analysis). These efforts have uncovered some vital questions, addressing which could help to further specify predictive coding and clarify some of its basic assumptions. The goal of the current review is to motivate these questions and show how unresolved issues in explaining some auditory phenomena lead to general questions of the theoretical framework. We focus on experimental and computational modelling issues related to sequential grouping in auditory scene analysis (auditory pattern detection and bistable perception), as we believe that this is the research topic where predictive coding has the highest potential for advancing our understanding. In addition to specific questions, our analysis led us to identify three more general questions that require further clarification: (1) What exactly is meant by prediction in predictive coding? (2) What governs which generative models make the predictions? and (3) What (if it exists) is the correlate of perceptual experience within the predictive coding framework?

Event-Related Potential Measures of Attention Capture in Adolescent Inpatients With Acute Suicidal Behavior

Impaired executive functions, modulated by the frontal lobes, have been suggested to be associated with suicidal behavior. The present study examines one of these executive functions, attentional control, maintaining attention to the task-at-hand. A group of inpatient adolescents with acute suicidal behavior and healthy controls were studied using a passively presented auditory optimal paradigm. This "optimal" paradigm consisted of a series of frequently presented homogenous pure tone "standards" and different "deviants," constructed by changing one or more features of the standard. The optimal paradigm has been shown to be a more time-efficient replacement to the traditional oddball paradigm, which makes it suitable for use in clinical populations. The extent of processing of these "to-be-ignored" auditory stimuli was measured by recording event-related potentials (ERPs). The P3a ERP component is thought to reflect processes associated with the capturing of attention. Rare and novel stimuli may result in an executive decision to switch attention away from the current cognitive task and toward a probe of the potentially more relevant "interrupting" auditory input. On the other hand, stimuli that are quite similar to the standard should not elicit P3a. The P3a has been shown to be larger in immature brains in early compared to later adolescence. An overall enhanced P3a was observed in the suicidal group. The P3a was larger in this group for both the environmental sound and white noise deviants, although only the environmental sound P3a attained significance. Other deviants representing only a small change from the standard did not elicit a P3a in healthy controls. They did elicit a small P3a in the suicidal group. These findings suggest a lowered threshold for the triggering of the involuntary switch of attention in these patients, which may play a role in their reported distractibility. The enhanced P3a is also suggestive of an immature frontal central executive and may provide a promising marker for early identification of some of the risk factors for some of the cognitive difficulties linked to suicidality.

Predictive cues for auditory stream formation in humans and monkeys

Auditory perception is improved when stimuli are predictable, and this effect is evident in a modulation of the activity of neurons in the auditory cortex as shown previously. Human listeners can better predict the presence of duration deviants embedded in stimulus streams with fixed interonset interval (isochrony) and repeated duration pattern (regularity), and neurons in the auditory cortex of macaque monkeys have stronger sustained responses in the 60-140 ms post-stimulus time window under these conditions. Subsequently, the question has arisen whether isochrony or regularity in the sensory input contributed to the enhancement of the neuronal and behavioural responses. Therefore, we varied the two factors isochrony and regularity independently and measured the ability of human subjects to detect deviants embedded in these sequences as well as measuring the responses of neurons the primary auditory cortex of macaque monkeys during presentations of the sequences. The performance of humans in detecting deviants was significantly increased by regularity. Isochrony enhanced detection only in the presence of the regularity cue. In monkeys, regularity increased the sustained component of neuronal tone responses in auditory cortex while isochrony had no consistent effect. Although both regularity and isochrony can be considered as parameters that would make a sequence of sounds more predictable, our results from the human and monkey experiments converge in that regularity has a greater influence on behavioural performance and neuronal responses.

Neurons along the auditory pathway exhibit a hierarchical organization of prediction error

Perception is characterized by a reciprocal exchange of predictions and prediction error signals between neural regions. However, the relationship between such sensory mismatch responses and hierarchical predictive processing has not yet been demonstrated at the neuronal level in the auditory pathway. We recorded single-neuron activity from different auditory centers in anaesthetized rats and awake mice while animals were played a sequence of sounds, designed to separate the responses due to prediction error from those due to adaptation effects. Here we report that prediction error is organized hierarchically along the central auditory pathway. These prediction error signals are detectable in subcortical regions and increase as the signals move towards auditory cortex, which in turn demonstrates a large-scale mismatch potential. Finally, the predictive activity of single auditory neurons underlies automatic deviance detection at subcortical levels of processing. These results demonstrate that prediction error is a fundamental component of singly auditory neuron responses.

The effect of inhibition on stimulus-specific adaptation in the inferior colliculus

The inferior colliculus is a center of convergence for inhibitory and excitatory synaptic inputs that may be activated simultaneously by sound stimulation. Stimulus repetition may generate response habituation by changing the efficacy of neuron's synaptic inputs. Specialized IC neurons reduce their response to repetitive tones, but restore their firing when a different and infrequent tone occurs, a phenomenon known as stimulus specific adaptation. Here, using the microiontophoresis technique, we determined the role of GABA_A-, GABA_B-, and glycinergic receptors in stimulus-specific adaptation (SSA). We found that blockade of postsynaptic GABA_B receptors selectively modulated response adaptation to repetitive sounds, whereas blockade of presynaptic GABA_B receptors exerted a gain control effect on neuron excitability. Adaptation decreased when postsynaptic GABA_B receptors were blocked, but increased if the blockade affected the presynaptic GABA_B receptors. A dual, paradoxical effect was elicited by blockade of glycinergic receptors, i.e., both increase and decrease in adaptation. Moreover, simultaneous co-application of GABA_A, GABA_B, and glycinergic antagonists demonstrated that local GABA- and glycine-mediated inhibition contributes to only about 50% of SSA. Therefore, inhibition via chemical synapses dynamically modulate the strength and dynamics of stimulus-specific adaptation, but does not generate it.

The effect of illusionary perception on mismatch negativity (MMN): An electroencephalography study

Mismatch negativity (MMN) is a unique brain response elicited by any discernible change of features in a tone sequence. Although the occurrence of MMN is dependent upon the difference of a stimulus parameter, such as frequency or intensity, recent studies have suggested that MMN occurs as a result of a comparison between an internal representation created by perception and an incoming tone. The present study aimed to investigate MMN occurs based upon the physical properties of stimuli or as a result of the perception of the scale illusion. A scale illusion occurs during presentation of ascending and descending musical scales between C4 and C5. The tones of these scales are presented to the right and left ear alternately using a dichotic listening paradigm. Although the ascending/descending sequences are alternated between ears after each tone, we perceive the illusion of progressively ascending/descending tones as being separated by ear. The experiment was designed as an oddball task using the illusionary sequence and three different types of tone sequences as control conditions. Brain response to these sequences and infrequently presented deviants was measured using electroencephalography (EEG). All of the control sequences showed MMN in response to the deviant. However, the illusionary sequence did not result in a significant MMN. These results suggest that in the case of scale illusion, the occurrence of MMN is based upon the representation of tones created by perception, but not upon the physical properties of a tone sequence.

Rhythmic Density Affects Listeners' Emotional Response to Microtiming

Microtiming has been assumed to be vital for the experience of groove, but past research presented conflicting results: some studies found that microtiming is irrelevant for groove, others reported that microtiming has a detrimental effect on the groove experience, yet others described circumstances under which microtiming has no negative impact on groove. The three studies in this paper aim at explaining some of these discrepancies by clarifying to what extent listeners' emotional responses to microtiming depend on the distribution of microtiming deviations across instrumental parts (voicing) or other moderating factors like tempo or rhythmic density. The studies use data from two listening experiments involving expert bass and drums duo recordings in swing and funk style. - Study A investigates the effect of fixed time displacements within and between the parts played by different musicians. Listeners (n = 160) reacted negatively to irregularities within the drum track, but the mutual displacement of bass vs. drums did not have an effect.- Study B develops three metrics to calculate the average microtiming magnitude in a musical excerpt. The experiment showed that listeners' (n = 160) emotional responses to expert performance microtiming aligned with each other across styles, when microtiming magnitude was adjusted for rhythmic density. This indicates that rhythmic density is a unifying moderator for listeners' emotional response to microtiming in swing and funk.- Study C used the data from both experiments in order to compare the effect of fixed microtiming displacements (from Study A) with scaled versions of the originally performed microtiming patterns (from Study B). It showed that fixed snare drum displacements irritated expert listeners more than the more flexible deviations occurring in the original performances. This provides some evidence that listeners' emotional response to microtiming deviations not only depends on the magnitude of the deviations, but also on the kind and origin of the microtiming patterns (fixed lab displacements vs. flexible performance microtiming).

The implicit learning of metrical and non-metrical rhythms in blind and sighted adults

Forming temporal expectancies plays a crucial role in our survival as it allows us to identify the occurrence of temporal deviants that might signal potential dangers. The dynamic attending theory suggests that temporal expectancies are formed more readily for rhythms that imply a beat (i.e., metrical rhythms) compared to those that do not (i.e., nonmetrical rhythms). Moreover, metrical frameworks can be used to detect temporal deviants. Although several studies have demonstrated that congenital or early blindness correlates with modality-specific neural changes that reflect compensatory mechanisms, few have examined whether blind individuals show a learning advantage for auditory rhythms and whether learning can occur unintentionally and without awareness, that is, implicitly. We compared blind to sighted controls in their ability to implicitly learn metrical and nonmetrical auditory rhythms. We reasoned that the loss of sight in blindness might lead to improved sensitivity to rhythms and predicted that the blind learn rhythms more readily than the sighted. We further hypothesized that metrical rhythms are learned more readily than nonmetrical rhythms. Results partially confirmed our predictions; the blind group learned nonmetrical rhythms more readily than the sighted group but the blind group learned metrical rhythms less readily than the sighted group. Only the sighted group learned metrical rhythms more readily than nonmetrical rhythms. The blind group demonstrated awareness of the nonmetrical rhythms while learning was implicit for all other conditions. Findings suggest that improved deviant-sensitivity might have provided the blind group a learning advantage for nonmetrical rhythms. Future research could explore the plastic changes that affect deviance-detection and stimulus-specific adaptation in blindness.

Mismatch negativity to pitch pattern deviants in schizophrenia

Simple mismatch negativity (MMN) to infrequent pitch deviants is impaired in individuals with long-term schizophrenia (Sz). The complex MMN elicited by pattern deviance often manifes is cut from here]->ts later after deviant onset than simple MMN and can ascertain deficits in abstracting relationships between stimuli. Sz exhibit reduced complex MMN, but so far this has only been measured when deviance detection relies on a grouping rule. We measured MMN to deviants in pitch-based rules to see whether MMN is also abnormal in Sz under these conditions. Three experiments were conducted. Twenty-seven Sz and 28 healthy matched controls (HC) participated in Experiments 1 and 2, and 24 Sz and 26 HC participated in Experiment 3. Experiment 1 was a standard pitch MMN task, and Sz showed the expected MMN reduction (~ 115 ms) in the simple pitch deviant compared to HC. Experiment 2 comprised standard groups of six tones that ascended in pitch, and deviant groups where the last tone descended in pitch. Complex MMN was late (~ 510 ms) and significantly blunted in Sz. Experiment 3 comprised standard groups of 12 tones (six tones ascending in pitch followed by six tones descending in pitch, like a scale), and deviant groups containing two repetitions of six ascending tones (the scale restarted midstream). Complex MMN was also late (~ 460 ms) and significantly blunted in Sz. These results identify a late pitch pattern deviance-related MMN that is deficient in schizophrenia. This suggests specific deficits in later more complex deviance detection in schizophrenia for abstract patterns.

Great Expectations: Is there Evidence for Predictive Coding in Auditory Cortex?

Predictive coding is possibly one of the most influential, comprehensive, and controversial theories of neural function. While proponents praise its explanatory potential, critics object that key tenets of the theory are untested or even untestable. The present article critically examines existing evidence for predictive coding in the auditory modality. Specifically, we identify five key assumptions of the theory and evaluate each in the light of animal, human and modeling studies of auditory pattern processing. For the first two assumptions - that neural responses are shaped by expectations and that these expectations are hierarchically organized - animal and human studies provide compelling evidence. The anticipatory, predictive nature of these expectations also enjoys empirical support, especially from studies on unexpected stimulus omission. However, for the existence of separate error and prediction neurons, a key assumption of the theory, evidence is lacking. More work exists on the proposed oscillatory signatures of predictive coding, and on the relation between attention and precision. However, results on these latter two assumptions are mixed or contradictory. Looking to the future, more collaboration between human and animal studies, aided by model-based analyses will be needed to test specific assumptions and implementations of predictive coding - and, as such, help determine whether this popular grand theory can fulfill its expectations.