Modeling the auditory scene: predictive regularity representations and perceptual objects

Pragmatic Prediction in the Processing of Referring Expressions Containing Scalar Quantifiers

Previous research in cognitive science and psycholinguistics has shown that language users are able to predict upcoming linguistic input probabilistically, pre-activating material on the basis of cues emerging from different levels of linguistic abstraction, from phonology to semantics. Current evidence suggests that linguistic prediction also operates at the level of pragmatics, where processing is strongly constrained by context. To test a specific theory of contextually-constrained processing, termed pragmatic surprisal theory here, we used a self-paced reading task where participants were asked to view visual scenes and then read descriptions of those same scenes. Crucially, we manipulated whether the visual context biased readers into specific pragmatic expectations about how the description might unfold word by word. Contrary to the predictions of pragmatic surprisal theory, we found that participants took longer reading the main critical term in scenarios where they were biased by context and pragmatic constraints to expect a given word, as opposed to scenarios where there was no pragmatic expectation for any particular referent.

A neural signature of regularity in sound is reduced in older adults

Sensitivity to repetitions in sound amplitude and frequency is crucial for sound perception. As with other aspects of sound processing, sensitivity to such patterns may change with age, and may help explain some age-related changes in hearing such as segregating speech from background sound. We recorded magnetoencephalography to characterize differences in the processing of sound patterns between younger and older adults. We presented tone sequences that either contained a pattern (made of a repeated set of tones) or did not contain a pattern. We show that auditory cortex in older, compared to younger, adults is hyperresponsive to sound onsets, but that sustained neural activity in auditory cortex, indexing the processing of a sound pattern, is reduced. Hence, the sensitivity of neural populations in auditory cortex fundamentally differs between younger and older individuals, overresponding to sound onsets, while underresponding to patterns in sounds. This may help to explain some age-related changes in hearing such as increased sensitivity to distracting sounds and difficulties tracking speech in the presence of other sound.

The Relationship Between Tinnitus Pitch, Audiogram Edge Frequency, and Auditory Stream Segregation Abilities in Individuals With Tinnitus

Purpose Around 80%-93% of the individuals with tinnitus have hearing loss. Researchers have found that tinnitus pitch was related to the frequencies of hearing loss, but unclear about the relationship between tinnitus pitch and audiometry edge frequency. The comorbidity of tinnitus and speech perception in noise problems had also been reported, but the relationship between tinnitus pitch and speech perception in noise had seldom been investigated. This study was designed to estimate the relationship between tinnitus pitch, audiogram edge frequency, and speech perception in noise. The speech perception in noise was measured using auditory stream segregation paradigm. Method Thirteen individuals with bilateral mild-to-severe tonal tinnitus and minimal-to-mild cochlear hearing loss were selected. Thirteen individuals with hearing loss without tinnitus were also selected. The audiogram of each participant with tinnitus was matched with that of the participant without tinnitus. Tinnitus pitch of the participants with tinnitus was measured and compared with audiogram edge frequency. The stream segregation thresholds were calculated at the participants' admitted tinnitus pitch and one octave below the tinnitus pitch. The stream segregation thresholds were estimated at fission and fusion boundary using pure-tone stimuli in ABA paradigm. Results High correlation between tinnitus pitch and audiogram edge frequency was noted. Overall stream segregation thresholds were higher for individuals with tinnitus. Higher thresholds indicated poorer stream segregation abilities. Within tinnitus participants, the thresholds were significantly lesser at frequency corresponding to admitted tinnitus pitch than at one octave below the tinnitus pitch. Conclusions The information from this study may be helpful in educating the patients about the relationship between hearing loss and tinnitus. The findings may also account for speech-perception-in-noise difficulties often reported by the individuals with tinnitus.

The influence of variability on mismatch negativity amplitude

Mismatch Negativity (MMN) to pattern deviations reveals exquisite pattern detection ability in the brain. MMN amplitude is proposed to be precision-weighted, being inversely proportional to variability within a patterned sound sequence. Two experiments were conducted to determine whether pattern variability, shown to influence MMN to simple pattern deviance, also extends to MMN elicited to abstract pattern deviants. Participants were presented with 3-tone triplet sequences that were defined by regular frequency ascendance with adjacent (A<B<C) or non-adjacent (A<C) dependency. The triplets were defined by an abstract pattern in that the starting frequency of A roamed randomly between 500-3700 Hz. Using variants of these sequences over two studies the results show that MMN was elicited to rare A > C deviants for adjacent and non-adjacent dependencies, was smaller for the latter, was impervious to variance in tone loudness, but showed prolonged sensitivity to the level of variability at sequence onset.

The effects of sleep on objective measures of gap detection using a time-efficient multi-deviant paradigm

Auditory temporal resolution, measured through gap detection, is critical for the perception of speech. A time-efficient multi-deviant paradigm has previously been developed for gap detection. The purpose of the present study was to determine if this multi-deviant paradigm could be used for gap detection during NREM sleep. ERPs were recorded in 10 young adults while awake and during the first two hours of NREM sleep. A multi-deviant paradigm was employed with six different deviants varying in gap duration, ranging from 2 to 40 ms. During waking, a DRN was observed for the 10, 20, 30 and 40 ms gaps. The DRN was absent during sleep. A P2 was present in NREM for the 20, 30 and 40 ms gaps followed by a P3a to the 30 and 40 ms gaps. An N350 was observed following the 10, 20, 30 and 40 ms gaps. Previous studies have reported significant ERPs to gaps having shorter durations than the present study. The multi-deviant paradigm may not be suitable for the determination of gap threshold during sleep. Nevertheless, it provides an exquisite means to determine perceptibility and the extent of processing of longer duration, supra-threshold gaps during sleep.

Sustained neural activity correlates with rapid perceptual learning of auditory patterns

Repeating structures forming regular patterns are common in sounds. Learning such patterns may enable accurate perceptual organization. In five experiments, we investigated the behavioral and neural signatures of rapid perceptual learning of regular sound patterns. We show that recurring (compared to novel) patterns are detected more quickly and increase sensitivity to pattern deviations and to the temporal order of pattern onset relative to a visual stimulus. Sustained neural activity reflected perceptual learning in two ways. Firstly, sustained activity increased earlier for recurring than novel patterns when participants attended to sounds, but not when they ignored them; this earlier increase mirrored the rapid perceptual learning we observed behaviorally. Secondly, the magnitude of sustained activity was generally lower for recurring than novel patterns, but only for trials later in the experiment, and independent of whether participants attended to or ignored sounds. The late manifestation of sustained activity reduction suggests that it is not directly related to rapid perceptual learning, but to a mechanism that does not require attention to sound. In sum, we demonstrate that the latency of sustained activity reflects rapid perceptual learning of auditory patterns, while the magnitude may reflect a result of learning, such as better prediction of learned auditory patterns.

Auditory Agnosia for Environmental Sounds in Alzheimer's Disease: Not Hearing and Not Listening?

Auditory agnosia for environmental sounds (AES) is an example of central auditory dysfunction. It is presumed to be independent of language deficits and in presence of normal hearing. We undertook a detailed neuropsychological assessment including environmental sound naming and recognition in 34 clinically mild Alzheimer's disease (AD) patients and 29 age-matched healthy control subjects. In patients with AD, audiometry was performed to assess the impact on test performance, and in normal controls the Hearing Handicap Inventory for the Elderly - Screening Version to exclude more than mild hearing loss. We adapted a validated environmental sound battery and found near perfect scores in controls. We found that environmental sound agnosia is common in mild AD. We found a statistically significant difference in mean pure tone audiometry in the best ear between patients with and those patients without naming deficits of 11.3 dB (p = 0.010) and of 14.7 dB (p = 0.000) between those with and without recognition deficits. Statistical significance remained after correcting for age, aphasia, Mini-Mental State Examination score, and working memory. Slight and moderate peripheral hearing loss increases the odds ratio of recognition deficits by 13.75 (confidence interval 2.3-81.5) compared to normal hearing patients. We did not find evidence for different forms of AES. This work suggests that an interaction between peripheral hearing loss and AD pathology produces problems with environmental sound recognition. It confirms that the relationship between hearing and dementia is complex but also suggests that interventions to prevent and treat hearing loss could have an effect on AD in its clinical expression.

Computational framework for investigating predictive processing in auditory perception

BACKGROUND

The brain tracks sound sources as they evolve in time, collecting contextual information to predict future sensory inputs. Previous work in predictive coding typically focuses on the perception of predictable stimuli, leaving the implementation of these same neural processes in more complex, real-world environments containing randomness and uncertainty up for debate.

NEW METHOD

To facilitate investigation into the perception of less tightly-controlled listening scenarios, we present a computational model as a tool to ask targeted questions about the underlying predictive processes that connect complex sensory inputs to listener behavior and neural responses. In the modeling framework, observed sound features (e.g. pitch) are tracked sequentially using Bayesian inference. Sufficient statistics are inferred from past observations at multiple time scales and used to make predictions about future observation while tracking the statistical structure of the sensory input.

RESULTS

Facets of the model are discussed in terms of their application to perceptual research, and examples taken from real-world audio demonstrate the model's flexibility to capture a variety of statistical structures along various perceptual dimensions.

COMPARISON WITH EXISTING METHODS

Previous models are often targeted toward interpreting a particular experimental paradigm (e.g., oddball paradigm), perceptual dimension (e.g., pitch processing), or task (e.g., speech segregation), thus limiting their ability to generalize to other domains. The presented model is designed as a flexible and practical tool for broad application.

CONCLUSION

The model is presented as a general framework for generating new hypotheses and guiding investigation into the neural processes underlying predictive coding of complex scenes.

Altered mismatch response precedes gray matter atrophy in subjective cognitive decline

The cross-sectional identification of subjective cognitive decline (SCD) in cognitively normal adults is particularly important for the early effective prevention or intervention of the future development of mild cognitive impairments (MCI) or Alzheimer's disease (AD). A pre-attentive neurophysiological signal that reflects the brain's ability to detect the changes of the environment is called mismatch negativity (MMN) or its magnetic counterpart (MMNm). It has been shown that patients with MCI or AD demonstrate reduced MMN/MMNm responses, while the exact profile of MMN/MMNm in SCD is substantially unknown. We applied magnetoencephalographic recordings to interrogate MMNm activities in healthy controls (HC, n = 29) and individuals with SCD (n = 26). Furthermore, we analyzed gray matter (GM) volumes in the MMNm-related regions through voxel-based morphometry and performed apolipoprotein E4 (APOE4) genotyping for all the participants. Our results showed that there were no significant differences in GM volume and proportions of APOE4 carriers between HC and SCD groups. However, individuals with SCD exhibited weakened z-corrected MMNm responses in the left inferior parietal lobule and right inferior frontal gyrus (IFG) as compared to HC. Based on the regions showing significant between-group differences, z-corrected MMNm amplitudes of the right IFG significantly correlated with the memory performance among the SCD participants. Our data suggest that neurophysiological changes of the brain, as indexed by MMNm, precede structural atrophy in the individuals with SCD compared to those without SCD.

Cross-modal predictive processing depends on context rather than local contingencies

Visual symbols or events may provide predictive information on to-be-expected sound events. When the perceived sound does not confirm the visual prediction, the incongruency response (IR), a prediction error signal of the event-related brain potentials, is elicited. It is unclear whether predictions are derived from lower-level local contingencies (e.g., recent events or repetitions) or from higher-level global rules applied top-down. In a recent study, sound pitch was predicted by a preceding note symbol. IR elicitation was confined to the condition where one of two sounds was presented more frequently and was not present with equal probability of both sounds. These findings suggest that local repetitions support predictive cross-modal processing. On the other hand, IR has also been observed with equal stimulus probabilities, where visual patterns predicted the upcoming sound sequence. This suggests the application of global rules. Here, we investigated the influence of stimulus repetition on the elicitation of the IR by presenting identical trial trains of a particular visual note symbol cueing a particular sound resulting either in a congruent or an incongruent pair. Trains of four different lengths: 1, 2, 4, or 7 were presented. The IR was observed already after a single presentation of a congruent visual-cue-sound combination and did not change in amplitude as trial train length increased. We conclude that higher-level associations applied in a top-down manner are involved in elicitation of the prediction error signal reflected by the IR, independent from local contingencies.

Propagation and update of auditory perceptual priors through alpha and theta rhythms

To maintain a continuous and coherent percept over time, the brain makes use of past sensory information to anticipate forthcoming stimuli. We recently showed that auditory experience of the immediate past is propagated through ear-specific reverberations, manifested as rhythmic fluctuations of decision bias at alpha frequencies. Here, we apply the same time-resolved behavioural method to investigate how perceptual performance changes over time under conditions of stimulus expectation and to examine the effect of unexpected events on behaviour. As in our previous study, participants were required to discriminate the ear-of-origin of a brief monaural pure tone embedded in uncorrelated dichotic white noise. We manipulated stimulus expectation by increasing the target probability in one ear to 80%. Consistent with our earlier findings, performance did not remain constant across trials, but varied rhythmically with delay from noise onset. Specifically, decision bias showed a similar oscillation at ~9 Hz, which depended on ear congruency between successive targets. This suggests rhythmic communication of auditory perceptual history occurs early and is not readily influenced by top-down expectations. In addition, we report a novel observation specific to infrequent, unexpected stimuli that gave rise to oscillations in accuracy at ~7.6 Hz one trial after the target occurred in the non-anticipated ear. This new behavioural oscillation may reflect a mechanism for updating the sensory representation once a prediction error has been detected.

Neural surprise in somatosensory Bayesian learning

Tracking statistical regularities of the environment is important for shaping human behavior and perception. Evidence suggests that the brain learns environmental dependencies using Bayesian principles. However, much remains unknown about the employed algorithms, for somesthesis in particular. Here, we describe the cortical dynamics of the somatosensory learning system to investigate both the form of the generative model as well as its neural surprise signatures. Specifically, we recorded EEG data from 40 participants subjected to a somatosensory roving-stimulus paradigm and performed single-trial modeling across peri-stimulus time in both sensor and source space. Our Bayesian model selection procedure indicates that evoked potentials are best described by a non-hierarchical learning model that tracks transitions between observations using leaky integration. From around 70ms post-stimulus onset, secondary somatosensory cortices are found to represent confidence-corrected surprise as a measure of model inadequacy. Indications of Bayesian surprise encoding, reflecting model updating, are found in primary somatosensory cortex from around 140ms. This dissociation is compatible with the idea that early surprise signals may control subsequent model update rates. In sum, our findings support the hypothesis that early somatosensory processing reflects Bayesian perceptual learning and contribute to an understanding of its underlying mechanisms.

Our environment features statistical regularities, such as a drop of rain predicting imminent rainfall. Despite the importance for behavior and survival, much remains unknown about how these dependencies are learned, particularly for somatosensation. As surprise signalling about novel observations indicates a mismatch between one’s beliefs and the world, it has been hypothesized that surprise computation plays an important role in perceptual learning. By analyzing EEG data from human participants receiving sequences of tactile stimulation, we compare different formulations of surprise and investigate the employed underlying learning model. Our results indicate that the brain estimates transitions between observations. Furthermore, we identified different signatures of surprise computation and thereby provide a dissociation of the neural correlates of belief inadequacy and belief updating. Specifically, early surprise responses from around 70ms were found to signal the need for changes to the model, with encoding of its subsequent updating occurring from around 140ms. These results provide insights into how somatosensory surprise signals may contribute to the learning of environmental statistics.

Dysfunctional frontal activation of mismatch negativity in panic disorder: A magnetoencephalographic study

BACKGROUND

Mismatch negativity (MMN) or its magnetic counterpart (MMNm) is a neurophysiological signal to reflect the automatic change-detection ability. However, MMN studies in patients with panic disorder (PD) showed contrasting results using electroencephalographic (EEG) recordings. The present study attempted to overcome the limitations of EEG methodology by means of a whole-head magnetoencephalography (MEG) combined with the depth-weighted minimum norm estimate method to conduct an in-depth investigation on the MMNm at the cortical level in patients with PD.

METHODS

We recruited 22 healthy controls (HC) and 20 patients with PD to perform auditory oddball paradigm during MEG recordings. The cortical MMNm amplitudes and latencies in the superior temporal gyrus, inferior parietal lobule, and inferior frontal gyrus (IFG) were compared between the HC and PD groups. The correlations between MMNm responses and clinical measurement were also examined.

RESULTS

Compared with the HC group, the PD group demonstrated significantly reduced MMNm amplitudes in the IFG. Furthermore, higher trait scores of the State-Trait Anxiety Inventory were associated with lower MMNm amplitudes of the right IFG among patients with PD.

LIMITATIONS

Generalization of the current results to other settings or samples should be made cautiously due to the use of different medication regimens and presence of comorbidities in our patients.

CONCLUSIONS

Our data suggest dysfunctional pre-attentive change-detection ability in patients with PD, particularly in the IFG.

Arcuate fasciculus architecture is associated with individual differences in pre-attentive detection of unpredicted music changes

The mismatch negativity (MMN) is an event related brain potential (ERP) elicited by unpredicted sounds presented in a sequence of repeated auditory stimuli. The neural sources of the MMN have been previously attributed to a fronto-temporo-parietal network which crucially overlaps with the so-called auditory dorsal stream, involving inferior and middle frontal, inferior parietal, and superior and middle temporal regions. These cortical areas are structurally connected by the arcuate fasciculus (AF), a three-branch pathway supporting the feedback-feedforward loop involved in auditory-motor integration, auditory working memory, storage of acoustic templates, as well as comparison and update of those templates. Here, we characterized the individual differences in the white-matter macrostructural properties of the AF and explored their link to the electrophysiological marker of passive change detection gathered in a melodic multifeature MMN-EEG paradigm in 26 healthy young adults without musical training. Our results show that left fronto-temporal white-matter connectivity plays an important role in the pre-attentive detection of rhythm modulations within a melody. Previous studies have shown that this AF segment is also critical for language processing and learning. This strong coupling between structure and function in auditory change detection might be related to life-time linguistic (and possibly musical) exposure and experiences, as well as to timing processing specialization of the left auditory cortex. To the best of our knowledge, this is the first time in which the relationship between neurophysiological (EEG) and brain white-matter connectivity indexes using DTI-tractography are studied together. Thus, the present results, although still exploratory, add to the existing evidence on the importance of studying the constraints imposed on cognitive functions by the underlying structural connectivity.

Extracting human cortical responses to sound onsets and acoustic feature changes in real music, and their relation to event rate

Evoked cortical responses (ERs) have mainly been studied in controlled experiments using simplified stimuli. Though, an outstanding question is how the human cortex responds to the complex stimuli encountered in realistic situations. Few electroencephalography (EEG) studies have used Music Information Retrieval (MIR) tools to extract cortical P1/N1/P2 to acoustical changes in real music. However, less than ten events per music piece could be detected leading to ERs due to limitations in automatic detection of sound onsets. Also, the factors influencing a successful extraction of the ERs have not been identified. Finally, previous studies did not localize the sources of the cortical generators. This study is based on an EEG/MEG dataset from 48 healthy normal hearing participants listening to three real music pieces. Acoustic features were computed from the audio signal of the music with the MIR Toolbox. To overcome limits in automatic methods, sound onsets were also manually detected. The chance of obtaining detectable ERs based on ten randomly picked onset points was less than 1:10,000. For the first time, we show that naturalistic P1/N1/P2 ERs can be reliably measured across 100 manually identified sound onsets, substantially improving the signal-to-noise level compared to <10 trials. More ERs were measurable in musical sections with slow event rates (0.2 Hz-2.5 Hz) than with fast event rates (>2.5 Hz). Furthermore, during monophonic sections of the music only P1/P2 were measurable, and during polyphonic sections only N1. Finally, MEG source analysis revealed that naturalistic P2 is located in core areas of the auditory cortex.

Active listening

This paper introduces active listening, as a unified framework for synthesising and recognising speech. The notion of active listening inherits from active inference, which considers perception and action under one universal imperative: to maximise the evidence for our (generative) models of the world. First, we describe a generative model of spoken words that simulates (i) how discrete lexical, prosodic, and speaker attributes give rise to continuous acoustic signals; and conversely (ii) how continuous acoustic signals are recognised as words. The 'active' aspect involves (covertly) segmenting spoken sentences and borrows ideas from active vision. It casts speech segmentation as the selection of internal actions, corresponding to the placement of word boundaries. Practically, word boundaries are selected that maximise the evidence for an internal model of how individual words are generated. We establish face validity by simulating speech recognition and showing how the inferred content of a sentence depends on prior beliefs and background noise. Finally, we consider predictive validity by associating neuronal or physiological responses, such as the mismatch negativity and P300, with belief updating under active listening, which is greatest in the absence of accurate prior beliefs about what will be heard next.

The critical role of the inferior frontal cortex in establishing a prediction model for generating subsequent mismatch negativity (MMN): A TMS-EEG study

BACKGROUND

The prediction violation account of automatic or pre-attentive change detection assumed that the inferior frontal cortex (IFC) is involved in establishing a prediction model for detecting unexpected changes. Evidence supporting the IFC's contribution to prediction model is mainly based on the Mismatch Negativity (MMN) to deviants violating predictions that are established based on the frequently presented standard events. However, deviant detection involves processes, such as events comparison, other than prediction model establishment.

OBJECTIVE

The current study investigated the critical role of the IFC in establishing a prediction model during standards processing for subsequent deviant detection.

METHODS

Transcranial Magnetic Stimulation (TMS) was applied at the IFC to disrupt the processing of the initial 2 or 5 standards of a 3-, 6-, or 9-standard train, while the MMN responses to pitch deviant presented after the standard trains were recorded and compared.

RESULTS

An abolishment of MMN was only observed when TMS was delivered to the IFC at the initial 2 standards of the 3-standard train, but not at the initial 5 standards, or when TMS at the vertex or TMS sound recording was applied. The MMNs were also preserved when IFC TMS, vertex TMS, or TMS sound recording was applied at the initial 2 or 5 standards of longer trains.

CONCLUSION

The IFC plays a critical role in processing the initial standards of a short standard train for subsequent deviant detection. This result is consistent with the prediction violation account that the IFC is important for establishing the prediction model.

Sensitivity to temporal structure facilitates perceptual analysis of complex auditory scenes

•

Perception relies on sensitivity to predictable structure in the environment.

•

We used artificial acoustic scenes to investigate this in the auditory modality.

•

Listeners track the temporal structure of multiple concurrent acoustic streams.

•

Sensitivity to predictable structure supports auditory scene analysis, even when scenes are complex.

•

Benefit of regularity observed even when listeners are unaware of the predictable structure.

The notion that sensitivity to the statistical structure of the environment is pivotal to perception has recently garnered considerable attention. Here we investigated this issue in the context of hearing. Building on previous work (Sohoglu and Chait, 2016a; elife), stimuli were artificial ‘soundscapes’ populated by multiple (up to 14) simultaneous streams (‘auditory objects’) comprised of tone-pip sequences, each with a distinct frequency and pattern of amplitude modulation. Sequences were either temporally regular or random.

We show that listeners’ ability to detect abrupt appearance or disappearance of a stream is facilitated when scene streams were characterized by a temporally regular fluctuation pattern. The regularity of the changing stream as well as that of the background (non-changing) streams contribute independently to this effect. Remarkably, listeners benefit from regularity even when they are not consciously aware of it. These findings establish that perception of complex acoustic scenes relies on the availability of detailed representations of the regularities automatically extracted from multiple concurrent streams.

PPM-Decay: A computational model of auditory prediction with memory decay

Statistical learning and probabilistic prediction are fundamental processes in auditory cognition. A prominent computational model of these processes is Prediction by Partial Matching (PPM), a variable-order Markov model that learns by internalizing n-grams from training sequences. However, PPM has limitations as a cognitive model: in particular, it has a perfect memory that weights all historic observations equally, which is inconsistent with memory capacity constraints and recency effects observed in human cognition. We address these limitations with PPM-Decay, a new variant of PPM that introduces a customizable memory decay kernel. In three studies-one with artificially generated sequences, one with chord sequences from Western music, and one with new behavioral data from an auditory pattern detection experiment-we show how this decay kernel improves the model's predictive performance for sequences whose underlying statistics change over time, and enables the model to capture effects of memory constraints on auditory pattern detection. The resulting model is available in our new open-source R package, ppm (https://github.com/pmcharrison/ppm).

The Influence of Form- and Meaning-Based Predictions on Cortical Speech Processing Under Challenging Listening Conditions: A MEG Study

Under adverse listening conditions, prior linguistic knowledge about the form (i.e., phonology) and meaning (i.e., semantics) help us to predict what an interlocutor is about to say. Previous research has shown that accurate predictions of incoming speech increase speech intelligibility, and that semantic predictions enhance the perceptual clarity of degraded speech even when exact phonological predictions are possible. In addition, working memory (WM) is thought to have specific influence over anticipatory mechanisms by actively maintaining and updating the relevance of predicted vs. unpredicted speech inputs. However, the relative impact on speech processing of deviations from expectations related to form and meaning is incompletely understood. Here, we use MEG to investigate the cortical temporal processing of deviations from the expected form and meaning of final words during sentence processing. Our overall aim was to observe how deviations from the expected form and meaning modulate cortical speech processing under adverse listening conditions and investigate the degree to which this is associated with WM capacity. Results indicated that different types of deviations are processed differently in the auditory N400 and Mismatch Negativity (MMN) components. In particular, MMN was sensitive to the type of deviation (form or meaning) whereas the N400 was sensitive to the magnitude of the deviation rather than its type. WM capacity was associated with the ability to process phonological incoming information and semantic integration.

Early cortical processing of pitch height and the role of adaptation and musicality

Pitch is an important perceptual feature; however, it is poorly understood how its cortical correlates are shaped by absolute vs relative fundamental frequency (f₀), and by neural adaptation. In this study, we assessed transient and sustained auditory evoked fields (AEFs) at the onset, progression, and offset of short pitch height sequences, taking into account the listener's musicality. We show that neuromagnetic activity reflects absolute f₀ at pitch onset and offset, and relative f₀ at transitions within pitch sequences; further, sequences with fixed f₀ lead to larger response suppression than sequences with variable f₀ contour, and to enhanced offset activity. Musical listeners exhibit stronger f₀-related AEFs and larger differences between their responses to fixed vs variable sequences, both within sequences and at pitch offset. The results resemble prominent psychoacoustic phenomena in the perception of pitch contours; moreover, they suggest a strong influence of adaptive mechanisms on cortical pitch processing which, in turn, might be modulated by a listener's musical expertise.

Breaking down the cocktail party: Attentional modulation of cerebral audiovisual speech processing

Recent studies utilizing electrophysiological speech envelope reconstruction have sparked renewed interest in the cocktail party effect by showing that auditory neurons entrain to selectively attended speech. Yet, the neural networks of attention to speech in naturalistic audiovisual settings with multiple sound sources remain poorly understood. We collected functional brain imaging data while participants viewed audiovisual video clips of lifelike dialogues with concurrent distracting speech in the background. Dialogues were presented in a full-factorial design, comprising task (listen to the dialogues vs. ignore them), audiovisual quality and semantic predictability. We used univariate analyses in combination with multivariate pattern analysis (MVPA) to study modulations of brain activity related to attentive processing of audiovisual speech. We found attentive speech processing to cause distinct spatiotemporal modulation profiles in distributed cortical areas including sensory and frontal-control networks. Semantic coherence modulated attention-related activation patterns in the earliest stages of auditory cortical processing, suggesting that the auditory cortex is involved in high-level speech processing. Our results corroborate views that emphasize the dynamic nature of attention, with task-specificity and context as cornerstones of the underlying neuro-cognitive mechanisms.

The effect of healthy aging on change detection and sensitivity to predictable structure in crowded acoustic scenes

The auditory system plays a critical role in supporting our ability to detect abrupt changes in our surroundings. Here we study how this capacity is affected in the course of healthy ageing. Artifical acoustic 'scenes', populated by multiple concurrent streams of pure tones ('sources') were used to capture the challenges of listening in complex acoustic environments. Two scene conditions were included: REG scenes consisted of sources characterized by a regular temporal structure. Matched RAND scenes contained sources which were temporally random. Changes, manifested as the abrupt disappearance of one of the sources, were introduced to a subset of the trials and participants ('young' group N = 41, age 20-38 years; 'older' group N = 41, age 60-82 years) were instructed to monitor the scenes for these events. Previous work demonstrated that young listeners exhibit better change detection performance in REG scenes, reflecting sensitivity to temporal structure. Here we sought to determine: (1) Whether 'baseline' change detection ability (i.e. in RAND scenes) is affected by age. (2) Whether aging affects listeners' sensitivity to temporal regularity. (3) How change detection capacity relates to listeners' hearing and cognitive profile (a battery of tests that capture hearing and cognitive abilities hypothesized to be affected by aging). The results demonstrated that healthy aging is associated with reduced sensitivity to abrupt scene changes in RAND scenes but that performance does not correlate with age or standard audiological measures such as pure tone audiometry or speech in noise performance. Remarkably older listeners' change detection performance improved substantially (up to the level exhibited by young listeners) in REG relative to RAND scenes. This suggests that the ability to extract and track the regularity associated with scene sources, even in crowded acoustic environments, is relatively preserved in older listeners.

Pitch, Timbre and Intensity Interdependently Modulate Neural Responses to Salient Sounds

As we listen to everyday sounds, auditory perception is heavily shaped by interactions between acoustic attributes such as pitch, timbre and intensity; though it is not clear how such interactions affect judgments of acoustic salience in dynamic soundscapes. Salience perception is believed to rely on an internal brain model that tracks the evolution of acoustic characteristics of a scene and flags events that do not fit this model as salient. The current study explores how the interdependency between attributes of dynamic scenes affects the neural representation of this internal model and shapes encoding of salient events. Specifically, the study examines how deviations along combinations of acoustic attributes interact to modulate brain responses, and subsequently guide perception of certain sound events as salient given their context. Human volunteers have their attention focused on a visual task and ignore acoustic melodies playing in the background while their brain activity using electroencephalography is recorded. Ambient sounds consist of musical melodies with probabilistically-varying acoustic attributes. Salient notes embedded in these scenes deviate from the melody's statistical distribution along pitch, timbre and/or intensity. Recordings of brain responses to salient notes reveal that neural power in response to the melodic rhythm as well as cross-trial phase alignment in the theta band are modulated by degree of salience of the notes, estimated across all acoustic attributes given their probabilistic context. These neural nonlinear effects across attributes strongly parallel behavioral nonlinear interactions observed in perceptual judgments of auditory salience using similar dynamic melodies; suggesting a neural underpinning of nonlinear interactions that underlie salience perception.

Manipulation of low-level features modulates grouping strength of auditory objects

A central challenge of auditory processing involves the segregation, analysis, and integration of acoustic information into auditory perceptual objects for processing by higher order cognitive operations. This study explores the influence of low-level features on auditory object perception. Participants provided perceived musicality ratings in response to randomly generated pure tone sequences. Previous work has shown that music perception relies on the integration of discrete sounds into a holistic structure. Hence, high (versus low) ratings were viewed as indicative of strong (versus weak) object formation. Additionally, participants rated sequences in which random subsets of tones were manipulated along one of three low-level dimensions (timbre, amplitude, or fade-in) at one of three strengths (low, medium, or high). Our primary findings demonstrate how low-level acoustic features modulate the perception of auditory objects, as measured by changes in musicality ratings for manipulated sequences. Secondarily, we used principal component analysis to categorize participants into subgroups based on differential sensitivities to low-level auditory dimensions, thereby highlighting the importance of individual differences in auditory perception. Finally, we report asymmetries regarding the effects of low-level dimensions; specifically, the perceptual significance of timbre. Together, these data contribute to our understanding of how low-level auditory features modulate auditory object perception.

Selecting among competing models of talker adaptation: Attention, cognition, and memory in speech processing efficiency

Phonetic variability across talkers imposes additional processing costs during speech perception, often measured by performance decrements between single- and mixed-talker conditions. However, models differ in their predictions about whether accommodating greater phonetic variability (i.e., more talkers) imposes greater processing costs. We measured speech processing efficiency in a speeded word identification task, in which we manipulated the number of talkers (1, 2, 4, 8, or 16) listeners heard. Word identification was less efficient in every mixed-talker condition compared to the single-talker condition, but the magnitude of this performance decrement was not affected by the number of talkers. Furthermore, in a condition with uniform transition probabilities between two talkers, word identification was more efficient when the talker was the same as the prior trial compared to trials when the talker switched. These results support an auditory streaming model of talker adaptation, where processing costs associated with changing talkers result from attentional reorientation.

Neural Responses and Perceptual Sensitivity to Sound Depend on Sound-Level Statistics

Sensitivity to sound-level statistics is crucial for optimal perception, but research has focused mostly on neurophysiological recordings, whereas behavioral evidence is sparse. We use electroencephalography (EEG) and behavioral methods to investigate how sound-level statistics affect neural activity and the detection of near-threshold changes in sound amplitude. We presented noise bursts with sound levels drawn from distributions with either a low or a high modal sound level. One participant group listened to the stimulation while EEG was recorded (Experiment I). A second group performed a behavioral amplitude-modulation detection task (Experiment II). Neural activity depended on sound-level statistical context in two different ways. Consistent with an account positing that the sensitivity of neurons to sound intensity adapts to ambient sound level, responses for higher-intensity bursts were larger in low-mode than high-mode contexts, whereas responses for lower-intensity bursts did not differ between contexts. In contrast, a concurrent slow neural response indicated prediction-error processing: The response was larger for bursts at intensities that deviated from the predicted statistical context compared to those not deviating. Behavioral responses were consistent with prediction-error processing, but not with neural adaptation. Hence, neural activity adapts to sound-level statistics, but fine-tuning of perceptual sensitivity appears to involve neural prediction-error responses.

Long-term implicit memory for sequential auditory patterns in humans

Memory, on multiple timescales, is critical to our ability to discover the structure of our surroundings, and efficiently interact with the environment. We combined behavioural manipulation and modelling to investigate the dynamics of memory formation for rarely reoccurring acoustic patterns. In a series of experiments, participants detected the emergence of regularly repeating patterns within rapid tone-pip sequences. Unbeknownst to them, a few patterns reoccurred every ~3 min. All sequences consisted of the same 20 frequencies and were distinguishable only by the order of tone-pips. Despite this, reoccurring patterns were associated with a rapidly growing detection-time advantage over novel patterns. This effect was implicit, robust to interference, and persisted for 7 weeks. The results implicate an interplay between short (a few seconds) and long-term (over many minutes) integration in memory formation and demonstrate the remarkable sensitivity of the human auditory system to sporadically reoccurring structure within the acoustic environment.

Effect of Auditory Predictability on the Human Peripheral Auditory System

Auditory perception is facilitated by prior knowledge about the statistics of the acoustic environment. Predictions about upcoming auditory stimuli are processed at various stages along the human auditory pathway, including the cortex and midbrain. Whether such auditory predictions are processed also at hierarchically lower stages-in the peripheral auditory system-is unclear. To address this question, we assessed outer hair cell (OHC) activity in response to isochronous tone sequences and varied the predictability and behavioral relevance of the individual tones (by manipulating tone-to-tone probabilities and the human participants' task, respectively). We found that predictability alters the amplitude of distortion-product otoacoustic emissions (DPOAEs, a measure of OHC activity) in a manner that depends on the behavioral relevance of the tones. Simultaneously recorded cortical responses showed a significant effect of both predictability and behavioral relevance of the tones, indicating that their experimental manipulations were effective in central auditory processing stages. Our results provide evidence for a top-down effect on the processing of auditory predictability in the human peripheral auditory system, in line with previous studies showing peripheral effects of auditory attention.

Streaming of Repeated Noise in Primary and Secondary Fields of Auditory Cortex

Statistical regularities in natural sounds facilitate the perceptual segregation of auditory sources, or streams. Repetition is one cue that drives stream segregation in humans, but the neural basis of this perceptual phenomenon remains unknown. We demonstrated a similar perceptual ability in animals by training ferrets of both sexes to detect a stream of repeating noise samples (foreground) embedded in a stream of random samples (background). During passive listening, we recorded neural activity in primary auditory cortex (A1) and secondary auditory cortex (posterior ectosylvian gyrus, PEG). We used two context-dependent encoding models to test for evidence of streaming of the repeating stimulus. The first was based on average evoked activity per noise sample and the second on the spectro-temporal receptive field. Both approaches tested whether differences in neural responses to repeating versus random stimuli were better modeled by scaling the response to both streams equally (global gain) or by separately scaling the response to the foreground versus background stream (stream-specific gain). Consistent with previous observations of adaptation, we found an overall reduction in global gain when the stimulus began to repeat. However, when we measured stream-specific changes in gain, responses to the foreground were enhanced relative to the background. This enhancement was stronger in PEG than A1. In A1, enhancement was strongest in units with low sparseness (i.e., broad sensory tuning) and with tuning selective for the repeated sample. Enhancement of responses to the foreground relative to the background provides evidence for stream segregation that emerges in A1 and is refined in PEG.SIGNIFICANCE STATEMENT To interact with the world successfully, the brain must parse behaviorally important information from a complex sensory environment. Complex mixtures of sounds often arrive at the ears simultaneously or in close succession, yet they are effortlessly segregated into distinct perceptual sources. This process breaks down in hearing-impaired individuals and speech recognition devices. By identifying the underlying neural mechanisms that facilitate perceptual segregation, we can develop strategies for ameliorating hearing loss and improving speech recognition technology in the presence of background noise. Here, we present evidence to support a hierarchical process, present in primary auditory cortex and refined in secondary auditory cortex, in which sound repetition facilitates segregation.

Ensemble modeling of auditory streaming reveals potential sources of bistability across the perceptual hierarchy

Perceptual bistability-the spontaneous, irregular fluctuation of perception between two interpretations of a stimulus-occurs when observing a large variety of ambiguous stimulus configurations. This phenomenon has the potential to serve as a tool for, among other things, understanding how function varies across individuals due to the large individual differences that manifest during perceptual bistability. Yet it remains difficult to interpret the functional processes at work, without knowing where bistability arises during perception. In this study we explore the hypothesis that bistability originates from multiple sources distributed across the perceptual hierarchy. We develop a hierarchical model of auditory processing comprised of three distinct levels: a Peripheral, tonotopic analysis, a Central analysis computing features found more centrally in the auditory system, and an Object analysis, where sounds are segmented into different streams. We model bistable perception within this system by applying adaptation, inhibition and noise into one or all of the three levels of the hierarchy. We evaluate a large ensemble of variations of this hierarchical model, where each model has a different configuration of adaptation, inhibition and noise. This approach avoids the assumption that a single configuration must be invoked to explain the data. Each model is evaluated based on its ability to replicate two hallmarks of bistability during auditory streaming: the selectivity of bistability to specific stimulus configurations, and the characteristic log-normal pattern of perceptual switches. Consistent with a distributed origin, a broad range of model parameters across this hierarchy lead to a plausible form of perceptual bistability.

Auditory-vocal control system is object for predictive processing within seconds time range

Predictive processing across hierarchically organized time scales is one of the fundamental principles of neural computations in the cerebral cortex. We hypothesize that relatively complex aggregation of auditory and vocal brain systems that use auditory feedback for reflexive control of vocalizations can be an object for predictive processing. We used repetitive patterns of perturbations in auditory feedback during vocalizations to elicit implicit expectations that were violated by surprising direction of perturbations in one of the experimental conditions. Our results provide empirical support for the idea that formation of expectancy for integrated auditory-vocal brain systems, within the time range of seconds, resulted in two sequential neuronal processes. The first process reflects monitoring and error detection in prediction about perturbations in auditory feedback during vocalizations within the time range of seconds. The second neuronal process can be attributed to the optimization of brain predictions for sensory contingencies during vocalizations at separable and distinct timescales.

Emergence of prediction error along the human auditory hierarchy

Auditory prediction errors have been extensively associated with the mismatch negativity (MMN), a cortical auditory evoked potential that denotes deviance detection. Yet, many studies lacked the appropriate controls to disentangle sensory adaptation from prediction error. Furthermore, subcortical deviance detection has been shown in humans through recordings of the frequency-following response (FFR), an early auditory evoked potential that reflects the neural tracking of the periodic characteristics of a sound, suggesting the possibility that prediction errors emerge subcortically in the auditory pathway. The present study aimed at investigating the emergence of prediction error along the auditory hierarchy in humans through combined recordings of the FFR and the MMN, tapping at subcortical and cortical levels, respectively, while disentangling prediction error from sensory adaptation with the use of appropriate controls. "Oddball" sequences of pure tones featuring repeated "standard" stimuli (269 Hz; p = 0.8) and rare "deviant" stimuli (p = 0.2; of 289, 329 and 409 Hz delivered in separated blocks to test "frequency separation" effects) were presented to nineteen healthy young participants. "Reversed" oddball sequences (where standard and deviant tones swapped their roles) were presented allowing comparison of responses to same physical stimuli as a function of functional role (i.e., standard, deviant). Critically, control sequences featuring five equiprobable tones (p = 0.2) allowed to dissociate sensory adaptation from prediction error. Results revealed that the MMN amplitude increased as a function of frequency separation yet displayed the same amplitude when retrieved against the control sequences, confirming previous results. FFRs showed repetition enhancement effects across all frequency separations, as supported by larger spectral amplitude to standard than to deviant and control stimuli. This pattern of results provides insights into the hierarchy of the human prediction error system in audition, suggesting that prediction errors in humans emerge at cortical level.

Evaluating the neurophysiological evidence for predictive processing as a model of perception

For many years, the dominant theoretical framework guiding research into the neural origins of perceptual experience has been provided by hierarchical feedforward models, in which sensory inputs are passed through a series of increasingly complex feature detectors. However, the long-standing orthodoxy of these accounts has recently been challenged by a radically different set of theories that contend that perception arises from a purely inferential process supported by two distinct classes of neurons: those that transmit predictions about sensory states and those that signal sensory information that deviates from those predictions. Although these predictive processing (PP) models have become increasingly influential in cognitive neuroscience, they are also criticized for lacking the empirical support to justify their status. This limited evidence base partly reflects the considerable methodological challenges that are presented when trying to test the unique predictions of these models. However, a confluence of technological and theoretical advances has prompted a recent surge in human and nonhuman neurophysiological research seeking to fill this empirical gap. Here, we will review this new research and evaluate the degree to which its findings support the key claims of PP.

Modulation change detection in human auditory cortex: Evidence for asymmetric, non-linear edge detection

Changes in modulation rate are important cues for parsing acoustic signals, such as speech. We parametrically controlled modulation rate via the correlation coefficient (r) of amplitude spectra across fixed frequency channels between adjacent time frames: broadband modulation spectra are biased toward slow modulate rates with increasing r, and vice versa. By concatenating segments with different r, acoustic changes of various directions (e.g., changes from low to high correlation coefficients, that is, random-to-correlated or vice versa) and sizes (e.g., changes from low to high or from medium to high correlation coefficients) can be obtained. Participants listened to sound blocks and detected changes in correlation while MEG was recorded. Evoked responses to changes in correlation demonstrated (a) an asymmetric representation of change direction: random-to-correlated changes produced a prominent evoked field around 180 ms, while correlated-to-random changes evoked an earlier response with peaks at around 70 and 120 ms, whose topographies resemble those of the canonical P50m and N100m responses, respectively, and (b) a highly non-linear representation of correlation structure, whereby even small changes involving segments with a high correlation coefficient were much more salient than relatively large changes that did not involve segments with high correlation coefficients. Induced responses revealed phase tracking in the delta and theta frequency bands for the high correlation stimuli. The results confirm a high sensitivity for low modulation rates in human auditory cortex, both in terms of their representation and their segregation from other modulation rates.

The effect of aging, Parkinson's disease, and exogenous dopamine on the neural response associated with auditory regularity processing

Processing regular patterns in auditory scenes is important for navigating complex environments. Electroencephalography studies find enhancement of sustained brain activity, correlating with the emergence of a regular pattern in sounds. How aging, aging-related diseases such as Parkinson's disease (PD), and treatment of PD with dopaminergic therapy affect this fundamental function remain unknown. We addressed this knowledge gap. Healthy younger and older adults and patients with PD listened to sounds that contained or were devoid of regular patterns. Healthy older adults and patients with PD were tested twice-off and on dopaminergic medication, in counterbalanced order. Regularity-evoked, sustained electroencephalography activity was reduced in older, compared with younger adults. Patients with PD and older controls evidenced comparable attenuation of the sustained response. Dopaminergic therapy further weakened the sustained response in both older controls and patients with PD. These findings suggest that fundamental regularity processing is impacted by aging but not specifically by PD. The finding that dopaminergic therapy attenuates rather than improves the sustained response coheres with the dopamine overdose response and is in line with previous findings that regularity processing implicates brain regions receiving dopamine from the ventral tegmental area that is relatively spared in PD and normal aging.

Encoding of deterministic and stochastic auditory rules in the human brain: The mismatch negativity mechanism does not reflect basic probability

Regularities in a sequence of sounds can be automatically encoded in a predictive model by the auditory system. When a sound deviates from the one predicted by the model, a mismatch negativity (MMN) is elicited, which is taken to reflect a prediction error at a particular level of the model hierarchy. Although there are many studies on deterministic regularities, only a few have investigated the brain's ability to encode non-deterministic regularities. We studied a simple stochastic regularity: two tone pitches (standards, each occurring on 45% of trials); this regularity was occasionally violated by another tone pitch (deviant, occurring on 10% of trials). We found MMN when the deviant's pitch was outside those of the standards, but not when it was between them. Importantly, when we alternated the occurrence of the same two standards, making them deterministic, the deviant elicited MMN, even when its pitch was between those of the standards. Thus, although the MMN system is extremely powerful in establishing even quite complex deterministic regularities, it fails with a simple stochastic regularity. We argue that the MMN system does not know basic probability.

Audio object classification using distributed beliefs and attention

One of the unique characteristics of human hearing is its ability to recognize acoustic objects even in presence of severe noise and distortions. In this work, we explore two mechanisms underlying this ability: 1) redundant mapping of acoustic waveforms along distributed latent representations and 2) adaptive feedback based on prior knowledge to selectively attend to targets of interest. We propose a bio-mimetic account of acoustic object classification by developing a novel distributed deep belief network validated for the task of robust acoustic object classification using the UrbanSound database. The proposed distributed belief network (DBN) encompasses an array of independent sub-networks trained generatively to capture different abstractions of natural sounds. A supervised classifier then performs a readout of this distributed mapping. The overall architecture not only matches the state of the art system for acoustic object classification but leads to significant improvement over the baseline in mismatched noisy conditions (31.4% relative improvement in 0dB conditions). Furthermore, we incorporate mechanisms of attentional feedback that allows the DBN to deploy local memories of sounds targets estimated at multiple views to bias network activation when attending to a particular object. This adaptive feedback results in further improvement of object classification in unseen noise conditions (relative improvement of 54% over the baseline in 0dB conditions).

Ecological origins of perceptual grouping principles in the auditory system

Events and objects in the world must be inferred from sensory signals to support behavior. Because sensory measurements are temporally and spatially local, the estimation of an object or event can be viewed as the grouping of these measurements into representations of their common causes. Perceptual grouping is believed to reflect internalized regularities of the natural environment, yet grouping cues have traditionally been identified using informal observation and investigated using artificial stimuli. The relationship of grouping to natural signal statistics has thus remained unclear, and additional or alternative cues remain possible. Here, we develop a general methodology for relating grouping to natural sensory signals and apply it to derive auditory grouping cues from natural sounds. We first learned local spectrotemporal features from natural sounds and measured their co-occurrence statistics. We then learned a small set of stimulus properties that could predict the measured feature co-occurrences. The resulting cues included established grouping cues, such as harmonic frequency relationships and temporal coincidence, but also revealed previously unappreciated grouping principles. Human perceptual grouping was predicted by natural feature co-occurrence, with humans relying on the derived grouping cues in proportion to their informativity about co-occurrence in natural sounds. The results suggest that auditory grouping is adapted to natural stimulus statistics, show how these statistics can reveal previously unappreciated grouping phenomena, and provide a framework for studying grouping in natural signals.

Time and information in perceptual adaptation to speech

Perceptual adaptation to a talker enables listeners to efficiently resolve the many-to-many mapping between variable speech acoustics and abstract linguistic representations. However, models of speech perception have not delved into the variety or the quantity of information necessary for successful adaptation, nor how adaptation unfolds over time. In three experiments using speeded classification of spoken words, we explored how the quantity (duration), quality (phonetic detail), and temporal continuity of talker-specific context contribute to facilitating perceptual adaptation to speech. In single- and mixed-talker conditions, listeners identified phonetically-confusable target words in isolation or preceded by carrier phrases of varying lengths and phonetic content, spoken by the same talker as the target word. Word identification was always slower in mixed-talker conditions than single-talker ones. However, interference from talker variability decreased as the duration of preceding speech increased but was not affected by the amount of preceding talker-specific phonetic information. Furthermore, efficiency gains from adaptation depended on temporal continuity between preceding speech and the target word. These results suggest that perceptual adaptation to speech may be understood via models of auditory streaming, where perceptual continuity of an auditory object (e.g., a talker) facilitates allocation of attentional resources, resulting in more efficient perceptual processing.

Logarithmic Versus Linear Change in Step Size When Using an Adaptive Threshold-Seeking Procedure in a Frequency Discrimination Task: Does It Matter?

Different rules for changing step sizes (e.g., logarithmic, linear) are alternately used in adaptive threshold-seeking procedures, with no clear justification. We hypothesized that the linear rule may yield more accurate thresholds for poor performers because the step sizes are predetermined and fixed across listeners and thus can be small, in contrast to the logarithmic rule, in which step sizes are changed with respect to the listener's performance. Purpose The aim of this study was to test the effect of logarithmic and linear rules on frequency discrimination (FD) thresholds. Method Three experiments involving human subjects and Monte Carlo computer simulations were designed and conducted. In the 1st experiment, FD thresholds were estimated in 40 young adults with either 3-interval 2-alternative forced choice (3I2AFC; n = 19) or 2-interval 2AFC (n = 21) in a within-subject design. In the 2nd experiment, thresholds were estimated in 16 children (7-8 years old) in a within-subject design, using 3I2AFC. In the 3rd experiment, thresholds were estimated in 30 young adults in a between-subjects design using 3I2AFC. Results No significant differences were shown between the 2 rules, regardless of age group, method, or level of FD performance. Computer simulations supported the empirical findings, predicting similar FD thresholds for both rules in the majority of runs. However, they also yielded more accurate thresholds with the linear rule, but with a larger number of outliers, which increased as the listener's attention level decreased. Conclusion Overall, the use of a particular rule has little influence on FD thresholds. Possible outliers may be minimized by monitoring the participant's attention at the beginning of the run.

A lateral inhibition mechanism explains the dissociation between mismatch negativity and behavioral pitch discrimination

Although mismatch negativity (MMN), a change-specific component of auditory event-related potential, is considered to be an index of sound discrimination accuracy, the amplitude of the MMN responses elicited by pitch height deviations in musicians and tone language speakers with superior pitch discrimination is usually not enhanced compared to that elicited in individuals with inferior pitch discrimination. We hypothesized that superior pitch discrimination is accompanied by enhanced lateral inhibition, a critical neural mechanism that sharpens the tuning curves of the auditory neurons in the tonotopy. Forty Mandarin-speaking healthy adults completed an auditory EEG experiment in which MMN was elicited by pitch height deviations in both pure and harmonic tones. Their behavioral pitch discrimination was indexed by the difference limens measured using pure and harmonic tones. Behavioral pitch discrimination correlated significantly with the MMN elicited by pure tones, but not by harmonic tones; this could be due to lateral inhibition strongly influencing the MMN elicited by harmonic tones but having less effect on the MMN elicited by pure tones. As lateral inhibition is a neural mechanism for attenuating the amplitude of MMN, our results support the notion that an enhanced lateral inhibition mechanism underlies superior pitch discrimination.

DEVELOPMENT AND VALIDATION OF A METHOD TO ENHANCE AUDITORY ATTENTION DURING CONTINUOUS SPEECH-SHAPED NOISE ENVIRONMENT

Auditory training (AT) may strengthen auditory skills that help human not only in on-task auditory perception performance but in continuous speech-shaped noise (SSN) environment. AT based on musical material has provided some evidence for an “auditory advantage” in understanding speech-in-noise (SIN), but with a long period training and complex procedure. Experimental research is essential to develop a simplified method named auditory target tracking training (ATT) which refined from musical material is necessary to determine the benefits of training. We developed two kinds of refined AT method: basic auditory target tracking (BAT) training and enhanced auditory target tracking (EAT) training to adult participants ([Formula: see text]) separately for 20 units, assessing performance to perceive speech in noise environment after training. The EAT group presented better speech perception performance than the other groups and no significant differences between BAT group and control group. The training effect of EAT is the most significant when uni-gender SSN and [Formula: see text] dB. Outcomes suggest that efficacy of trained EAT can improve speech perception performance and selective attention during SSN environment. These findings provide an important link between musical-based training and auditory selective attention in real-world, and extended to special vocational training.

Neural Encoding of Pitch Direction Is Enhanced in Musically Trained Children and Is Related to Reading Skills

Musical training in childhood has been linked to enhanced sound encoding at different stages of the auditory processing. In the current study, we used auditory event-related potentials to investigate cortical sound processing in 9- to 15-year-old children (N = 88) with and without musical training. Specifically, we recorded the mismatch negativity (MMN) and P3a responses in an oddball paradigm consisting of standard tone pairs with ascending pitch and deviant tone pairs with descending pitch. A subsample of the children (N = 44) also completed a standardized test of reading ability. The musically trained children showed a larger P3a response to the deviant sound pairs. Furthermore, the amplitude of the P3a correlated with a pseudo-word reading test score. These results corroborate previous findings on enhanced sound encoding in musically trained children and are in line with studies suggesting that neural discrimination of spectrotemporal sound patterns is predictive of reading ability.

Multimodal Language Processing in Human Communication

The natural ecology of human language is face-to-face interaction comprising the exchange of a plethora of multimodal signals. Trying to understand the psycholinguistic processing of language in its natural niche raises new issues, first and foremost the binding of multiple, temporally offset signals under tight time constraints posed by a turn-taking system. This might be expected to overload and slow our cognitive system, but the reverse is in fact the case. We propose cognitive mechanisms that may explain this phenomenon and call for a multimodal, situated psycholinguistic framework to unravel the full complexities of human language processing.