Effects of attention on neuroelectric correlates of auditory stream segregation

Perceptually salient differences in a species recognition cue do not promote auditory streaming in eastern grey treefrogs (Hyla versicolor)

Auditory streaming underlies a receiver's ability to organize complex mixtures of auditory input into distinct perceptual "streams" that represent different sound sources in the environment. During auditory streaming, sounds produced by the same source are integrated through time into a single, coherent auditory stream that is perceptually segregated from other concurrent sounds. Based on human psychoacoustic studies, one hypothesis regarding auditory streaming is that any sufficiently salient perceptual difference may lead to stream segregation. Here, we used the eastern grey treefrog, Hyla versicolor, to test this hypothesis in the context of vocal communication in a non-human animal. In this system, females choose their mate based on perceiving species-specific features of a male's pulsatile advertisement calls in social environments (choruses) characterized by mixtures of overlapping vocalizations. We employed an experimental paradigm from human psychoacoustics to design interleaved pulsatile sequences (ABAB…) that mimicked key features of the species' advertisement call, and in which alternating pulses differed in pulse rise time, which is a robust species recognition cue in eastern grey treefrogs. Using phonotaxis assays, we found no evidence that perceptually salient differences in pulse rise time promoted the segregation of interleaved pulse sequences into distinct auditory streams. These results do not support the hypothesis that any perceptually salient acoustic difference can be exploited as a cue for stream segregation in all species. We discuss these findings in the context of cues used for species recognition and auditory streaming.

Why are listeners hindered by talker variability?

Though listeners readily recognize speech from a variety of talkers, accommodating talker variability comes at a cost: Myriad studies have shown that listeners are slower to recognize a spoken word when there is talker variability compared with when talker is held constant. This review focuses on two possible theoretical mechanisms for the emergence of these processing penalties. One view is that multitalker processing costs arise through a resource-demanding talker accommodation process, wherein listeners compare sensory representations against hypothesized perceptual candidates and error signals are used to adjust the acoustic-to-phonetic mapping (an active control process known as contextual tuning). An alternative proposal is that these processing costs arise because talker changes involve salient stimulus-level discontinuities that disrupt auditory attention. Some recent data suggest that multitalker processing costs may be driven by both mechanisms operating over different time scales. Fully evaluating this claim requires a foundational understanding of both talker accommodation and auditory streaming; this article provides a primer on each literature and also reviews several studies that have observed multitalker processing costs. The review closes by underscoring a need for comprehensive theories of speech perception that better integrate auditory attention and by highlighting important considerations for future research in this area.

Electrophysiological evidence for increased auditory crossmodal activity in adult ADHD

Background

Attention deficit and hyperactivity disorder (ADHD) is a neurodevelopmental disorder characterized by core symptoms of inattention, and/or impulsivity and hyperactivity. In order to understand the basis for this multifaceted disorder, the investigation of sensory processing aberrancies recently reaches more interest. For example, during the processing of auditory stimuli comparable low sensory thresholds account for symptoms like higher distractibility and auditory hypersensitivity in patients with ADHD. It has further been shown that deficiencies not only exist on an intramodal, but also on a multimodal level. There is evidence that the visual cortex shows more activation during a focused auditory task in adults with ADHD than in healthy controls. This crossmodal activation is interpreted as the reallocation of more attentional resources to the visual domain as well as deficient sensory inhibition. In this study, we used, for the first time, electroencephalography to identify a potential abnormal regulated crossmodal activation in adult ADHD.

Methods

15 adult subjects with clinically diagnosed ADHD and 14 healthy controls comparable in age and gender were included. ERP components P50, P100, N100, P200 and N200 were measured during the performance of a unimodal auditory and visual discrimination task in a block design. Sensory profiles and ADHD symptoms were assessed with inattention as well as childhood ADHD scores. For evaluating intramodal and crossmodal activations, we chose four EEG channels for statistical analysis and group-wise comparison.

Results

At the occipital channel O2 that reflects possible crossmodal activations, a significantly enhanced P200 amplitude was measured in the patient group. At the intramodal channels, a significantly enhanced N200 amplitude was observed in the control group. Statistical analysis of behavioral data showed poorer performance of subjects with ADHD as well as higher discrimination thresholds. Further, the correlation of the assessed sensory profiles with the EEG parameters revealed a negative correlation between the P200 component and sensation seeking behavior.

Conclusion

Our findings show increased auditory crossmodal activity that might reflect an altered stimulus processing resource allocation in ADHD. This might induce consequences for later, higher order attentional deployment. Further, the enhanced P200 amplitude might reflect more sensory registration and therefore deficient inhibition mechanisms in adults with ADHD.

Early maturation of sound duration processing in the infant's brain

The ability to process sound duration is crucial already at a very early age for laying the foundation for the main functions of auditory perception, such as object perception and music and language acquisition. With the availability of age-appropriate structural anatomical templates, we can reconstruct EEG source activity with much-improved reliability. The current study capitalized on this possibility by reconstructing the sources of event-related potential (ERP) waveforms sensitive to sound duration in 4- and 9-month-old infants. Infants were presented with short (200 ms) and long (300 ms) sounds equiprobable delivered in random order. Two temporally separate ERP waveforms were found to be modulated by sound duration. Generators of these waveforms were mainly located in the primary and secondary auditory areas and other language-related regions. The results show marked developmental changes between 4 and 9 months, partly reflected by scalp-recorded ERPs, but appearing in the underlying generators in a far more nuanced way. The results also confirm the feasibility of the application of anatomical templates in developmental populations.

Laminar specificity of the auditory perceptual awareness negativity: A biophysical modeling study

How perception of sensory stimuli emerges from brain activity is a fundamental question of neuroscience. To date, two disparate lines of research have examined this question. On one hand, human neuroimaging studies have helped us understand the large-scale brain dynamics of perception. On the other hand, work in animal models (mice, typically) has led to fundamental insight into the micro-scale neural circuits underlying perception. However, translating such fundamental insight from animal models to humans has been challenging. Here, using biophysical modeling, we show that the auditory awareness negativity (AAN), an evoked response associated with perception of target sounds in noise, can be accounted for by synaptic input to the supragranular layers of auditory cortex (AC) that is present when target sounds are heard but absent when they are missed. This additional input likely arises from cortico-cortical feedback and/or non-lemniscal thalamic projections and targets the apical dendrites of layer-5 (L5) pyramidal neurons. In turn, this leads to increased local field potential activity, increased spiking activity in L5 pyramidal neurons, and the AAN. The results are consistent with current cellular models of conscious processing and help bridge the gap between the macro and micro levels of perception-related brain activity.

Attractor-Like Dynamics Extracted from Human Electrocorticographic Recordings Underlie Computational Principles of Auditory Bistable Perception

In bistable perception, observers experience alternations between two interpretations of an unchanging stimulus. Neurophysiological studies of bistable perception typically partition neural measurements into stimulus-based epochs and assess neuronal differences between epochs based on subjects' perceptual reports. Computational studies replicate statistical properties of percept durations with modeling principles like competitive attractors or Bayesian inference. However, bridging neuro-behavioral findings with modeling theory requires the analysis of single-trial dynamic data. Here, we propose an algorithm for extracting nonstationary timeseries features from single-trial electrocorticography (ECoG) data. We applied the proposed algorithm to 5-min ECoG recordings from human primary auditory cortex obtained during perceptual alternations in an auditory triplet streaming task (six subjects: four male, two female). We report two ensembles of emergent neuronal features in all trial blocks. One ensemble consists of periodic functions that encode a stereotypical response to the stimulus. The other comprises more transient features and encodes dynamics associated with bistable perception at multiple time scales: minutes (within-trial alternations), seconds (duration of individual percepts), and milliseconds (switches between percepts). Within the second ensemble, we identified a slowly drifting rhythm that correlates with the perceptual states and several oscillators with phase shifts near perceptual switches. Projections of single-trial ECoG data onto these features establish low-dimensional attractor-like geometric structures invariant across subjects and stimulus types. These findings provide supporting neural evidence for computational models with oscillatory-driven attractor-based principles. The feature extraction techniques described here generalize across recording modality and are appropriate when hypothesized low-dimensional dynamics characterize an underlying neural system.SIGNIFICANCE STATEMENT Irrespective of the sensory modality, neurophysiological studies of multistable perception have typically investigated events time-locked to the perceptual switching rather than the time course of the perceptual states per se. Here, we propose an algorithm that extracts neuronal features of bistable auditory perception from largescale single-trial data while remaining agnostic to the subject's perceptual reports. The algorithm captures the dynamics of perception at multiple timescales, minutes (within-trial alternations), seconds (durations of individual percepts), and milliseconds (timing of switches), and distinguishes attributes of neural encoding of the stimulus from those encoding the perceptual states. Finally, our analysis identifies a set of latent variables that exhibit alternating dynamics along a low-dimensional manifold, similar to trajectories in attractor-based models for perceptual bistability.

Possibility of additive effects by the presentation of visual information related to distractor sounds on the contra-sound effects of the N100m responses

Auditory-evoked responses can be affected by different types of contralateral sounds or by attention modulation. The present study examined the additive effects of presenting visual information about contralateral sounds as distractions during dichotic listening tasks on the contralateral effects of N100m responses in the auditory-evoked cortex in 16 subjects (12 males and 4 females). In magnetoencephalography, a tone-burst of 500 ms duration at a frequency of 1000 Hz was played to the left ear at a level of 70 dB as a stimulus to elicit the N100m response, and a movie clip was used as a distractor stimulus under audio-only, visual-only, and audio-visual conditions. Subjects were instructed to pay attention to the left ear and press the response button each time they heard a tone-burst stimulus in their left ear. The results suggest that the presentation of visual information related to the contralateral sound, which worked as a distractor, significantly suppressed the amplitude of the N100m response compared with only the contralateral sound condition. In contrast, the presentation of visual information related to contralateral sound did not affect the latency of the N100m response. These results suggest that the integration of contralateral sounds and related movies may have resulted in a more perceptually loaded stimulus and reduced the intensity of attention to tone-bursts. Our findings suggest that selective attention and saliency mechanisms may have cross-modal effects on other modes of perception.

Left Lateralization of the Cortical Auditory-Evoked Potential Reflects Aided Processing and Speech-in-Noise Performance of Older Listeners With a Hearing Loss

OBJECTIVES

We analyzed the lateralization of the cortical auditory-evoked potential recorded previously from aided hearing-impaired listeners as part of a study on noise-mitigating hearing aid technologies. Specifically, we asked whether the degree of leftward lateralization in the magnitudes and latencies of these components was reduced by noise and, conversely, enhanced/restored by hearing aid technology. We further explored if individual differences in lateralization could predict speech-in-noise abilities in listeners when tested in the aided mode.

DESIGN

The study followed a double-blind within-subjects design. Nineteen older adults (8 females; mean age = 73.6 years, range = 56 to 86 years) with moderate to severe hearing loss participated. The cortical auditory-evoked potential was measured over 400 presentations of a synthetic /da/ stimulus which was delivered binaurally in a simulated aided mode using shielded ear-insert transducers. Sequences of the /da/ syllable were presented from the front at 75 dB SPL-C with continuous speech-shaped noise presented from the back at signal-to-noise ratios of 0, 5, and 10 dB. Four hearing aid conditions were tested: (1) omnidirectional microphone (OM) with noise reduction (NR) disabled, (2) OM with NR enabled, (3) directional microphone (DM) with NR disabled, and (4) DM with NR enabled. Lateralization of the P1 component and N1P2 complex was quantified across electrodes spanning the mid-coronal plane. Subsequently, listener speech-in-noise performance was assessed using the Repeat-Recall Test at the same signal-to-noise ratios and hearing aid conditions used to measure cortical activity.

RESULTS

As expected, both the P1 component and the N1P2 complex were of greater magnitude in electrodes over the left compared to the right hemisphere. In addition, N1 and P2 peaks tended to occur earlier over the left hemisphere, although the effect was mediated by an interaction of signal-to-noise ratio and hearing aid technology. At a group level, degrees of lateralization for the P1 component and the N1P2 complex were enhanced in the DM relative to the OM mode. Moreover, linear mixed-effects models suggested that the degree of leftward lateralization in the N1P2 complex, but not the P1 component, accounted for a significant portion of variability in speech-in-noise performance that was not related to age, hearing loss, hearing aid processing, or signal-to-noise ratio.

CONCLUSIONS

A robust leftward lateralization of cortical potentials was observed in older listeners when tested in the aided mode. Moreover, the degree of lateralization was enhanced by hearing aid technologies that improve the signal-to-noise ratio for speech. Accounting for the effects of signal-to-noise ratio, hearing aid technology, semantic context, and audiometric thresholds, individual differences in left-lateralized speech-evoked cortical activity were found to predict listeners' speech-in-noise abilities. Quantifying cortical auditory-evoked potential component lateralization may then be useful for profiling listeners' likelihood of communication success following clinical amplification.

Attentional modulation of auditory cortical activity in individuals with single-sided deafness

Persons with single-sided deafness (SSD) typically complain about the impaired ability to locate sounds and to understand speech within background noise. However, the findings from previous studies suggest that paying attention to sounds could mitigate the degraded spatial and speech-in-noise perception. In the present study, we characterize the pattern of cortical activation depending on the side of deafness, and attentional modulation of neural responses to determine if it can assist better sound processing in people with SSD. For the active listening condition, adult subjects with SSD performed sound localization tasks. On the other hand, they watched movies without attending to speech stimuli during passive listening. The sensor-level global field power of N1 and source-level N1 activation were computed to compare the active- and passive-listening conditions and left- and right-sided deafness. The results show that attentional modulation differs depending on the side of deafness: active listening increased the cortical activity in individuals with left-sided deafness but not in those with right-sided deafness. At the source level, the attentional gain was more apparent in left-sided deafness in that paying attention enhanced brain activation in both hemispheres. In addition, SSD participants with larger cortical activities in the right primary auditory cortex had shorter durations of deafness. Our results indicate that the side of deafness can change top-down attentional processing in the auditory cortical pathway in SSD patients.

Adaptation in the sensory cortex drives bistable switching during auditory stream segregation

Current theories of perception emphasize the role of neural adaptation, inhibitory competition, and noise as key components that lead to switches in perception. Supporting evidence comes from neurophysiological findings of specific neural signatures in modality-specific and supramodal brain areas that appear to be critical to switches in perception. We used functional magnetic resonance imaging to study brain activity around the time of switches in perception while participants listened to a bistable auditory stream segregation stimulus, which can be heard as one integrated stream of tones or two segregated streams of tones. The auditory thalamus showed more activity around the time of a switch from segregated to integrated compared to time periods of stable perception of integrated; in contrast, the rostral anterior cingulate cortex and the inferior parietal lobule showed more activity around the time of a switch from integrated to segregated compared to time periods of stable perception of segregated streams, consistent with prior findings of asymmetries in brain activity depending on the switch direction. In sound-responsive areas in the auditory cortex, neural activity increased in strength preceding switches in perception and declined in strength over time following switches in perception. Such dynamics in the auditory cortex are consistent with the role of adaptation proposed by computational models of visual and auditory bistable switching, whereby the strength of neural activity decreases following a switch in perception, which eventually destabilizes the current percept enough to lead to a switch to an alternative percept.

Informational Masking in Aging and Brain-lesioned Individuals

Auditory stream segregation and informational masking were investigated in brain-lesioned individuals, age-matched controls with no neurological disease, and young college-age students. A psychophysical paradigm known as rhythmic masking release (RMR) was used to examine the ability of participants to identify a change in the rhythmic sequence of 20-ms Gaussian noise bursts presented through headphones and filtered through generalized head-related transfer functions to produce the percept of an externalized auditory image (i.e., a 3D virtual reality sound). The target rhythm was temporally interleaved with a masker sequence comprising similar noise bursts in a manner that resulted in a uniform sequence with no information remaining about the target rhythm when the target and masker were presented from the same location (an impossible task). Spatially separating the target and masker sequences allowed participants to determine if there was a change in the target rhythm midway during its presentation. RMR thresholds were defined as the minimum spatial separation between target and masker sequences that resulted in 70.7% correct-performance level in a single-interval 2-alternative forced-choice adaptive tracking procedure. The main findings were (1) significantly higher RMR thresholds for individuals with brain lesions (especially those with damage to parietal areas) and (2) a left-right spatial asymmetry in performance for lesion (but not control) participants. These findings contribute to a better understanding of spatiotemporal relations in informational masking and the neural bases of auditory scene analysis.

Endogenous sources of interbrain synchrony in duetting pianists

When people interact with each other, their brains synchronize. However, it remains unclear whether interbrain synchrony (IBS) is functionally relevant for social interaction or stems from exposure of individual brains to identical sensorimotor information. To disentangle these views, the current dual-EEG study investigated amplitude-based IBS in pianists jointly performing duets containing a silent pause followed by a tempo change. First, we manipulated the similarity of the anticipated tempo change and measured IBS during the pause, hence, capturing the alignment of purely endogenous, temporal plans without sound or movement. Notably, right posterior gamma IBS was higher when partners planned similar tempi, it predicted whether partners' tempi matched after the pause, and it was modulated only in real, not in surrogate pairs. Second, we manipulated the familiarity with the partner's actions and measured IBS during joint performance with sound. Although sensorimotor information was similar across conditions, gamma IBS was higher when partners were unfamiliar with each other's part and had to attend more closely to the sound of the performance. These combined findings demonstrate that IBS is not merely an epiphenomenon of shared sensorimotor information but can also hinge on endogenous, cognitive processes crucial for behavioral synchrony and successful social interaction.

Age-Related Differences in Early Cortical Representations of Target Speech Masked by Either Steady-State Noise or Competing Speech

Word in noise identification is facilitated by acoustic differences between target and competing sounds and temporal separation between the onset of the masker and that of the target. Younger and older adults are able to take advantage of onset delay when the masker is dissimilar (Noise) to the target word, but only younger adults are able to do so when the masker is similar (Babble). We examined the neural underpinning of this age difference using cortical evoked responses to words masked by either Babble or Noise when the masker preceded the target word by 100 or 600 ms in younger and older adults, after adjusting the signal-to-noise ratios (SNRs) to equate behavioural performance across age groups and conditions. For the 100 ms onset delay, the word in noise elicited an acoustic change complex (ACC) response that was comparable in younger and older adults. For the 600 ms onset delay, the ACC was modulated by both masker type and age. In older adults, the ACC to a word in babble was not affected by the increase in onset delay whereas younger adults showed a benefit from longer delays. Hence, the age difference in sensitivity to temporal delay is indexed by early activity in the auditory cortex. These results are consistent with the hypothesis that an increase in onset delay improves stream segregation in younger adults in both noise and babble, but only in noise for older adults and that this change in stream segregation is evident in early cortical processes.

Speech Processing as a Far-Transfer Gauge of Serious Games for Cognitive Training in Aging: Randomized Controlled Trial of Web-Based Effectivate Training

BACKGROUND

The number of serious games for cognitive training in aging (SGCTAs) is proliferating in the market and attempting to combat one of the most feared aspects of aging-cognitive decline. However, the efficacy of many SGCTAs is still questionable. Even the measures used to validate SGCTAs are up for debate, with most studies using cognitive measures that gauge improvement in trained tasks, also known as near transfer. This study takes a different approach, testing the efficacy of the SGCTA-Effectivate-in generating tangible far-transfer improvements in a nontrained task-the Eye tracking of Word Identification in Noise Under Memory Increased Load (E-WINDMIL)-which tests speech processing in adverse conditions.

OBJECTIVE

This study aimed to validate the use of a real-time measure of speech processing as a gauge of the far-transfer efficacy of an SGCTA designed to train executive functions.

METHODS

In a randomized controlled trial that included 40 participants, we tested 20 (50%) older adults before and after self-administering the SGCTA Effectivate training and compared their performance with that of the control group of 20 (50%) older adults. The E-WINDMIL eye-tracking task was administered to all participants by blinded experimenters in 2 sessions separated by 2 to 8 weeks.

RESULTS

Specifically, we tested the change between sessions in the efficiency of segregating the spoken target word from its sound-sharing alternative, as the word unfolds in time. We found that training with the SGCTA Effectivate improved both early and late speech processing in adverse conditions, with higher discrimination scores in the training group than in the control group (early processing: F_1,38=7.371; P=.01; η_p²=0.162 and late processing: F_1,38=9.003; P=.005; η_p²=0.192).

CONCLUSIONS

This study found the E-WINDMIL measure of speech processing to be a valid gauge for the far-transfer effects of executive function training. As the SGCTA Effectivate does not train any auditory task or language processing, our results provide preliminary support for the ability of Effectivate to create a generalized cognitive improvement. Given the crucial role of speech processing in healthy and successful aging, we encourage researchers and developers to use speech processing measures, the E-WINDMIL in particular, to gauge the efficacy of SGCTAs. We advocate for increased industry-wide adoption of far-transfer metrics to gauge SGCTAs.

Spatial release from informational masking enhances the early cortical representation of speech sounds

INTRODUCTION

Spatial separation between competing speech streams reduces their confusion (informational masking), improving speech processing under challenging listening conditions. The precise stages of auditory processing involved in this benefit are not fully understood. This study used event-related potentials to examine the processing of target speech under conditions of informational masking and its spatial release.

METHODS

Participants detected noise-vocoded target speech presented with two-talker noise-vocoded masking speech. In separate conditions, the same set of targets were spatially co-located with maskers to produce informational masking and spatially separated from maskers using a perceptual manipulation to release the informational masking.

RESULTS

An increase in N1 and P2 amplitude, consistent with cortical auditory evoked potentials, and a later sustained positivity (P300) were observed in response to target onsets only under conditions supporting release from informational masking. At target intensities above masking threshold in both spatial conditions, N1 and P2 latencies were shorter when targets and maskers were perceptually separated.

DISCUSSION

These results indicate that spatial release from informational masking benefits speech representation beginning in the early stages of auditory perception. Additionally, these results suggest that the auditory evoked potential itself may be heavily dependent upon how information is perceptually organized rather than physically organized.

Temporal dynamics of auditory bistable perception correlated with fluctuation of baseline pupil size

A dynamic neural network change, accompanied by cognitive shifts such as internal perceptual alternation in bistable stimuli, is reconciled by the discharge of noradrenergic locus coeruleus neurons. Transient pupil dilation as a consequence of the reconciliation with the neural network in bistable perception has been reported to precede the reported perceptual alternation. Here, we found that baseline pupil size, an index of temporal fluctuation of arousal level over a longer range of timescales than that for the transient pupil changes, relates to the frequency of perceptual alternation in auditory bistability. Baseline pupil size was defined as the mean pupil diameter over a period of 1 s prior to the task requirement (i.e., before the observation period for counting the perceptual alternations in Experiment 1 and reporting whether participants experienced the perceptual alternations in Experiment 2). The results showed that the baseline pupil size monotonically increased with an increasing number of perceptual alternations and its occurrence probability. Furthermore, a cross‐correlation analysis indicates that baseline pupil size predicted perceptual alternation at least 35 s before the behavioral response and that the overall correspondence between pupil size and perceptual alternation was maintained over a sustained time window of 45 s at minimum. The overall results suggest that variability of baseline pupil size reflects the stochastic dynamics of arousal fluctuation in the brain related to bistable perception.

Voluntary auditory change: First-person access to agentive aspects of attention regulation

In perceptual psychology, audition and introspection have not yet received as much attention as other topics (e.g., vision) and methods (third-person paradigms). Practical examples and theoretical considerations show that it nevertheless seems promising to treat both topics in conjunction to gain insights into basic structures of attention regulation and respective agentive awareness. To this end, an empirical study on voluntary auditory change was conducted with a non-reactive first-person design. Data were analyzed with a mixed methods approach and compared with an analogous study on visual reversal. Qualitative hierarchical coding and explorative statistics yield a cross-modal replication of frequency patterns of mental activity as well as significant differences between the modalities. On this basis, the role of mental agency in perception is refined in terms of different levels of intention and discussed in the context of the philosophical mental action debate as well as of the Global Workspace/Working Memory account. As a main result, this work suggests the existence and structure of a gradual and developable agentive attention awareness on which voluntary attention regulation can build, and which justifies speaking, in a certain sense, of attentional self-perception.

Simple statistical regularities presented during sleep are detected but not retained

In recent years, there has been growing interest and excitement over the newly discovered cognitive capacities of the sleeping brain, including its ability to form novel associations. These recent discoveries raise the possibility that other more sophisticated forms of learning may also be possible during sleep. In the current study, we tested whether sleeping humans are capable of statistical learning - the process of becoming sensitive to repeating, hidden patterns in environmental input, such as embedded words in a continuous stream of speech. Participants' EEG was recorded while they were presented with one of two artificial languages, composed of either trisyllabic or disyllabic nonsense words, during slow-wave sleep. We used an EEG measure of neural entrainment to assess whether participants became sensitive to the repeating regularities during sleep-exposure to the language. We further probed for long-term memory representations by assessing participants' performance on implicit and explicit tests of statistical learning during subsequent wake. In the disyllabic-but not trisyllabic-language condition, participants' neural entrainment to words increased over time, reflecting a gradual gain in sensitivity to the embedded regularities. However, no significant behavioural effects of sleep-exposure were observed after the nap, for either language. Overall, our results indicate that the sleeping brain can detect simple, repeating pairs of syllables, but not more complex triplet regularities. However, the online detection of these regularities does not appear to produce any durable long-term memory traces that persist into wake - at least none that were revealed by our current measures and sample size. Although some perceptual aspects of statistical learning are preserved during sleep, the lack of memory benefits during wake indicates that exposure to a novel language during sleep may have limited practical value.

Long Latency Auditory Evoked Potentials and Object-Related Negativity Based on Harmonicity in Hearing-Impaired Children

Hearing-impaired children (HIC) have difficulty understanding speech in noise, which may be due to difficulty parsing concurrent sound object based on harmonicity cues. Using long latency auditory evoked potentials (LLAEPs) and object-related negativity (ORN), a neural metric of concurrent sound segregation, this study investigated the sensitivity of HIC in processing harmonic relation. The participants were 14 normal-hearing children (NHC) with an average age of 7.82 ± 1.31 years and 17 HIC with an average age of 7.98 ± 1.25 years. They were presented with a sequence of 200 Hz harmonic complex tones that had either all harmonic in tune or the third harmonic mistuned by 2%, 4%, 8%, and 16% of its original value while neuroelectric brain activity was recorded. The analysis of scalp-recorded LLAEPs revealed lower N2 amplitudes elicited by the tuned stimuli in HIC than control. The ORN, isolated in difference wave between LLAEP elicited by tuned and mistuned stimuli, was delayed and smaller in HIC than NHC. This study showed that deficits in processing harmonic relation in HIC, which may contribute to their difficulty in understanding speech in noise. As a result, top-down and bottom-up rehabilitations aiming to improve processing of basic acoustic characteristics, including harmonics are recommended for children with hearing loss.

Dimensionally Specific Attention Capture in Birds Performing Auditory Streaming Task

Previous studies in budgerigars (Melopsittacus undulatus) have indicated that they experience attention capture in a qualitatively similar way to humans. Here, we apply a similar objective auditory streaming paradigm, using modified budgerigar vocalizations instead of ABAB-… patterned pure tones, in the sound sequences. The birds were trained to respond to deviants in the target stream while ignoring the distractors in the background stream. The background distractor could vary among five different categories and two different sequential positions, while the target deviants could randomly appear at five different sequential positions and vary among two different categories. We found that unpredictable background distractors could deteriorate birds' sensitivity to the target deviants. Compared to conditions where the background distractor appeared right before the target deviant, the attention capture effect decayed in conditions when the background distractor appeared earlier. In contrast to results from the same paradigm using pure tones, the results here are evidence for a faster recovery from attention capture using modified vocalization segments. We found that the temporally modulated background distractor captured birds' attention more and deteriorated birds' performance more than other categories of background distractor, as the temporally modulated target deviant enabled the birds to focus their attention toward the temporal modulation dimension. However, different from humans, birds have lower tolerances for suppressing the distractors from the same feature dimensions as the targets, which is evidenced by higher false alarm rates for the temporally modulated distractor than other distractors from different feature dimensions.

Attentional control via synaptic gain mechanisms in auditory streaming

Attention is a crucial component in sound source segregation allowing auditory objects of interest to be both singled out and held in focus. Our study utilizes a fundamental paradigm for sound source segregation: a sequence of interleaved tones, A and B, of different frequencies that can be heard as a single integrated stream or segregated into two streams (auditory streaming paradigm). We focus on the irregular alternations between integrated and segregated that occur for long presentations, so-called auditory bistability. Psychaoustic experiments demonstrate how attentional control, a listener's intention to experience integrated or segregated, biases perception in favour of different perceptual interpretations. Our data show that this is achieved by prolonging the dominance times of the attended percept and, to a lesser extent, by curtailing the dominance times of the unattended percept, an effect that remains consistent across a range of values for the difference in frequency between A and B. An existing neuromechanistic model describes the neural dynamics of perceptual competition downstream of primary auditory cortex (A1). The model allows us to propose plausible neural mechanisms for attentional control, as linked to different attentional strategies, in a direct comparison with behavioural data. A mechanism based on a percept-specific input gain best accounts for the effects of attentional control.

Binding the Acoustic Features of an Auditory Source through Temporal Coherence

Numerous studies have suggested that the perception of a target sound stream (or source) can only be segregated from a complex acoustic background mixture if the acoustic features underlying its perceptual attributes (e.g., pitch, location, and timbre) induce temporally modulated responses that are mutually correlated (or coherent), and that are uncorrelated (incoherent) from those of other sources in the mixture. This "temporal coherence" hypothesis asserts that attentive listening to one acoustic feature of a target enhances brain responses to that feature but would also concomitantly (1) induce mutually excitatory influences with other coherently responding neurons, thus enhancing (or binding) them all as they respond to the attended source; by contrast, (2) suppressive interactions are hypothesized to build up among neurons driven by temporally incoherent sound features, thus relatively reducing their activity. In this study, we report on EEG measurements in human subjects engaged in various sound segregation tasks that demonstrate rapid binding among the temporally coherent features of the attended source regardless of their identity (pure tone components, tone complexes, or noise), harmonic relationship, or frequency separation, thus confirming the key role temporal coherence plays in the analysis and organization of auditory scenes.

Dynamic selective auditory attention detection using RNN and reinforcement learning

The cocktail party phenomenon describes the ability of the human brain to focus auditory attention on a particular stimulus while ignoring other acoustic events. Selective auditory attention detection (SAAD) is an important issue in the development of brain-computer interface systems and cocktail party processors. This paper proposes a new dynamic attention detection system to process the temporal evolution of the input signal. The proposed dynamic SAAD is modeled as a sequential decision-making problem, which is solved by recurrent neural network (RNN) and reinforcement learning methods of Q-learning and deep Q-learning. Among different dynamic learning approaches, the evaluation results show that the deep Q-learning approach with RNN as agent provides the highest classification accuracy (94.2%) with the least detection delay. The proposed SAAD system is advantageous, in the sense that the detection of attention is performed dynamically for the sequential inputs. Also, the system has the potential to be used in scenarios, where the attention of the listener might be switched in time in the presence of various acoustic events.

Selective attention modulates neural envelope tracking of informationally masked speech in healthy older adults

Speech understanding in noisy situations is compromised in old age. This study investigated the energetic and informational masking components of multi‐talker babble noise and their influence on neural tracking of the speech envelope in a sample of healthy older adults. Twenty‐three older adults (age range 65–80 years) listened to an audiobook embedded in noise while their electroencephalogram (EEG) was recorded. Energetic masking was manipulated by varying the signal‐to‐noise ratio (SNR) between target speech and background talkers and informational masking was manipulated by varying the number of background talkers. Neural envelope tracking was measured by calculating temporal response functions (TRFs) between speech envelope and EEG. Number of background talkers, but not SNR modulated the amplitude of an earlier (around 50 ms time lag) and a later (around 300 ms time lag) peak in the TRFs. Selective attention, but not working memory or peripheral hearing additionally modulated the amplitude of the later TRF peak. Finally, amplitude of the later TRF peak was positively related to accuracy in the comprehension task. The results suggest that stronger envelope tracking is beneficial for speech‐in‐noise understanding and that selective attention is an important ability supporting speech‐in‐noise understanding in multi‐talker scenes.

Multi-Voiced Music Bypasses Attentional Limitations in the Brain

Attentional limits make it difficult to comprehend concurrent speech streams. However, multiple musical streams are processed comparatively easily. Coherence may be a key difference between music and stimuli like speech, which does not rely on the integration of multiple streams for comprehension. The musical organization between melodies in a composition may provide a cognitive scaffold to overcome attentional limitations when perceiving multiple lines of music concurrently. We investigated how listeners attend to multi–voiced music, examining biological indices associated with processing structured versus unstructured music. We predicted that musical structure provides coherence across distinct musical lines, allowing listeners to attend to simultaneous melodies, and that a lack of organization causes simultaneous melodies to be heard as separate streams. Musician participants attended to melodies in a Coherent music condition featuring flute duets and a Jumbled condition where those duets were manipulated to eliminate coherence between the parts. Auditory–evoked cortical potentials were collected to a tone probe. Analysis focused on the N100 response which is primarily generated within the auditory cortex and is larger for attended versus ignored stimuli. Results suggest that participants did not attend to one line over the other when listening to Coherent music, instead perceptually integrating the streams. Yet, for the Jumbled music, effects indicate that participants attended to one line while ignoring the other, abandoning their integration. Our findings lend support for the theory that musical organization aids attention when perceiving multi–voiced music.

Extracting human cortical responses to sound onsets and acoustic feature changes in real music, and their relation to event rate

Evoked cortical responses (ERs) have mainly been studied in controlled experiments using simplified stimuli. Though, an outstanding question is how the human cortex responds to the complex stimuli encountered in realistic situations. Few electroencephalography (EEG) studies have used Music Information Retrieval (MIR) tools to extract cortical P1/N1/P2 to acoustical changes in real music. However, less than ten events per music piece could be detected leading to ERs due to limitations in automatic detection of sound onsets. Also, the factors influencing a successful extraction of the ERs have not been identified. Finally, previous studies did not localize the sources of the cortical generators. This study is based on an EEG/MEG dataset from 48 healthy normal hearing participants listening to three real music pieces. Acoustic features were computed from the audio signal of the music with the MIR Toolbox. To overcome limits in automatic methods, sound onsets were also manually detected. The chance of obtaining detectable ERs based on ten randomly picked onset points was less than 1:10,000. For the first time, we show that naturalistic P1/N1/P2 ERs can be reliably measured across 100 manually identified sound onsets, substantially improving the signal-to-noise level compared to <10 trials. More ERs were measurable in musical sections with slow event rates (0.2 Hz-2.5 Hz) than with fast event rates (>2.5 Hz). Furthermore, during monophonic sections of the music only P1/P2 were measurable, and during polyphonic sections only N1. Finally, MEG source analysis revealed that naturalistic P2 is located in core areas of the auditory cortex.

Central auditory system responses from children while listening to speech in noise

Cortical auditory evoked potentials (CAEPs) have been successfully used to explore the effects of noise on speech processing at the cortical level in adults and children. The purpose of this study was to determine whether +15 dB signal-to-noise ratios (SNRs), often recommended for optimal speech perception in children, elicit higher amplitude CAEPs than more realistic SNRs encountered by children during their daily lives (+10 dB SNR). Moreover, we aimed to investigate whether cortical speech categorization is observable in children in quiet and in noise and whether CAEPs to speech in noise are related to behavioral speech perception in noise performance in children. CAEPs were measured during a passive speech-syllable task in 51 normal hearing children aged 8 to 11 years. The speech syllables /da/ and /ga/ were presented in quiet and in the presence of a 4-talker-babble noise at +15 dB and +10 dB SNR. N1 latencies and P2 amplitudes and latencies varied as a function of SNR, with poorer SNRs (+10 dB) eliciting significantly smaller P2 amplitudes and delayed N1 and P2 latencies relative to the higher SNR (+15 dB). Finally, speech categorization was present at the cortical level in this group of children in quiet and at both SNRs; however, N1 and P2 amplitudes and latencies were not related to behavioral speech-in-noise perception of children.

Rhythm Complexity Modulates Behavioral and Neural Dynamics During Auditory–Motor Synchronization

We addressed how rhythm complexity influences auditory–motor synchronization in musically trained individuals who perceived and produced complex rhythms while EEG was recorded. Participants first listened to two-part auditory sequences (Listen condition). Each part featured a single pitch presented at a fixed rate; the integer ratio formed between the two rates varied in rhythmic complexity from low (1:1) to moderate (1:2) to high (3:2). One of the two parts occurred at a constant rate across conditions. Then, participants heard the same rhythms as they synchronized their tapping at a fixed rate (Synchronize condition). Finally, they tapped at the same fixed rate (Motor condition). Auditory feedback from their taps was present in all conditions. Behavioral effects of rhythmic complexity were evidenced in all tasks; detection of missing beats (Listen) worsened in the most complex (3:2) rhythm condition, and tap durations (Synchronize) were most variable and least synchronous with stimulus onsets in the 3:2 condition. EEG power spectral density was lowest at the fixed rate during the 3:2 rhythm and greatest during the 1:1 rhythm (Listen and Synchronize). ERP amplitudes corresponding to an N1 time window were smallest for the 3:2 rhythm and greatest for the 1:1 rhythm (Listen). Finally, synchronization accuracy (Synchronize) decreased as amplitudes in the N1 time window became more positive during the high rhythmic complexity condition (3:2). Thus, measures of neural entrainment corresponded to synchronization accuracy, and rhythmic complexity modulated the behavioral and neural measures similarly.

Effects of aging on event-related potentials to single-cycle binaural beats and diotic amplitude modulation of a tone

Aim of the study is to determine whether the auditory processing of temporal fine structure (TFS) is affected with normal aging, even in the presence of normal audiometric hearing and fine cognitive state; and, if it is, to see whether a comparable effect is also observed in the processing of a diotic change in sound envelope. The event-related potentials (ERPs) to binaural beats (BBs), which are the responses of the binaural mechanisms processing TFS of a sound, and the ERPs to diotic amplitude modulation (AM) stimuli, which are the responses of the monaural mechanisms processing the changes in its envelope, were recorded from thirteen young university students and ten senior but active university professors, all with normal hearing in low frequencies. To obtain directly the specific BB responses without confounding monaural frequency change-evoked responses, we used single-cycle BB stimuli with temporary sub-threshold frequency shifts. BBs of a 250-Hz tone and diotic AM of the same tone with similar perceptual salience were presented with 2-second stimulus onset asynchrony. The N1 components of the ERPs to both stimuli displayed notable age-dependent changes in their scalp topography and significant amplitude reduction and latency prolongation in the elderly. These amplitude and latency changes were at similar rates for the two stimulus types, implying that the auditory TFS and envelope processing mechanisms are proportionally affected by physiological aging. These results may serve as control data in future studies investigating the effect of aging-associated cognitive pathologies on auditory TFS processing.

Increased fronto-temporal connectivity by modified melody in real music

In real music, the original melody may appear intact, with little elaboration only, or significantly modified. Since a melody is most easily perceived in music, hearing significantly modified melody may change a brain connectivity. Mozart KV 265 is comprised of a theme with an original melody of “Twinkle Twinkle Little Star” and its significant variations. We studied whether effective connectivity changes with significantly modified melody, between bilateral inferior frontal gyri (IFGs) and Heschl’s gyri (HGs) using magnetoencephalography (MEG). Among the 12 connectivities, the connectivity from the left IFG to the right HG was consistently increased with significantly modified melody compared to the original melody in 2 separate sets of the same rhythmic pattern with different melody (p = 0.005 and 0.034, Bonferroni corrected). Our findings show that the modification of an original melody in a real music changes the brain connectivity.

Attention capture in birds performing an auditory streaming task

Numerous animal models have been used to investigate the neural mechanisms of auditory processing in complex acoustic environments, but it is unclear whether an animal’s auditory attention is functionally similar to a human’s in processing competing auditory scenes. Here we investigated the effects of attention capture in birds performing an objective auditory streaming paradigm. The classical ABAB… patterned pure tone sequences were modified and used for the task. We trained the birds to selectively attend to a target stream and only respond to the deviant appearing in the target stream, even though their attention may be captured by a deviant in the background stream. When no deviant appeared in the background stream, the birds experience the buildup of streaming process in a qualitatively similar way as they did in a subjective paradigm. Although the birds were trained to selectively attend to the target stream, they failed to avoid the involuntary attention switch caused by the background deviant, especially when the background deviant was sequentially unpredictable. Their global performance deteriorated more with increasingly salient background deviants, where the buildup process was reset by the background distractor. Moreover, sequential predictability of the background deviant facilitated the recovery of the buildup process after attention capture. This is the first study that addresses the perceptual consequences of the joint effects of top-down and bottom-up attention in behaving animals.

Prior Knowledge Guides Speech Segregation in Human Auditory Cortex

Segregating concurrent sound streams is a computationally challenging task that requires integrating bottom-up acoustic cues (e.g. pitch) and top-down prior knowledge about sound streams. In a multi-talker environment, the brain can segregate different speakers in about 100 ms in auditory cortex. Here, we used magnetoencephalographic (MEG) recordings to investigate the temporal and spatial signature of how the brain utilizes prior knowledge to segregate 2 speech streams from the same speaker, which can hardly be separated based on bottom-up acoustic cues. In a primed condition, the participants know the target speech stream in advance while in an unprimed condition no such prior knowledge is available. Neural encoding of each speech stream is characterized by the MEG responses tracking the speech envelope. We demonstrate that an effect in bilateral superior temporal gyrus and superior temporal sulcus is much stronger in the primed condition than in the unprimed condition. Priming effects are observed at about 100 ms latency and last more than 600 ms. Interestingly, prior knowledge about the target stream facilitates speech segregation by mainly suppressing the neural tracking of the non-target speech stream. In sum, prior knowledge leads to reliable speech segregation in auditory cortex, even in the absence of reliable bottom-up speech segregation cue.

Cortical mechanisms of talker normalization in fluent sentences

Adjusting to the vocal characteristics of a new talker is important for speech recognition. Previous research has indicated that adjusting to talker differences is an active cognitive process that depends on attention and working memory (WM). These studies have not examined how talker variability affects perception and neural responses in fluent speech. Here we use source analysis from high-density EEG to show that perceiving fluent speech in which the talker changes recruits early involvement of parietal and temporal cortical areas, suggesting functional involvement of WM and attention in talker normalization. We extend these findings to acoustic source change in general by examining understanding environmental sounds in spoken sentence context. Though there may be differences in cortical recruitment to processing demands for non-speech sounds versus a changing talker, the underlying mechanisms are similar, supporting the view that shared cognitive-general mechanisms assist both talker normalization and speech-to-nonspeech transitions.

Estimation of frequency difference at which stream segregation precedes temporal integration as reflected by omission mismatch negativity

Both stream segregation and temporal integration are considered important for auditory scene analysis in the brain. Several previous studies have indicated that stream segregation may precede temporal integration when both processes are required. In the present study, we utilized mismatch negativity (MMN)-which reflects automatic change detection-to systematically estimate the threshold of the frequency difference at which stream segregation occurs prior to temporal integration when these functions occur together during a state of inattention. Electroencephalography (EEG) data were recorded from 22 healthy Japanese men presented with six blocks of alternating high pure tones (high tones) and low pure tones (low tones). Only high tones were omitted with 5 % probability in all blocks. Our results indicated that stream segregation should cancel temporal integration of close sounds, as indicated by omission-MMN elicitation, when the frequency difference is 1000 Hz or larger.

Noise-Induced Change of Cortical Temporal Processing in Cochlear Implant Users

OBJECTIVES

Cochlear implant (CI) users typically report impaired ability to understand speech in noise. Speech understanding in CI users decreases with noise due to reduced temporal processing ability, and speech perceptual errors involve stop consonants distinguished by voice onset time (VOT). The current study examined the effects of noise on various speech perception tests while at the same time used cortical auditory evoked potentials (CAEPs) to quantify the change of neural processing of speech sounds caused by noise. We hypothesized that the noise effects on VOT processing can be reflected in N1/P2 measures, the neural changes relate to behavioral speech perception performances.

METHODS

Ten adult CI users and 15 normal-hearing (NH) people participated in this study. CAEPs were recorded from 64 scalp electrodes in both quiet and noise (signal-to-noise ratio +5 dB) and in passive and active (requiring consonant discrimination) listening. Speech stimulus was synthesized consonant-vowels with VOTs of 0 and 50 ms. N1-P2 amplitudes and latencies were analyzed as a function of listening condition. For the active condition, the P3b also was analyzed. Behavioral measures included a variety of speech perception tasks.

RESULTS

For good performing CI users, performance in most speech test was lower in the presence of noise masking. N1 and P2 latencies became prolonged with noise masking. The P3b amplitudes were smaller in CI groups compared to NH. The degree of P2 latency change (0 vs. 50 ms VOT) was correlated with consonant perception in noise.

CONCLUSION

The effects of noise masking on temporal processing can be reflected in cortical responses in CI users. N1/P2 latencies were more sensitive to noise masking than amplitude measures. Additionally, P2 responses appear to have a better relationship to speech perception in CI users compared to N1.

Aesthetic appreciation of musical intervals enhances behavioural and neurophysiological indexes of attentional engagement and motor inhibition

From Kant to current perspectives in neuroaesthetics, the experience of beauty has been described as disinterested, i.e. focusing on the stimulus perceptual features while neglecting self-referred concerns. At a neurophysiological level, some indirect evidence suggests that disinterested aesthetic appreciation might be associated with attentional enhancement and inhibition of motor behaviour. To test this hypothesis, we performed three auditory-evoked potential experiments, employing consonant and dissonant two-note musical intervals. Twenty-two volunteers judged the beauty of intervals (Aesthetic Judgement task) or responded to them as fast as possible (Detection task). In a third Go-NoGo task, a different group of twenty-two participants had to refrain from responding when hearing intervals. Individual aesthetic judgements positively correlated with response times in the Detection task, with slower motor responses for more appreciated intervals. Electrophysiological indexes of attentional engagement (N1/P2) and motor inhibition (N2/P3) were enhanced for more appreciated intervals. These findings represent the first experimental evidence confirming the disinterested interest hypothesis and may have important applications in research areas studying the effects of stimulus features on learning and motor behaviour.

The Rapid Emergence of Auditory Object Representations in Cortex Reflect Central Acoustic Attributes

Human listeners are bombarded by acoustic information that the brain rapidly organizes into coherent percepts of objects and events in the environment, which aids speech and music perception. The efficiency of auditory object recognition belies the critical constraint that acoustic stimuli necessarily require time to unfold. Using magnetoencephalography, we studied the time course of the neural processes that transform dynamic acoustic information into auditory object representations. Participants listened to a diverse set of 36 tokens comprising everyday sounds from a typical human environment. Multivariate pattern analysis was used to decode the sound tokens from the magnetoencephalographic recordings. We show that sound tokens can be decoded from brain activity beginning 90 msec after stimulus onset with peak decoding performance occurring at 155 msec poststimulus onset. Decoding performance was primarily driven by differences between category representations (e.g., environmental vs. instrument sounds), although within-category decoding was better than chance. Representational similarity analysis revealed that these emerging neural representations were related to harmonic and spectrotemporal differences among the stimuli, which correspond to canonical acoustic features processed by the auditory pathway. Our findings begin to link the processing of physical sound properties with the perception of auditory objects and events in cortex.

Effect of Task and Attention on Neural Tracking of Speech

EEG-based measures of neural tracking of natural running speech are becoming increasingly popular to investigate neural processing of speech and have applications in audiology. When the stimulus is a single speaker, it is usually assumed that the listener actively attends to and understands the stimulus. However, as the level of attention of the listener is inherently variable, we investigated how this affected neural envelope tracking. Using a movie as a distractor, we varied the level of attention while we estimated neural envelope tracking. We varied the intelligibility level by adding stationary noise. We found a significant difference in neural envelope tracking between the condition with maximal attention and the movie condition. This difference was most pronounced in the right-frontal region of the brain. The degree of neural envelope tracking was highly correlated with the stimulus signal-to-noise ratio, even in the movie condition. This could be due to residual neural resources to passively attend to the stimulus. When envelope tracking is used to measure speech understanding objectively, this means that the procedure can be made more enjoyable and feasible by letting participants watch a movie during stimulus presentation.

Neural Signatures of Auditory Perceptual Bistability Revealed by Large-Scale Human Intracranial Recordings

A key challenge in neuroscience is understanding how sensory stimuli give rise to perception, especially when the process is supported by neural activity from an extended network of brain areas. Perception is inherently subjective, so interrogating its neural signatures requires, ideally, a combination of three factors: (1) behavioral tasks that separate stimulus-driven activity from perception per se; (2) human subjects who self-report their percepts while performing those tasks; and (3) concurrent neural recordings acquired at high spatial and temporal resolution. In this study, we analyzed human electrocorticographic recordings obtained during an auditory task which supported mutually exclusive perceptual interpretations. Eight neurosurgical patients (5 male; 3 female) listened to sequences of repeated triplets where tones were separated in frequency by several semitones. Subjects reported spontaneous alternations between two auditory perceptual states, 1-stream and 2-stream, by pressing a button. We compared averaged auditory evoked potentials (AEPs) associated with 1-stream and 2-stream percepts and identified significant differences between them in primary and nonprimary auditory cortex, surrounding auditory-related temporoparietal cortex, and frontal areas. We developed classifiers to identify spatial maps of percept-related differences in the AEP, corroborating findings from statistical analysis. We used one-dimensional embedding spaces to perform the group-level analysis. Our data illustrate exemplar high temporal resolution AEP waveforms in auditory core region; explain inconsistencies in perceptual effects within auditory cortex, reported across noninvasive studies of streaming of triplets; show percept-related changes in frontoparietal areas previously highlighted by studies that focused on perceptual transitions; and demonstrate that auditory cortex encodes maintenance of percepts and switches between them.

SIGNIFICANCE STATEMENT The human brain has the remarkable ability to discern complex and ambiguous stimuli from the external world by parsing mixed inputs into interpretable segments. However, one's perception can deviate from objective reality. But how do perceptual discrepancies occur? What are their anatomical substrates? To address these questions, we performed intracranial recordings in neurosurgical patients as they reported their perception of sounds associated with two mutually exclusive interpretations. We identified signatures of subjective percepts as distinct from sound-driven brain activity in core and non-core auditory cortex and frontoparietal cortex. These findings were compared with previous studies of auditory bistable perception and suggested that perceptual transitions and maintenance of perceptual states were supported by common neural substrates.

Electrical Brain Responses Reveal Sequential Constraints on Planning during Music Performance

Elements in speech and music unfold sequentially over time. To produce sentences and melodies quickly and accurately, individuals must plan upcoming sequence events, as well as monitor outcomes via auditory feedback. We investigated the neural correlates of sequential planning and monitoring processes by manipulating auditory feedback during music performance. Pianists performed isochronous melodies from memory at an initially cued rate while their electroencephalogram was recorded. Pitch feedback was occasionally altered to match either an immediately upcoming Near-Future pitch (next sequence event) or a more distant Far-Future pitch (two events ahead of the current event). Near-Future, but not Far-Future altered feedback perturbed the timing of pianists' performances, suggesting greater interference of Near-Future sequential events with current planning processes. Near-Future feedback triggered a greater reduction in auditory sensory suppression (enhanced response) than Far-Future feedback, reflected in the P2 component elicited by the pitch event following the unexpected pitch change. Greater timing perturbations were associated with enhanced cortical sensory processing of the pitch event following the Near-Future altered feedback. Both types of feedback alterations elicited feedback-related negativity (FRN) and P3a potentials and amplified spectral power in the theta frequency range. These findings suggest similar constraints on producers' sequential planning to those reported in speech production.

Auditory spatial attention modulates the unmasking effect of perceptual separation in a "cocktail party" environment

The perceptual separation between a signal speech and a competing speech (masker), induced by the precedence effect, plays an important role in releasing the signal speech from the masker, especially in a reverberant environment. The perceptual-separation-induced unmasking effect has been suggested to involve multiple cognitive processes, such as selective attention. However, whether listeners' spatial attention modulate the perceptual-separation-induced unmasking effect is not clear. The present study investigated how perceptual separation and auditory spatial attention interact with each other to facilitate speech perception under a simulated noisy and reverberant environment by analyzing the cortical auditory evoked potentials to the signal speech. The results showed that the N1 wave was significantly enhanced by perceptual separation between the signal and masker regardless of whether the participants' spatial attention was directed to the signal or not. However, the P2 wave was significantly enhanced by perceptual separation only when the participants attended to the signal speech. The results indicate that the perceptual-separation-induced facilitation of P2 needs more attentional resource than that of N1. The results also showed that the signal speech caused an enhanced N1 in the contralateral hemisphere regardless of whether participants' attention was directed to the signal or not. In contrast, the signal speech caused an enhanced P2 in the contralateral hemisphere only when the participant attended to the signal. The results indicate that the hemispheric distribution of N1 is mainly affected by the perceptual features of the acoustic stimuli, while that of P2 is affected by the listeners' attentional status.

Auditory Figure-Ground Segregation Is Impaired by High Visual Load

Figure-ground segregation is fundamental to listening in complex acoustic environments. An ongoing debate pertains to whether segregation requires attention or is "automatic" and preattentive. In this magnetoencephalography study, we tested a prediction derived from load theory of attention (e.g., Lavie, 1995) that segregation requires attention but can benefit from the automatic allocation of any "leftover" capacity under low load. Complex auditory scenes were modeled with stochastic figure-ground stimuli (Teki et al., 2013), which occasionally contained repeated frequency component "figures." Naive human participants (both sexes) passively listened to these signals while performing a visual attention task of either low or high load. While clear figure-related neural responses were observed under conditions of low load, high visual load substantially reduced the neural response to the figure in auditory cortex (planum temporale, Heschl's gyrus). We conclude that fundamental figure-ground segregation in hearing is not automatic but draws on resources that are shared across vision and audition.SIGNIFICANCE STATEMENT This work resolves a long-standing question of whether figure-ground segregation, a fundamental process of auditory scene analysis, requires attention or is underpinned by automatic, encapsulated computations. Task-irrelevant sounds were presented during performance of a visual search task. We revealed a clear magnetoencephalography neural signature of figure-ground segregation in conditions of low visual load, which was substantially reduced in conditions of high visual load. This demonstrates that, although attention does not need to be actively allocated to sound for auditory segregation to occur, segregation depends on shared computational resources across vision and hearing. The findings further highlight that visual load can impair the computational capacity of the auditory system, even when it does not simply dampen auditory responses as a whole.

Cortical markers of auditory stream segregation revealed for streaming based on tonotopy but not pitch

The brain decomposes mixtures of sounds, such as competing talkers, into perceptual streams that can be attended to individually. Attention can enhance the cortical representation of streams, but it is unknown what acoustic features the enhancement reflects, or where in the auditory pathways attentional enhancement is first observed. Here, behavioral measures of streaming were combined with simultaneous low- and high-frequency envelope-following responses (EFR) that are thought to originate primarily from cortical and subcortical regions, respectively. Repeating triplets of harmonic complex tones were presented with alternating fundamental frequencies. The tones were filtered to contain either low-numbered spectrally resolved harmonics, or only high-numbered unresolved harmonics. The behavioral results confirmed that segregation can be based on either tonotopic or pitch cues. The EFR results revealed no effects of streaming or attention on subcortical responses. Cortical responses revealed attentional enhancement under conditions of streaming, but only when tonotopic cues were available, not when streaming was based only on pitch cues. The results suggest that the attentional modulation of phase-locked responses is dominated by tonotopically tuned cortical neurons that are insensitive to pitch or periodicity cues.

Cognitive Load Changes during Music Listening and its Implication in Earcon Design in Public Environments: An fNIRS Study

A key for earcon design in public environments is to incorporate an individual’s perceived level of cognitive load for better communication. This study aimed to examine the cognitive load changes required to perform a melodic contour identification task (CIT). While healthy college students (N = 16) were presented with five CITs, behavioral (reaction time and accuracy) and cerebral hemodynamic responses were measured using functional near-infrared spectroscopy. Our behavioral findings showed a gradual increase in cognitive load from CIT1 to CIT3 followed by an abrupt increase between CIT4 (i.e., listening to two concurrent melodic contours in an alternating manner and identifying the direction of the target contour, p < 0.001) and CIT5 (i.e., listening to two concurrent melodic contours in a divided manner and identifying the directions of both contours, p < 0.001). Cerebral hemodynamic responses showed a congruent trend with behavioral findings. Specific to the frontopolar area (Brodmann’s area 10), oxygenated hemoglobin increased significantly between CIT4 and CIT5 (p < 0.05) while the level of deoxygenated hemoglobin decreased. Altogether, the findings indicate that the cognitive threshold for young adults (CIT5) and appropriate tuning of the relationship between timbre and pitch contour can lower the perceived cognitive load and, thus, can be an effective design strategy for earcon in a public environment.

Neuronal correlates of auditory streaming in the auditory cortex of behaving monkeys

This study tested the hypothesis that spiking activity in the primary auditory cortex of monkeys is related to auditory stream formation. Evidence for this hypothesis was previously obtained in animals that were passively exposed to stimuli and in which differences in the streaming percept were confounded with differences between the stimuli. In this study, monkeys performed an operant task on sequences that were composed of light flashes and tones. The tones alternated between a high and a low frequency and could be perceived either as one auditory stream or two auditory streams. The flashes promoted either a one-stream percept or a two-stream percept. Comparison of different types of sequences revealed that the neuronal responses to the alternating tones were more similar when the flashes promoted auditory stream integration, and were more dissimilar when the flashes promoted auditory stream segregation. Thus our findings show that the spiking activity in the monkey primary auditory cortex is related to auditory stream formation.

Neuronal Correlates of Auditory Streaming in Monkey Auditory Cortex for Tone Sequences without Spectral Differences

This study finds a neuronal correlate of auditory perceptual streaming in the primary auditory cortex for sequences of tone complexes that have the same amplitude spectrum but a different phase spectrum. Our finding is based on microelectrode recordings of multiunit activity from 270 cortical sites in three awake macaque monkeys. The monkeys were presented with repeated sequences of a tone triplet that consisted of an A tone, a B tone, another A tone and then a pause. The A and B tones were composed of unresolved harmonics formed by adding the harmonics in cosine phase, in alternating phase, or in random phase. A previous psychophysical study on humans revealed that when the A and B tones are similar, humans integrate them into a single auditory stream; when the A and B tones are dissimilar, humans segregate them into separate auditory streams. We found that the similarity of neuronal rate responses to the triplets was highest when all A and B tones had cosine phase. Similarity was intermediate when the A tones had cosine phase and the B tones had alternating phase. Similarity was lowest when the A tones had cosine phase and the B tones had random phase. The present study corroborates and extends previous reports, showing similar correspondences between neuronal activity in the primary auditory cortex and auditory streaming of sound sequences. It also is consistent with Fishman’s population separation model of auditory streaming.

Neural Decoding of Bistable Sounds Reveals an Effect of Intention on Perceptual Organization

Auditory signals arrive at the ear as a mixture that the brain must decompose into distinct sources based to a large extent on acoustic properties of the sounds. An important question concerns whether listeners have voluntary control over how many sources they perceive. This has been studied using pure high (H) and low (L) tones presented in the repeating pattern HLH-HLH-, which can form a bistable percept heard either as an integrated whole (HLH-) or as segregated into high (H-H-) and low (-L-) sequences. Although instructing listeners to try to integrate or segregate sounds affects reports of what they hear, this could reflect a response bias rather than a perceptual effect. We had human listeners (15 males, 12 females) continuously report their perception of such sequences and recorded neural activity using MEG. During neutral listening, a classifier trained on patterns of neural activity distinguished between periods of integrated and segregated perception. In other conditions, participants tried to influence their perception by allocating attention either to the whole sequence or to a subset of the sounds. They reported hearing the desired percept for a greater proportion of time than when listening neutrally. Critically, neural activity supported these reports; stimulus-locked brain responses in auditory cortex were more likely to resemble the signature of segregation when participants tried to hear segregation than when attempting to perceive integration. These results indicate that listeners can influence how many sound sources they perceive, as reflected in neural responses that track both the input and its perceptual organization.

SIGNIFICANCE STATEMENT Can we consciously influence our perception of the external world? We address this question using sound sequences that can be heard either as coming from a single source or as two distinct auditory streams. Listeners reported spontaneous changes in their perception between these two interpretations while we recorded neural activity to identify signatures of such integration and segregation. They also indicated that they could, to some extent, choose between these alternatives. This claim was supported by corresponding changes in responses in auditory cortex. By linking neural and behavioral correlates of perception, we demonstrate that the number of objects that we perceive can depend not only on the physical attributes of our environment, but also on how we intend to experience it.