Encoding of illusory continuity in primary auditory cortex

Dorsal Anterior Cingulate Cortex Coordinates Contextual Mental Imagery for Single-Beat Manipulation during Rhythmic Sensorimotor Synchronization

Flexible pulse-by-pulse regulation of sensorimotor synchronization is crucial for voluntarily showing rhythmic behaviors synchronously with external cueing; however, the underpinning neurophysiological mechanisms remain unclear. We hypothesized that the dorsal anterior cingulate cortex (dACC) plays a key role by coordinating both proactive and reactive motor outcomes based on contextual mental imagery. To test our hypothesis, a missing-oddball task in finger-tapping paradigms was conducted in 33 healthy young volunteers. The dynamic properties of the dACC were evaluated by event-related deep-brain activity (ER-DBA), supported by event-related potential (ERP) analysis and behavioral evaluation based on signal detection theory. We found that ER-DBA activation/deactivation reflected a strategic choice of motor control modality in accordance with mental imagery. Reverse ERP traces, as omission responses, confirmed that the imagery was contextual. We found that mental imagery was updated only by environmental changes via perceptual evidence and response-based abductive reasoning. Moreover, stable on-pulse tapping was achievable by maintaining proactive control while creating an imagery of syncopated rhythms from simple beat trains, whereas accuracy was degraded with frequent erroneous tapping for missing pulses. We conclude that the dACC voluntarily regulates rhythmic sensorimotor synchronization by utilizing contextual mental imagery based on experience and by creating novel rhythms.

Noises on-How the Brain Deals with Acoustic Noise

What is noise? When does a sound form part of the acoustic background and when might it come to our attention as part of the foreground? Our brain seems to filter out irrelevant sounds in a seemingly effortless process, but how this is achieved remains opaque and, to date, unparalleled by any algorithm. In this review, we discuss how noise can be both background and foreground, depending on what a listener/brain is trying to achieve. We do so by addressing questions concerning the brain's potential bias to interpret certain sounds as part of the background, the extent to which the interpretation of sounds depends on the context in which they are heard, as well as their ethological relevance, task-dependence, and a listener's overall mental state. We explore these questions with specific regard to the implicit, or statistical, learning of sounds and the role of feedback loops between cortical and subcortical auditory structures.

At what level is the gap transfer illusion illusory?

The gap transfer illusion is an auditory phenomenon in which a temporal gap in a longer glide transfers perceptually to a crossing shorter glide, making the longer glide illusorily continuous. This continuity is often considered a variation of classic illusory auditory continuity attributed to auditory peripheral activity, but a new view is given here supported by a series of sound demonstrations indicating that this illusory continuity is purely caused by a higher mechanism of perceptual organization.

Roles of temporal proximity between sound edges in the perceptual organization of veridical and illusory auditory events

The gap transfer illusion is an auditory illusion in which a temporal gap in a long glide is perceived as if it had transferred to a physically continuous shorter glide. The illusion typically occurs when the long and the shorter glide cross each other at their temporal midpoints, where the long glide is physically divided by the gap. The occurrence of the gap transfer illusion was investigated in stimuli in which the duration and the slope of the long glide were 5000 ms and ∼0.8 oct/s. The shorter glide was given different frequency ranges and different temporal ranges, and thus its time-frequency slope was also varied. The overlap configuration of these crossing glides was varied as well. As control stimuli, we used stimuli in which a continuous long glide crossed a shorter glide with a gap, i.e., the opposite configuration of the gap-transfer stimuli as above, as well as stimuli in which both crossing glides were continuous. The perception of two crossing tones tended to be facilitated when the glides differed in duration and/or slope. When the glides were relatively similar in duration and slope, however, bouncing percepts appeared more often. Similarity between the crossing tones thus promoted auditory bouncing, while dissimilarity between them facilitated the crossing percept. If the crossing percept dominated in gap-transfer stimuli, the gap transfer illusion took place in a typical manner, but the illusory transfer of the gap could occur even when the crossing percept was not dominant. When the shorter glide was as short as 500 ms, the crossing percept and the gap transfer illusion were robust. The mechanism of the illusion was examined in terms of factors that can influence the perceptual integration of auditory stimulus edges, i.e., onsets and offsets, of physically different sounds. Much like the perceptual construction of speech units, we suggest that the auditory system utilizes a rough time window of several hundreds of milliseconds to construct an initial skeleton percept of auditory events. The present data indicated the importance of the temporal proximity, rather than the frequency proximity, between sound edges in the illusory tone construction.

Dynamics of the Auditory Continuity Illusion

Illusions give intriguing insights into perceptual and neural dynamics. In the auditory continuity illusion, two brief tones separated by a silent gap may be heard as one continuous tone if a noise burst with appropriate characteristics fills the gap. This illusion probes the conditions under which listeners link related sounds across time and maintain perceptual continuity in the face of sudden changes in sound mixtures. Conceptual explanations of this illusion have been proposed, but its neural basis is still being investigated. In this work we provide a dynamical systems framework, grounded in principles of neural dynamics, to explain the continuity illusion. We construct an idealized firing rate model of a neural population and analyze the conditions under which firing rate responses persist during the interruption between the two tones. First, we show that sustained inputs and hysteresis dynamics (a mismatch between tone levels needed to activate and inactivate the population) can produce continuous responses. Second, we show that transient inputs and bistable dynamics (coexistence of two stable firing rate levels) can also produce continuous responses. Finally, we combine these input types together to obtain neural dynamics consistent with two requirements for the continuity illusion as articulated in a well-known theory of auditory scene analysis: responses persist through the noise-filled gap if noise provides sufficient evidence that the tone continues and if there is no evidence of discontinuities between the tones and noise. By grounding these notions in a quantitative model that incorporates elements of neural circuits (recurrent excitation, and mutual inhibition, specifically), we identify plausible mechanisms for the continuity illusion. Our findings can help guide future studies of neural correlates of this illusion and inform development of more biophysically-based models of the auditory continuity illusion.

Cochlear neural degeneration disrupts hearing in background noise by increasing auditory cortex internal noise

Correlational evidence in humans suggests that selective difficulties hearing in noisy, social settings may reflect premature auditory nerve degeneration. Here, we induced primary cochlear neural degeneration (CND) in adult mice and found direct behavioral evidence for selective detection deficits in background noise. To identify central determinants for this perceptual disorder, we tracked daily changes in ensembles of layer 2/3 auditory cortex parvalbumin-expressing inhibitory neurons and excitatory pyramidal neurons with chronic two-photon calcium imaging. CND induced distinct forms of plasticity in cortical excitatory and inhibitory neurons that culminated in net hyperactivity, increased neural gain, and reduced adaptation to background noise. Ensemble activity measured while mice detected targets in noise could accurately decode whether individual behavioral trials were hits or misses. After CND, random surges of hypercorrelated cortical activity occurring just before target onset reliably predicted impending detection failures, revealing a source of internal cortical noise underlying perceptual difficulties in external noise.

Deafness Weakens Interareal Couplings in the Auditory Cortex

The function of the cerebral cortex essentially depends on the ability to form functional assemblies across different cortical areas serving different functions. Here we investigated how developmental hearing experience affects functional and effective interareal connectivity in the auditory cortex in an animal model with years-long and complete auditory deprivation (deafness) from birth, the congenitally deaf cat (CDC). Using intracortical multielectrode arrays, neuronal activity of adult hearing controls and CDCs was registered in the primary auditory cortex and the secondary posterior auditory field (PAF). Ongoing activity as well as responses to acoustic stimulation (in adult hearing controls) and electric stimulation applied via cochlear implants (in adult hearing controls and CDCs) were analyzed. As functional connectivity measures pairwise phase consistency and Granger causality were used. While the number of coupled sites was nearly identical between controls and CDCs, a reduced coupling strength between the primary and the higher order field was found in CDCs under auditory stimulation. Such stimulus-related decoupling was particularly pronounced in the alpha band and in top–down direction. Ongoing connectivity did not show such a decoupling. These findings suggest that developmental experience is essential for functional interareal interactions during sensory processing. The outcomes demonstrate that corticocortical couplings, particularly top-down connectivity, are compromised following congenital sensory deprivation.

Illusory sound texture reveals multi-second statistical completion in auditory scene analysis

Sound sources in the world are experienced as stable even when intermittently obscured, implying perceptual completion mechanisms that “fill in” missing sensory information. We demonstrate a filling-in phenomenon in which the brain extrapolates the statistics of background sounds (textures) over periods of several seconds when they are interrupted by another sound, producing vivid percepts of illusory texture. The effect differs from previously described completion effects in that 1) the extrapolated sound must be defined statistically given the stochastic nature of texture, and 2) the effect lasts much longer, enabling introspection and facilitating assessment of the underlying representation. Illusory texture biases subsequent texture statistic estimates indistinguishably from actual texture, suggesting that it is represented similarly to actual texture. The illusion appears to represent an inference about whether the background is likely to continue during concurrent sounds, providing a stable statistical representation of the ongoing environment despite unstable sensory evidence.

Auditory textures are sounds defined by a particular statistical distribution, e.g. as is produced by rain, or a swarm of insects. Here, the authors describe a striking perceptual illusion in which sound textures are heard to continue, even though they have in fact been replaced by white noise.

Restoration and Efficiency of the Neural Processing of Continuous Speech Are Promoted by Prior Knowledge

Sufficiently noisy listening conditions can completely mask the acoustic signal of significant parts of a sentence, and yet listeners may still report the perception of hearing the masked speech. This occurs even when the speech signal is removed entirely, if the gap is filled with stationary noise, a phenomenon known as perceptual restoration. At the neural level, however, it is unclear the extent to which the neural representation of missing extended speech sequences is similar to the dynamic neural representation of ordinary continuous speech. Using auditory magnetoencephalography (MEG), we show that stimulus reconstruction, a technique developed for use with neural representations of ordinary speech, works also for the missing speech segments replaced by noise, even when spanning several phonemes and words. The reconstruction fidelity of the missing speech, up to 25% of what would be attained if present, depends however on listeners' familiarity with the missing segment. This same familiarity also speeds up the most prominent stage of the cortical processing of ordinary speech by approximately 5 ms. Both effects disappear when listeners have no or little prior experience with the speech segment. The results are consistent with adaptive expectation mechanisms that consolidate detailed representations about speech sounds as identifiable factors assisting automatic restoration over ecologically relevant timescales.

Reduced low-frequency power and phase locking reflect restoration in the auditory continuity illusion

Interruptions in auditory input can be perceptually restored if they coincide with a masking sound, resulting in a continuity illusion. Previous studies have shown that this continuity illusion is associated with reduced low-frequency neural oscillations in the auditory cortex. However, the precise contribution of oscillatory amplitude changes and phase alignment to auditory restoration remains unclear. Using electroencephalography, we investigated induced power changes and phase locking in response to 3 Hz amplitude-modulated tones during the interval of an interrupting noise. We experimentally manipulated both the physical continuity of the tone (continuous vs. interrupted) and the masking potential of the noise (notched vs. full). We observed an attenuation of 3 Hz power during continuity illusions in comparison with both continuous tones and veridically perceived interrupted tones. This illusion-related suppression of low-frequency oscillations likely reflects a blurring of auditory object boundaries that supports continuity perception. We further observed increased 3 Hz phase locking during fully masked continuous tones compared with the other conditions. This low-frequency phase alignment may reflect the neural registration of the interrupting noise as a newly appearing object, whereas during continuity illusions, a spectral portion of this noise is delegated to filling the interruption. Taken together, our findings suggest that the suppression of slow cortical oscillations in both the power and phase domains supports perceptual restoration of interruptions in auditory input.

Dynamic cortical representations of perceptual filling-in for missing acoustic rhythm

In the phenomenon of perceptual filling-in, missing sensory information can be reconstructed via interpolation or extrapolation from adjacent contextual cues by what is necessarily an endogenous, not yet well understood, neural process. In this investigation, sound stimuli were chosen to allow observation of fixed cortical oscillations driven by contextual (but missing) sensory input, thus entirely reflecting endogenous neural activity. The stimulus employed was a 5 Hz frequency-modulated tone, with brief masker probes (noise bursts) occasionally added. For half the probes, the rhythmic frequency modulation was moreover removed. Listeners reported whether the tone masked by each probe was perceived as being rhythmic or not. Time-frequency analysis of neural responses obtained by magnetoencephalography (MEG) shows that for maskers without the underlying acoustic rhythm, trials where rhythm was nonetheless perceived show higher evoked sustained rhythmic power than trials for which no rhythm was reported. The results support a model in which perceptual filling-in is aided by differential co-modulations of cortical activity at rates directly relevant to human speech communication. We propose that the presence of rhythmically-modulated neural dynamics predicts the subjective experience of a rhythmically modulated sound in real time, even when the perceptual experience is not supported by corresponding sensory data.

A roadmap for the study of conscious audition and its neural basis

How and which aspects of neural activity give rise to subjective perceptual experience-i.e. conscious perception-is a fundamental question of neuroscience. To date, the vast majority of work concerning this question has come from vision, raising the issue of generalizability of prominent resulting theories. However, recent work has begun to shed light on the neural processes subserving conscious perception in other modalities, particularly audition. Here, we outline a roadmap for the future study of conscious auditory perception and its neural basis, paying particular attention to how conscious perception emerges (and of which elements or groups of elements) in complex auditory scenes. We begin by discussing the functional role of the auditory system, particularly as it pertains to conscious perception. Next, we ask: what are the phenomena that need to be explained by a theory of conscious auditory perception? After surveying the available literature for candidate neural correlates, we end by considering the implications that such results have for a general theory of conscious perception as well as prominent outstanding questions and what approaches/techniques can best be used to address them.This article is part of the themed issue 'Auditory and visual scene analysis'.

Tonotopic and Field-Specific Representation of Long-Lasting Sustained Activity in Rat Auditory Cortex

Cortical information processing of the onset, offset, and continuous plateau of an acoustic stimulus should play an important role in acoustic object perception. To date, transient activities responding to the onset and offset of a sound have been well investigated and cortical subfields and topographic representation in these subfields, such as place code of sound frequency, have been well characterized. However, whether these cortical subfields with tonotopic representation are inherited in the sustained activities that follow transient activities and persist during the presentation of a long-lasting stimulus remains unknown, because sustained activities do not exhibit distinct, reproducible, and time-locked responses in their amplitude to be characterized by grand averaging. To address this gap in understanding, we attempted to decode sound information from densely mapped sustained activities in the rat auditory cortex using a sparse parameter estimation method called sparse logistic regression (SLR), and investigated whether and how these activities represent sound information. A microelectrode array with a grid of 10 × 10 recording sites within an area of 4.0 mm × 4.0 mm was implanted in the fourth layer of the auditory cortex in rats under isoflurane anesthesia. Sustained activities in response to long-lasting constant pure tones were recorded. SLR then was applied to discriminate the sound-induced band-specific power or phase-locking value from those of spontaneous activities. The highest decoding performance was achieved in the high-gamma band, indicating that cortical inhibitory interneurons may contribute to the sparse tonotopic representation in sustained activities by mediating synchronous activities. The estimated parameter in the SLR decoding revealed that the informative recording site had a characteristic frequency close to the test frequency. In addition, decoding of the four test frequencies demonstrated that the decoding performance of the SLR deteriorated when the test frequencies were close, supporting the hypothesis that the sustained activities were organized in a tonotopic manner. Finally, unlike transient activities, sustained activities were more informative in the belt than in the core region, indicating that higher-order auditory areas predominate over lower-order areas during sustained activities. Taken together, our results indicate that the auditory cortex processes sound information tonotopically and in a hierarchical manner.

An analysis of nonlinear dynamics underlying neural activity related to auditory induction in the rat auditory cortex

A sound interrupted by silence is perceived as discontinuous. However, when high-intensity noise is inserted during the silence, the missing sound may be perceptually restored and be heard as uninterrupted. This illusory phenomenon is called auditory induction. Recent electrophysiological studies have revealed that auditory induction is associated with the primary auditory cortex (A1). Although experimental evidence has been accumulating, the neural mechanisms underlying auditory induction in A1 neurons are poorly understood. To elucidate this, we used both experimental and computational approaches. First, using an optical imaging method, we characterized population responses across auditory cortical fields to sound and identified five subfields in rats. Next, we examined neural population activity related to auditory induction with high temporal and spatial resolution in the rat auditory cortex (AC), including the A1 and several other AC subfields. Our imaging results showed that tone-burst stimuli interrupted by a silent gap elicited early phasic responses to the first tone and similar or smaller responses to the second tone following the gap. In contrast, tone stimuli interrupted by broadband noise (BN), considered to cause auditory induction, considerably suppressed or eliminated responses to the tone following the noise. Additionally, tone-burst stimuli that were interrupted by notched noise centered at the tone frequency, which is considered to decrease the strength of auditory induction, partially restored the second responses from the suppression caused by BN. To phenomenologically mimic the neural population activity in the A1 and thus investigate the mechanisms underlying auditory induction, we constructed a computational model from the periphery through the AC, including a nonlinear dynamical system. The computational model successively reproduced some of the above-mentioned experimental results. Therefore, our results suggest that a nonlinear, self-exciting system is a key element for qualitatively reproducing A1 population activity and to understand the underlying mechanisms.

Ambiguity involving two illusory melodies induced by a simple configuration of tones

Auditory scene analysis is essential in daily life to extract necessary information from complex acoustic environment and also from intricate development of music compositions. Auditory illusions and ambiguity are important factors in auditory scene analysis and have been studied extensively. We here report a novel form of ambiguity involving two illusory melodies implied by a very simple stimulus consisting of two sustained tones of different frequencies and an intermittently repeated tone of a frequency between the sustained tones. The measured time elapsed before a first perception change showed that illusion, ambiguity and disambiguation actually took place. We anticipate that the proposed illusion and ambiguity will provide a well-controlled approach for behavioral and neurophysiological studies of music recognition because of the simplicity of stimulus.

Language processing is not a race against time

We agree with Christiansen & Chater (C&C) that language processing and acquisition are tightly constrained by the limits of sensory and memory systems. However, the human brain supports a range of cognitive functions that mitigate the effects of information processing bottlenecks. The language system is partly organised around these moderating factors, not just around restrictions on storage and computation.

Female túngara frogs do not experience the continuity illusion

In humans and some nonhuman vertebrates, a sound containing brief silent gaps can be rendered perceptually continuous by inserting noise into the gaps. This so-called "continuity illusion" arises from a phenomenon known as "auditory induction" and results in the perception of complete auditory objects despite fragmentary or incomplete acoustic information. Previous studies of auditory induction in gray treefrogs (Hyla versicolor and H. chrysoscelis) have demonstrated an absence of this phenomenon. These treefrog species produce pulsatile (noncontinuous) vocalizations, whereas studies of auditory induction in other taxa, including humans, often present continuous sounds (e.g., frequency-modulated sweeps). This study investigated the continuity illusion in a frog (Physalaemus pustulosus) with an advertisement vocalization that is naturally continuous and thus similar to the tonal sweeps used in human psychophysical studies of auditory induction. In a series of playback experiments, female subjects were presented with sets of stimuli that included complete calls, calls with silent gaps, and calls with silent gaps filled with noise. The results failed to provide evidence of auditory induction. Current evidence, therefore, suggests that mammals and birds experience auditory induction, but frogs may not. This emerging pattern of taxonomic differences is considered in light of potential methodological, neurophysiological, and functional explanations.

Behavioral correlates of auditory streaming in rhesus macaques

Perceptual representations of auditory stimuli (i.e., sounds) are derived from the auditory system's ability to segregate and group the spectral, temporal, and spatial features of auditory stimuli-a process called "auditory scene analysis". Psychophysical studies have identified several of the principles and mechanisms that underlie a listener's ability to segregate and group acoustic stimuli. One important psychophysical task that has illuminated many of these principles and mechanisms is the "streaming" task. Despite the wide use of this task to study psychophysical mechanisms of human audition, no studies have explicitly tested the streaming abilities of non-human animals using the standard methodologies employed in human-audition studies. Here, we trained rhesus macaques to participate in the streaming task using methodologies and controls similar to those presented in previous human studies. Overall, we found that the monkeys' behavioral reports were qualitatively consistent with those of human listeners, thus suggesting that this task may be a valuable tool for future neurophysiological studies.

The what, where and how of auditory-object perception

The fundamental perceptual unit in hearing is the 'auditory object'. Similar to visual objects, auditory objects are the computational result of the auditory system's capacity to detect, extract, segregate and group spectrotemporal regularities in the acoustic environment; the multitude of acoustic stimuli around us together form the auditory scene. However, unlike the visual scene, resolving the component objects within the auditory scene crucially depends on their temporal structure. Neural correlates of auditory objects are found throughout the auditory system. However, neural responses do not become correlated with a listener's perceptual reports until the level of the cortex. The roles of different neural structures and the contribution of different cognitive states to the perception of auditory objects are not yet fully understood.

Illusory auditory continuity despite neural evidence to the contrary

Many previous studies have shown that a tone that is momentarily -interrupted can be perceived as continuous if the interruption is completely masked by noise. It has been suggested this "continuity illusion" occurs only when peripheral neural responses contain no evidence that the signal was interrupted. In this study, we used a combination of psychophysical measures and computational simulations of peripheral auditory responses to examine whether the continuity illusion can be experienced under conditions where peripheral neural responses contain evidence that the signal did not continue through the masker. Our results provide an example of a salient continuity illusion despite evidence of an interruption in the peripheral representation, indicating that the illusion may depend more on global features of the interrupting sound, such as its long-term specific loudness, than on its fine-grained temporal structure.

No time-stretching illusion when a tone is followed by a noise

A sine tone is perceived as longer when it is preceded by a more intense noise than when presented in isolation. This is the time-stretching illusion. We conducted an experiment where the method of constant stimuli was used to examine whether a tone would also be stretched when it was followed by a noise. The duration of a tone was overestimated when it was preceded by a noise, but not when followed by a noise or when located between two consecutive noises. Moreover, the increasing of the noise intensity (from -6 to +6 dB) relative to the tone intensity resulted in larger overestimations, but only in the condition where a tone was preceded by a noise. In brief, the duration of a tone is stretched when this tone is preceded by a noise and if this tone is not followed by a noise.

Attention to natural auditory signals

The challenge of understanding how the brain processes natural signals is compounded by the fact that such signals are often tied closely to specific natural behaviors and natural environments. This added complexity is especially true for auditory communication signals that can carry information at multiple hierarchical levels, and often occur in the context of other competing communication signals. Selective attention provides a mechanism to focus processing resources on specific components of auditory signals, and simultaneously suppress responses to unwanted signals or noise. Although selective auditory attention has been well-studied behaviorally, very little is known about how selective auditory attention shapes the processing on natural auditory signals, and how the mechanisms of auditory attention are implemented in single neurons or neural circuits. Here we review the role of selective attention in modulating auditory responses to complex natural stimuli in humans. We then suggest how the current understanding can be applied to the study of selective auditory attention in the context natural signal processing at the level of single neurons and populations in animal models amenable to invasive neuroscience techniques. This article is part of a Special Issue entitled "Communication Sounds and the Brain: New Directions and Perspectives".

Functional imaging of auditory scene analysis

Our auditory system is constantly faced with the task of decomposing the complex mixture of sound arriving at the ears into perceptually independent streams constituting accurate representations of individual sound sources. This decomposition, termed auditory scene analysis, is critical for both survival and communication, and is thought to underlie both speech and music perception. The neural underpinnings of auditory scene analysis have been studied utilizing invasive experiments with animal models as well as non-invasive (MEG, EEG, and fMRI) and invasive (intracranial EEG) studies conducted with human listeners. The present article reviews human neurophysiological research investigating the neural basis of auditory scene analysis, with emphasis on two classical paradigms termed streaming and informational masking. Other paradigms - such as the continuity illusion, mistuned harmonics, and multi-speaker environments - are briefly addressed thereafter. We conclude by discussing the emerging evidence for the role of auditory cortex in remapping incoming acoustic signals into a perceptual representation of auditory streams, which are then available for selective attention and further conscious processing. This article is part of a Special Issue entitled Human Auditory Neuroimaging.

Frequency-change aftereffect produced by adaptation to real and illusory unidirectional frequency sweeps

It was examined whether illusory and real continuities induce the frequency-change aftereffect, in which repeated exposure to a frequency sweep results in a shift in the perceived frequency change direction of a subsequent test sound. The magnitude of the aftereffect for different types of adaptors ("real sweep," "illusory sweep," and "sweep with gap") was compared. Listeners judged the direction of a frequency change of the test sound and showed a significant aftereffect only for the "real sweep" adaptors. The results suggest that the illusory sweeps are processed after the stage of frequency-change detection.

You can't stop the music: reduced auditory alpha power and coupling between auditory and memory regions facilitate the illusory perception of music during noise

Our brain has the capacity of providing an experience of hearing even in the absence of auditory stimulation. This can be seen as illusory conscious perception. While increasing evidence postulates that conscious perception requires specific brain states that systematically relate to specific patterns of oscillatory activity, the relationship between auditory illusions and oscillatory activity remains mostly unexplained. To investigate this we recorded brain activity with magnetoencephalography and collected intracranial data from epilepsy patients while participants listened to familiar as well as unknown music that was partly replaced by sections of pink noise. We hypothesized that participants have a stronger experience of hearing music throughout noise when the noise sections are embedded in familiar compared to unfamiliar music. This was supported by the behavioral results showing that participants rated the perception of music during noise as stronger when noise was presented in a familiar context. Time-frequency data show that the illusory perception of music is associated with a decrease in auditory alpha power pointing to increased auditory cortex excitability. Furthermore, the right auditory cortex is concurrently synchronized with the medial temporal lobe, putatively mediating memory aspects associated with the music illusion. We thus assume that neuronal activity in the highly excitable auditory cortex is shaped through extensive communication between the auditory cortex and the medial temporal lobe, thereby generating the illusion of hearing music during noise.

Sustained firing of model central auditory neurons yields a discriminative spectro-temporal representation for natural sounds

The processing characteristics of neurons in the central auditory system are directly shaped by and reflect the statistics of natural acoustic environments, but the principles that govern the relationship between natural sound ensembles and observed responses in neurophysiological studies remain unclear. In particular, accumulating evidence suggests the presence of a code based on sustained neural firing rates, where central auditory neurons exhibit strong, persistent responses to their preferred stimuli. Such a strategy can indicate the presence of ongoing sounds, is involved in parsing complex auditory scenes, and may play a role in matching neural dynamics to varying time scales in acoustic signals. In this paper, we describe a computational framework for exploring the influence of a code based on sustained firing rates on the shape of the spectro-temporal receptive field (STRF), a linear kernel that maps a spectro-temporal acoustic stimulus to the instantaneous firing rate of a central auditory neuron. We demonstrate the emergence of richly structured STRFs that capture the structure of natural sounds over a wide range of timescales, and show how the emergent ensembles resemble those commonly reported in physiological studies. Furthermore, we compare ensembles that optimize a sustained firing code with one that optimizes a sparse code, another widely considered coding strategy, and suggest how the resulting population responses are not mutually exclusive. Finally, we demonstrate how the emergent ensembles contour the high-energy spectro-temporal modulations of natural sounds, forming a discriminative representation that captures the full range of modulation statistics that characterize natural sound ensembles. These findings have direct implications for our understanding of how sensory systems encode the informative components of natural stimuli and potentially facilitate multi-sensory integration.

Effect of flanking sounds on the auditory continuity illusion

Background

The auditory continuity illusion or the perceptual restoration of a target sound briefly interrupted by an extraneous sound has been shown to depend on masking. However, little is known about factors other than masking.

Methodology/Principal Findings

We examined whether a sequence of flanking transient sounds affects the apparent continuity of a target tone alternated with a bandpass noise at regular intervals. The flanking sounds significantly increased the limit of perceiving apparent continuity in terms of the maximum target level at a fixed noise level, irrespective of the frequency separation between the target and flanking sounds: the flanking sounds enhanced the continuity illusion. This effect was dependent on the temporal relationship between the flanking sounds and noise bursts.

Conclusions/Significance

The spectrotemporal characteristics of the enhancement effect suggest that a mechanism to compensate for exogenous attentional distraction may contribute to the continuity illusion.

Behavioral evidence for auditory induction in a species of rodent: Mongolian gerbil (Meriones unguiculatus)

When a segment of sound of interest is interrupted by a loud extraneous noise, humans perceive that the missing sound continues during the intrusive noise. This restoration of auditory information occurs in perceptions of both speech and non-speech sounds (e.g., tone bursts), a phenomenon referred to as auditory induction. In this study, Mongolian gerbils were trained with standard Go/No-Go operant conditioning to discriminate continuous tone bursts (the Go stimulus) from tone bursts with a silent gap in the middle (the No-Go stimulus). Noise was added to Go and No-Go stimuli to determine the condition under which induction would occur. The Mongolian gerbils engaged in Go responses to No-Go stimuli only when the noise spectrally surrounding the tone was of the same duration as the silent portion of the No-Go stimulus; these results match those previously reported in primates (humans and macaque monkeys). The result presents not only the evidence of the auditory induction in a rodent species but also suggests that similar mechanisms for restoring missing sounds are shared among mammals. Additionally, our findings demonstrated that the rodent can serve as a valuable animal model for future studies of perceptual restoration.

Brain state-triggered stimulus delivery: An efficient tool for probing ongoing brain activity

What is the relationship between variability in ongoing brain activity preceding a sensory stimulus and subsequent perception of that stimulus? A challenge in the study of this key topic in systems neuroscience is the relative rarity of certain brain 'states'-left to chance, they may seldom align with sensory presentation. We developed a novel method for studying the influence of targeted brain states on subsequent perceptual performance by online identification of spatiotemporal brain activity patterns of interest, and brain-state triggered presentation of subsequent stimuli. This general method was applied to an electroencephalography study of human auditory selective listening. We obtained online, time-varying estimates of the instantaneous direction of neural bias (towards processing left or right ear sounds). Detection of target sounds was influenced by pre-target fluctuations in neural bias, within and across trials. We propose that brain state-triggered stimulus delivery will enable efficient, statistically tractable studies of rare patterns of ongoing activity in single neurons and distributed neural circuits, and their influence on subsequent behavioral and neural responses.

Tangent bundle curve completion with locally connected parallel networks

We propose a theory for cortical representation and computation of visually completed curves that are generated by the visual system to fill in missing visual information (e.g., due to occlusions). Recent computational theories and physiological evidence suggest that although such curves do not correspond to explicit image evidence along their length, their construction emerges from corresponding activation patterns of orientation-selective cells in the primary visual cortex. Previous theoretical work modeled these patterns as least energetic 3D curves in the mathematical continuous space R2 × S1, which abstracts the mammalian striate cortex. Here we discuss the biological plausibility of this theory and present a neural architecture that implements it with locally connected parallel networks. Part of this contribution is also a first attempt to bridge the physiological literature on curve completion with the shape problem and a shape theory. We present completion simulations of our model in natural and synthetic scenes and discuss various observations and predictions that emerge from this theory in the context of curve completion.

Spatiotemporal dynamics of neural activity related to auditory induction in the core and belt fields of guinea-pig auditory cortex

Auditory induction is a continuity illusion in which missing sounds are perceived under appropriate conditions, for example, when noise is inserted during silent gaps in the sound. To elucidate the neural mechanisms underlying auditory induction, neural responses to tones interrupted by a silent gap or noise were examined in the core and belt fields of the auditory cortex using real-time optical imaging with a voltage-sensitive dye. Tone stimuli interrupted by a silent gap elicited responses to the second tone following the gap as well as early phasic responses to the first tone. Tone stimuli interrupted by broad-band noise (BN), considered to cause auditory induction, considerably reduced or eliminated responses to the tone following the noise. This reduction was stronger in the dorsocaudal field (field DC) and belt fields compared with the anterior field (the primary auditory cortex of guinea pig). Tone stimuli interrupted by notched (band-stopped) noise centered at the tone frequency, considered to decrease the strength of auditory induction, partially restored the second responses from the suppression caused by BN. These results suggest that substantial changes between responses to silent gap-inserted tones and those to BN-inserted tones emerged in field DC and belt fields. Moreover, the findings indicate that field DC is the first area in which these changes emerge, suggesting that it may be an important region for auditory induction of simple sounds.

Hearing an illusory vowel in noise: suppression of auditory cortical activity

Human hearing is constructive. For example, when a voice is partially replaced by an extraneous sound (e.g., on the telephone due to a transmission problem), the auditory system may restore the missing portion so that the voice can be perceived as continuous (Miller and Licklider, 1950; for review, see Bregman, 1990; Warren, 1999). The neural mechanisms underlying this continuity illusion have been studied mostly with schematic stimuli (e.g., simple tones) and are still a matter of debate (for review, see Petkov and Sutter, 2011). The goal of the present study was to elucidate how these mechanisms operate under more natural conditions. Using psychophysics and electroencephalography (EEG), we assessed simultaneously the perceived continuity of a human vowel sound through interrupting noise and the concurrent neural activity. We found that vowel continuity illusions were accompanied by a suppression of the 4 Hz EEG power in auditory cortex (AC) that was evoked by the vowel interruption. This suppression was stronger than the suppression accompanying continuity illusions of a simple tone. Finally, continuity perception and 4 Hz power depended on the intactness of the sound that preceded the vowel (i.e., the auditory context). These findings show that a natural sound may be restored during noise due to the suppression of 4 Hz AC activity evoked early during the noise. This mechanism may attenuate sudden pitch changes, adapt the resistance of the auditory system to extraneous sounds across auditory scenes, and provide a useful model for assisted hearing devices.

β- And γ-band EEG power predicts illusory auditory continuity perception

Because acoustic landscapes are complex and rapidly changing, auditory systems have evolved mechanisms that permit rapid detection of novel sounds, sound source segregation, and perceptual restoration of sounds obscured by noise. Perceptual restoration is particularly important in noisy environments because it allows organisms to track sounds over time even when they are masked. The continuity illusion is a striking example of perceptual restoration with sounds perceived as intact even when parts of them have been replaced by gaps and rendered inaudible by being masked by an extraneous sound. The mechanisms of auditory filling-in are complex and are currently not well-understood. The present study used the high temporal resolution of EEG to examine brain activity related to continuity illusion perception. Masking noise loudness was adjusted individually for each subject so that physically identical sounds on some trials elicited a continuity illusion (failure to detect a gap in a sound) and on other trials resulted in correct gap detection. This design ensured that any measurable differences in brain activity would be due to perceptual differences rather than physical differences among stimuli. We found that baseline activity recorded immediately before presentation of the stimulus significantly predicted the occurrence of the continuity illusion in 10 out of 14 participants based on power differences in γ-band EEG (34-80 Hz). Across all participants, power in the β and γ (12- to 80-Hz range) was informative about the subsequent perceptual decision. These data suggest that a subject's baseline brain state influences the strength of continuity illusions.

Active engagement improves primary auditory cortical neurons' ability to discriminate temporal modulation

The effect of attention on single neuron responses in the auditory system is unresolved. We found that when monkeys discriminated temporally amplitude modulated (AM) from unmodulated sounds, primary auditory cortical (A1) neurons better discriminated those sounds than when the monkeys were not discriminating them. This was observed for both average firing rate and vector strength (VS), a measure of how well neurons temporally follow the stimulus' temporal modulation. When data were separated by nonsynchronized and synchronized responses, the firing rate of nonsynchronized responses best distinguished AM- noise from unmodulated noise, followed by VS for synchronized responses, with firing rate for synchronized neurons providing the poorest AM discrimination. Firing rate-based AM discrimination for synchronized neurons, however, improved most with task engagement, showing that the least sensitive code in the passive condition improves the most with task engagement. Rate coding improved due to larger increases in absolute firing rate at higher modulation depths than for lower depths and unmodulated sounds. Relative to spontaneous activity (which increased with engagement), the response to unmodulated sounds decreased substantially. The temporal coding improvement--responses more precisely temporally following a stimulus when animals were required to attend to it--expands the framework of possible mechanisms of attention to include increasing temporal precision of stimulus following. These findings provide a crucial step to understanding the coding of temporal modulation and support a model in which rate and temporal coding work in parallel, permitting a multiplexed code for temporal modulation, and for a complementary representation of rate and temporal coding.

Multistability in auditory stream segregation: a predictive coding view

Auditory stream segregation involves linking temporally separate acoustic events into one or more coherent sequences. For any non-trivial sequence of sounds, many alternative descriptions can be formed, only one or very few of which emerge in awareness at any time. Evidence from studies showing bi-/multistability in auditory streaming suggest that some, perhaps many of the alternative descriptions are represented in the brain in parallel and that they continuously vie for conscious perception. Here, based on a predictive coding view, we consider the nature of these sound representations and how they compete with each other. Predictive processing helps to maintain perceptual stability by signalling the continuation of previously established patterns as well as the emergence of new sound sources. It also provides a measure of how well each of the competing representations describes the current acoustic scene. This account of auditory stream segregation has been tested on perceptual data obtained in the auditory streaming paradigm.

Global not local masker features govern the auditory continuity illusion

When an acoustic signal is temporarily interrupted by another sound, it is sometimes heard as continuing through, even when the signal is actually turned off during the interruption-an effect known as the "auditory continuity illusion." A widespread view is that the illusion can only occur when peripheral neural responses contain no evidence that the signal was interrupted. Here we challenge this view using a combination of psychophysical measures from human listeners and computational simulations with a model of the auditory periphery. The results reveal that the illusion seems to depend more on the overall specific loudness than on the peripheral masking properties of the interrupting sound. This finding indicates that the continuity illusion is determined by the global features, rather than the fine-grained temporal structure, of the interrupting sound, and argues against the view that the illusion arises in the auditory periphery.

Neural mechanisms of rhythmic masking release in monkey primary auditory cortex: implications for models of auditory scene analysis

The ability to detect and track relevant acoustic signals embedded in a background of other sounds is crucial for hearing in complex acoustic environments. This ability is exemplified by a perceptual phenomenon known as "rhythmic masking release" (RMR). To demonstrate RMR, a sequence of tones forming a target rhythm is intermingled with physically identical "Distracter" sounds that perceptually mask the rhythm. The rhythm can be "released from masking" by adding "Flanker" tones in adjacent frequency channels that are synchronous with the Distracters. RMR represents a special case of auditory stream segregation, whereby the target rhythm is perceptually segregated from the background of Distracters when they are accompanied by the synchronous Flankers. The neural basis of RMR is unknown. Previous studies suggest the involvement of primary auditory cortex (A1) in the perceptual organization of sound patterns. Here, we recorded neural responses to RMR sequences in A1 of awake monkeys in order to identify neural correlates and potential mechanisms of RMR. We also tested whether two current models of stream segregation, when applied to these responses, could account for the perceptual organization of RMR sequences. Results suggest a key role for suppression of Distracter-evoked responses by the simultaneous Flankers in the perceptual restoration of the target rhythm in RMR. Furthermore, predictions of stream segregation models paralleled the psychoacoustics of RMR in humans. These findings reinforce the view that preattentive or "primitive" aspects of auditory scene analysis may be explained by relatively basic neural mechanisms at the cortical level.

Neural restoration of degraded audiovisual speech

When speech is interrupted by noise, listeners often perceptually "fill-in" the degraded signal, giving an illusion of continuity and improving intelligibility. This phenomenon involves a neural process in which the auditory cortex (AC) response to onsets and offsets of acoustic interruptions is suppressed. Since meaningful visual cues behaviorally enhance this illusory filling-in, we hypothesized that during the illusion, lip movements congruent with acoustic speech should elicit a weaker AC response to interruptions relative to static (no movements) or incongruent visual speech. AC response to interruptions was measured as the power and inter-trial phase consistency of the auditory evoked theta band (4-8 Hz) activity of the electroencephalogram (EEG) and the N1 and P2 auditory evoked potentials (AEPs). A reduction in the N1 and P2 amplitudes and in theta phase-consistency reflected the perceptual illusion at the onset and/or offset of interruptions regardless of visual condition. These results suggest that the brain engages filling-in mechanisms throughout the interruption, which repairs degraded speech lasting up to ~250 ms following the onset of the degradation. Behaviorally, participants perceived speech continuity over longer interruptions for congruent compared to incongruent or static audiovisual streams. However, this specific behavioral profile was not mirrored in the neural markers of interest. We conclude that lip-reading enhances illusory perception of degraded speech not by altering the quality of the AC response, but by delaying it during degradations so that longer interruptions can be tolerated.

Developmental neuroplasticity after cochlear implantation

Cortical development is dependent on stimulus-driven learning. The absence of sensory input from birth, as occurs in congenital deafness, affects normal growth and connectivity needed to form a functional sensory system, resulting in deficits in oral language learning. Cochlear implants bypass cochlear damage by directly stimulating the auditory nerve and brain, making it possible to avoid many of the deleterious effects of sensory deprivation. Congenitally deaf animals and children who receive implants provide a platform to examine the characteristics of cortical plasticity in the auditory system. In this review, we discuss the existence of time limits for, and mechanistic constraints on, sensitive periods for cochlear implantation and describe the effects of multimodal and cognitive reorganization that result from long-term auditory deprivation.

Effects of inducer continuity on auditory stream segregation: comparison of physical and perceived continuity in different contexts

The factors influencing the stream segregation of discrete tones and the perceived continuity of discrete tones as continuing through an interrupting masker are well understood as separate phenomena. Two experiments tested whether perceived continuity can influence the build-up of stream segregation by manipulating the perception of continuity during an induction sequence and measuring streaming in a subsequent test sequence comprising three triplets of low and high frequency tones (LHL-[ellipsis (horizontal)]). For experiment 1, a 1.2-s standard induction sequence comprising six 100-ms L-tones strongly promoted segregation, whereas a single extended L-inducer (1.1 s plus 100-ms silence) did not. Segregation was similar to that following the single extended inducer when perceived continuity was evoked by inserting noise bursts between the individual tones. Reported segregation increased when the noise level was reduced such that perceived continuity no longer occurred. Experiment 2 presented a 1.3-s continuous inducer created by bridging the 100-ms silence between an extended L-inducer and the first test-sequence tone. This configuration strongly promoted segregation. Segregation was also increased by filling the silence after the extended inducer with noise, such that it was perceived like a bridging inducer. Like physical continuity, perceived continuity can promote or reduce test-sequence streaming, depending on stimulus context.

The Continuity Illusion Does Not Depend on Attentional State: fMRI Evidence from Illusory Vowels

We investigate whether the neural correlates of the continuity illusion, as measured using fMRI, are modulated by attention. As we have shown previously, when two formants of a synthetic vowel are presented in an alternating pattern, the vowel can be identified if the gaps in each formant are filled with bursts of plausible masking noise, causing the illusory percept of a continuous vowel (“Illusion” condition). When the formant-to-noise ratio is increased so that noise no longer plausibly masks the formants, the formants are heard as interrupted (“Illusion Break” condition) and vowels are not identifiable. A region of the left middle temporal gyrus (MTG) is sensitive both to intact synthetic vowels (two formants present simultaneously) and to Illusion stimuli, compared to Illusion Break stimuli. Here, we compared these conditions in the presence and absence of attention. We examined fMRI signal for different sound types under three attentional conditions: full attention to the vowels; attention to a visual distracter; or attention to an auditory distracter. Crucially, although a robust main effect of attentional state was observed in many regions, the effect of attention did not differ systematically for the illusory vowels compared to either intact vowels or to the Illusion Break stimuli in the left STG/MTG vowel-sensitive region. This result suggests that illusory continuity of vowels is an obligatory perceptual process, and operates independently of attentional state. An additional finding was that the sensitivity of primary auditory cortex to the number of sound onsets in the stimulus was modulated by attention.

Perceived tonal continuity through two noise bursts separated by silence

Three experiments measured the perceived continuity of two pure tones "flankers" through a masker containing a silence. Experiment 1 used a 2I-2AFC procedure; one interval contained two noise bursts separated by a silent gap, and the other contained two noise bursts separated by a tone of the same duration as the silence. Discrimination between masker conditions was very accurate when the flankers were absent but was impaired substantially when the flankers were present. This was taken as evidence that illusory flanker continuity during the silent gap was heard as similar to the physical presence of a tone in the gap. In experiment 2, performance remained poor when the flankers were frequency glides aligned along a common trajectory. Performance improved significantly when the flankers were misaligned in trajectory. In experiment 3, listeners rated directly perceived flanker continuity. Strong continuity was reported in the silent gap conditions for which poor performance had been observed in experiments 1 and 2. These findings show that continuity may be heard through a masker that cannot mask a physically continuous tone but can mask the flankers' offset and onset. The results are explained in terms of the perceptual grouping of onsets and offsets of the flankers.

Binaural sensitivity changes between cortical on and off responses

Neurons exhibiting on and off responses with different frequency tuning have previously been described in the primary auditory cortex (A1) of anesthetized and awake animals, but it is unknown whether other tuning properties, including sensitivity to binaural localization cues, also differ between on and off responses. We measured the sensitivity of A1 neurons in anesthetized ferrets to 1) interaural level differences (ILDs), using unmodulated broadband noise with varying ILDs and average binaural levels, and 2) interaural time delays (ITDs), using sinusoidally amplitude-modulated broadband noise with varying envelope ITDs. We also assessed fine-structure ITD sensitivity and frequency tuning, using pure-tone stimuli. Neurons most commonly responded to stimulus onset only, but purely off responses and on-off responses were also recorded. Of the units exhibiting significant binaural sensitivity nearly one-quarter showed binaural sensitivity in both on and off responses, but in almost all (∼97%) of these units the binaural tuning of the on responses differed significantly from that seen in the off responses. Moreover, averaged, normalized ILD and ITD tuning curves calculated from all units showing significant sensitivity to binaural cues indicated that on and off responses displayed different sensitivity patterns across the population. A principal component analysis of ITD response functions suggested a continuous cortical distribution of binaural sensitivity, rather than discrete response classes. Rather than reflecting a release from inhibition without any functional significance, we propose that binaural off responses may be important to cortical encoding of sound-source location.

Individual differences in sound-in-noise perception are related to the strength of short-latency neural responses to noise

Important sounds can be easily missed or misidentified in the presence of extraneous noise. We describe an auditory illusion in which a continuous ongoing tone becomes inaudible during a brief, non-masking noise burst more than one octave away, which is unexpected given the frequency resolution of human hearing. Participants strongly susceptible to this illusory discontinuity did not perceive illusory auditory continuity (in which a sound subjectively continues during a burst of masking noise) when the noises were short, yet did so at longer noise durations. Participants who were not prone to illusory discontinuity showed robust early electroencephalographic responses at 40-66 ms after noise burst onset, whereas those prone to the illusion lacked these early responses. These data suggest that short-latency neural responses to auditory scene components reflect subsequent individual differences in the parsing of auditory scenes.

Recalibration of the auditory continuity illusion: sensory and decisional effects

An interrupted sound can be perceived as continuous when noise masks the interruption, creating an illusion of continuity. Recent findings have shown that adaptor sounds preceding an ambiguous target sound can influence listeners' rating of target continuity. However, it remains unclear whether these aftereffects on perceived continuity influence sensory processes, decisional processes (i.e., criterion shifts), or both. The present study addressed this question. Results show that the target sound was more likely to be rated as 'continuous' when preceded by adaptors that were perceived as clearly discontinuous than when it was preceded by adaptors that were heard (illusorily or veridically) as continuous. Detection-theory analyses indicated that these contrastive aftereffects reflect a combination of sensory and decisional processes. The contrastive sensory aftereffect persisted even when adaptors and targets were presented to opposite ears, suggesting a neural origin in structures that receive binaural inputs. Finally, physically identical but perceptually ambiguous adaptors that were rated as 'continuous' induced more reports of target continuity than adaptors that were rated as 'discontinuous'. This assimilative aftereffect was purely decisional. These findings confirm that judgments of auditory continuity can be influenced by preceding events, and reveal that these aftereffects have both sensory and decisional components.