The neuronal representation of pitch in primate auditory cortex

Reliability and stability of tactile perception in the whisker somatosensory system

Rodents rely on their whiskers as vital sensory tools for tactile perception, enabling them to distinguish textures and shapes. Ensuring the reliability and constancy of tactile perception under varying stimulus conditions remains a fascinating and fundamental inquiry. This study explores the impact of stimulus configurations, including whisker movement velocity and object spatial proximity, on texture discrimination and stability in rats. To address this issue, we employed three distinct approaches for our investigation. Stimulus configurations notably affected tactile inputs, altering whisker vibration's kinetic and kinematic aspects with consistent effects across various textures. Through a texture discrimination task, rats exhibited consistent discrimination performance irrespective of changes in stimulus configuration. However, alterations in stimulus configuration significantly affected the rats' ability to maintain stability in texture perception. Additionally, we investigated the influence of stimulus configurations on cortical neuronal responses by manipulating them experimentally. Notably, cortical neurons demonstrated substantial and intricate changes in firing rates without compromising the ability to discriminate between textures. Nevertheless, these changes resulted in a reduction in texture neuronal response stability. Stimulating multiple whiskers led to improved neuronal texture discrimination and maintained coding stability. These findings emphasize the importance of considering numerous factors and their interactions when studying the impact of stimulus configuration on neuronal responses and behavior.

Preliminary Evidence for Global Properties in Human Listeners During Natural Auditory Scene Perception

Theories of auditory and visual scene analysis suggest the perception of scenes relies on the identification and segregation of objects within it, resembling a detail-oriented processing style. However, a more global process may occur while analyzing scenes, which has been evidenced in the visual domain. It is our understanding that a similar line of research has not been explored in the auditory domain; therefore, we evaluated the contributions of high-level global and low-level acoustic information to auditory scene perception. An additional aim was to increase the field's ecological validity by using and making available a new collection of high-quality auditory scenes. Participants rated scenes on 8 global properties (e.g., open vs. enclosed) and an acoustic analysis evaluated which low-level features predicted the ratings. We submitted the acoustic measures and average ratings of the global properties to separate exploratory factor analyses (EFAs). The EFA of the acoustic measures revealed a seven-factor structure explaining 57% of the variance in the data, while the EFA of the global property measures revealed a two-factor structure explaining 64% of the variance in the data. Regression analyses revealed each global property was predicted by at least one acoustic variable (R2 = 0.33-0.87). These findings were extended using deep neural network models where we examined correlations between human ratings of global properties and deep embeddings of two computational models: an object-based model and a scene-based model. The results support that participants' ratings are more strongly explained by a global analysis of the scene setting, though the relationship between scene perception and auditory perception is multifaceted, with differing correlation patterns evident between the two models. Taken together, our results provide evidence for the ability to perceive auditory scenes from a global perspective. Some of the acoustic measures predicted ratings of global scene perception, suggesting representations of auditory objects may be transformed through many stages of processing in the ventral auditory stream, similar to what has been proposed in the ventral visual stream. These findings and the open availability of our scene collection will make future studies on perception, attention, and memory for natural auditory scenes possible.

Representational maps in the brain: concepts, approaches, and applications

Neural systems have evolved to process sensory stimuli in a way that allows for efficient and adaptive behavior in a complex environment. Recent technological advances enable us to investigate sensory processing in animal models by simultaneously recording the activity of large populations of neurons with single-cell resolution, yielding high-dimensional datasets. In this review, we discuss concepts and approaches for assessing the population-level representation of sensory stimuli in the form of a representational map. In such a map, not only are the identities of stimuli distinctly represented, but their relational similarity is also mapped onto the space of neuronal activity. We highlight example studies in which the structure of representational maps in the brain are estimated from recordings in humans as well as animals and compare their methodological approaches. Finally, we integrate these aspects and provide an outlook for how the concept of representational maps could be applied to various fields in basic and clinical neuroscience.

Evolutionary continuity and divergence of auditory dorsal and ventral pathways in primates revealed by ultra-high field diffusion MRI

Auditory dorsal and ventral pathways in the human brain play important roles in supporting speech and language processing. However, the evolutionary root of the dual auditory pathways in the primate brain is unclear. By parcellating the auditory cortex of marmosets (a New World monkey species), macaques (an Old World monkey species), and humans using the same individual-based analysis method and tracking the pathways from the auditory cortex based on multi-shell diffusion-weighted MRI (dMRI), homologous auditory dorsal and ventral fiber tracks were identified in these primate species. The ventral pathway was found to be well conserved in all three primate species analyzed but extend to more anterior temporal regions in humans. In contrast, the dorsal pathway showed a divergence between monkey and human brains. First, frontal regions in the human brain have stronger connections to the higher-level auditory regions than to the lower-level auditory regions along the dorsal pathway, while frontal regions in the monkey brain show opposite connection patterns along the dorsal pathway. Second, the left lateralization of the dorsal pathway is only found in humans. Moreover, the connectivity strength of the dorsal pathway in marmosets is more similar to that of humans than macaques. These results demonstrate the continuity and divergence of the dual auditory pathways in the primate brains along the evolutionary path, suggesting that the putative neural networks supporting human speech and language processing might have emerged early in primate evolution.

Encoding of melody in the human auditory cortex

Melody is a core component of music in which discrete pitches are serially arranged to convey emotion and meaning. Perception varies along several pitch-based dimensions: (i) the absolute pitch of notes, (ii) the difference in pitch between successive notes, and (iii) the statistical expectation of each note given prior context. How the brain represents these dimensions and whether their encoding is specialized for music remains unknown. We recorded high-density neurophysiological activity directly from the human auditory cortex while participants listened to Western musical phrases. Pitch, pitch-change, and expectation were selectively encoded at different cortical sites, indicating a spatial map for representing distinct melodic dimensions. The same participants listened to spoken English, and we compared responses to music and speech. Cortical sites selective for music encoded expectation, while sites that encoded pitch and pitch-change in music used the same neural code to represent equivalent properties of speech. Findings reveal how the perception of melody recruits both music-specific and general-purpose sound representations.

Spontaneous emergence of rudimentary music detectors in deep neural networks

Music exists in almost every society, has universal acoustic features, and is processed by distinct neural circuits in humans even with no experience of musical training. However, it remains unclear how these innate characteristics emerge and what functions they serve. Here, using an artificial deep neural network that models the auditory information processing of the brain, we show that units tuned to music can spontaneously emerge by learning natural sound detection, even without learning music. The music-selective units encoded the temporal structure of music in multiple timescales, following the population-level response characteristics observed in the brain. We found that the process of generalization is critical for the emergence of music-selectivity and that music-selectivity can work as a functional basis for the generalization of natural sound, thereby elucidating its origin. These findings suggest that evolutionary adaptation to process natural sounds can provide an initial blueprint for our sense of music.

Benefits of Harmonicity for Hearing in Noise Are Limited to Detection and Pitch-Related Discrimination Tasks

Harmonic complex tones are easier to detect in noise than inharmonic complex tones, providing a potential perceptual advantage in complex auditory environments. Here, we explored whether the harmonic advantage extends to other auditory tasks that are important for navigating a noisy auditory environment, such as amplitude- and frequency-modulation detection. Sixty young normal-hearing listeners were tested, divided into two equal groups with and without musical training. Consistent with earlier studies, harmonic tones were easier to detect in noise than inharmonic tones, with a signal-to-noise ratio (SNR) advantage of about 2.5 dB, and the pitch discrimination of the harmonic tones was more accurate than that of inharmonic tones, even after differences in audibility were accounted for. In contrast, neither amplitude- nor frequency-modulation detection was superior with harmonic tones once differences in audibility were accounted for. Musical training was associated with better performance only in pitch-discrimination and frequency-modulation-detection tasks. The results confirm a detection and pitch-perception advantage for harmonic tones but reveal that the harmonic benefits do not extend to suprathreshold tasks that do not rely on extracting the fundamental frequency. A general theory is proposed that may account for the effects of both noise and memory on pitch-discrimination differences between harmonic and inharmonic tones.

Encoding of melody in the human auditory cortex

Melody is a core component of music in which discrete pitches are serially arranged to convey emotion and meaning. Perception of melody varies along several pitch-based dimensions: (1) the absolute pitch of notes, (2) the difference in pitch between successive notes, and (3) the higher-order statistical expectation of each note conditioned on its prior context. While humans readily perceive melody, how these dimensions are collectively represented in the brain and whether their encoding is specialized for music remains unknown. Here, we recorded high-density neurophysiological activity directly from the surface of human auditory cortex while Western participants listened to Western musical phrases. Pitch, pitch-change, and expectation were selectively encoded at different cortical sites, indicating a spatial code for representing distinct dimensions of melody. The same participants listened to spoken English, and we compared evoked responses to music and speech. Cortical sites selective for music were systematically driven by the encoding of expectation. In contrast, sites that encoded pitch and pitch-change used the same neural code to represent equivalent properties of speech. These findings reveal the multidimensional nature of melody encoding, consisting of both music-specific and domain-general sound representations in auditory cortex.

Teaser

The human brain contains both general-purpose and music-specific neural populations for processing distinct attributes of melody.

Segregation and integration of sensory features by flexible temporal characteristics of independent neural representations

Segregation and integration are two fundamental yet competing computations in cognition. For example, in serial speech processing, stable perception necessitates the sequential establishment of perceptual representations to remove irrelevant features for achieving invariance. Whereas multiple features need to combine to create a coherent percept. How to simultaneously achieve seemingly contradicted computations of segregation and integration in a serial process is unclear. To investigate their neural mechanisms, we used loudness and lexical tones as a research model and employed a novel multilevel oddball paradigm with Electroencephalogram (EEG) recordings to explore the dynamics of mismatch negativity (MMN) responses to their deviants. When two types of deviants were presented separately, distinct topographies of MMNs to loudness and tones were observed at different latencies (loudness earlier), supporting the sequential dynamics of independent representations for two features. When they changed simultaneously, the latency of responses to tones became shorter and aligned with that to loudness, while the topographies remained independent, yielding the combined MMN as a linear additive of single MMNs of loudness and tones. These results suggest that neural dynamics can be temporally synchronized to distinct sensory features and balance the computational demands of segregation and integration, grounding for invariance and feature binding in serial processing.

Cortical representation of musical pitch in event-related potentials

Neural coding of auditory stimulus frequency is well-documented; however, the cortical signals and perceptual correlates of pitch have not yet been comprehensively investigated. This study examined the temporal patterns of event-related potentials (ERP) in response to single tones of pitch chroma, with an assumption that these patterns would be more prominent in musically-trained individuals than in non-musically-trained individuals. Participants with and without musical training (N = 20) were presented with seven notes on the C major scale (C4, D4, E4, F4, G4, A4, and B4), and whole-brain activities were recorded. A linear regression analysis between the ERP amplitude and the seven notes showed that the ERP amplitude increased or decreased as the frequency of the pitch increased. Remarkably, these linear correlations were anti-symmetric between the hemispheres. Specifically, we found that ERP amplitudes of the left and right frontotemporal areas decreased and increased, respectively, as the pitch frequency increased. Although linear slopes were significant in both groups, the musically-trained group exhibited marginally steeper slope, and their ERP amplitudes were most discriminant for frequency of tone of pitch at earlier latency than in the non-musically-trained group (~ 460 ms vs ~ 630 ms after stimulus onset). Thus, the ERP amplitudes in frontotemporal areas varied according to the pitch frequency, with the musically-trained participants demonstrating a wider range of amplitudes and inter-hemispheric anti-symmetric patterns. Our findings may provide new insights on cortical processing of musical pitch, revealing anti-symmetric processing of musical pitch between hemispheres, which appears to be more pronounced in musically-trained people.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13534-023-00274-y.

A small, but vocal, brain

In the May issue of Cell Reports, Jafari et al.1 used ultra-high-field fMRI to show that marmosets, like humans and macaques, possess an extensive network of voice-selective areas.

Distribution of multiunit pitch responses recorded intracranially from human auditory cortex

The perception of pitch is a fundamental percept, which is mediated by the auditory system, requiring the abstraction of stimulus properties related to the spectro-temporal structure of sound. Despite its importance, there is still debate as to the precise areas responsible for its encoding, which may be due to species differences or differences in the recording measures and choices of stimuli used in previous studies. Moreover, it was unknown whether the human brain contains pitch neurons and how distributed such neurons might be. Here, we present the first study to measure multiunit neural activity in response to pitch stimuli in the auditory cortex of intracranially implanted humans. The stimulus sets were regular-interval noise with a pitch strength that is related to the temporal regularity and a pitch value determined by the repetition rate and harmonic complexes. Specifically, we demonstrate reliable responses to these different pitch-inducing paradigms that are distributed throughout Heschl's gyrus, rather than being localized to a particular region, and this finding was evident regardless of the stimulus presented. These data provide a bridge across animal and human studies and aid our understanding of the processing of a critical percept associated with acoustic stimuli.

Sensitivity to Frequency Modulation is Limited Centrally

Modulations in both amplitude and frequency are prevalent in natural sounds and are critical in defining their properties. Humans are exquisitely sensitive to frequency modulation (FM) at the slow modulation rates and low carrier frequencies that are common in speech and music. This enhanced sensitivity to slow-rate and low-frequency FM has been widely believed to reflect precise, stimulus-driven phase locking to temporal fine structure in the auditory nerve. At faster modulation rates and/or higher carrier frequencies, FM is instead thought to be coded by coarser frequency-to-place mapping, where FM is converted to amplitude modulation (AM) via cochlear filtering. Here, we show that patterns of human FM perception that have classically been explained by limits in peripheral temporal coding are instead better accounted for by constraints in the central processing of fundamental frequency (F0) or pitch. We measured FM detection in male and female humans using harmonic complex tones with an F0 within the range of musical pitch but with resolved harmonic components that were all above the putative limits of temporal phase locking (>8 kHz). Listeners were more sensitive to slow than fast FM rates, even though all components were beyond the limits of phase locking. In contrast, AM sensitivity remained better at faster than slower rates, regardless of carrier frequency. These findings demonstrate that classic trends in human FM sensitivity, previously attributed to auditory nerve phase locking, may instead reflect the constraints of a unitary code that operates at a more central level of processing.SIGNIFICANCE STATEMENT Natural sounds involve dynamic frequency and amplitude fluctuations. Humans are particularly sensitive to frequency modulation (FM) at slow rates and low carrier frequencies, which are prevalent in speech and music. This sensitivity has been ascribed to encoding of stimulus temporal fine structure (TFS) via phase-locked auditory nerve activity. To test this long-standing theory, we measured FM sensitivity using complex tones with a low F0 but only high-frequency harmonics beyond the limits of phase locking. Dissociating the F0 from TFS showed that FM sensitivity is limited not by peripheral encoding of TFS but rather by central processing of F0, or pitch. The results suggest a unitary code for FM detection limited by more central constraints.

Neuroimaging evidence for the direct role of auditory scene analysis in object perception

Auditory Scene Analysis (ASA) refers to the grouping of acoustic signals into auditory objects. Previously, we have shown that perceived musicality of auditory sequences varies with high-level organizational features. Here, we explore the neural mechanisms mediating ASA and auditory object perception. Participants performed musicality judgments on randomly generated pure-tone sequences and manipulated versions of each sequence containing low-level changes (amplitude; timbre). Low-level manipulations affected auditory object perception as evidenced by changes in musicality ratings. fMRI was used to measure neural activation to sequences rated most and least musical, and the altered versions of each sequence. Next, we generated two partially overlapping networks: (i) a music processing network (music localizer) and (ii) an ASA network (base sequences vs. ASA manipulated sequences). Using Representational Similarity Analysis, we correlated the functional profiles of each ROI to a model generated from behavioral musicality ratings as well as models corresponding to low-level feature processing and music perception. Within overlapping regions, areas near primary auditory cortex correlated with low-level ASA models, whereas right IPS was correlated with musicality ratings. Shared neural mechanisms that correlate with behavior and underlie both ASA and music perception suggests that low-level features of auditory stimuli play a role in auditory object perception.

Sound localization acuity of the common marmoset (Callithrix jacchus)

The common marmoset (Callithrix jacchus) is a small arboreal New World primate which has emerged as a promising model in auditory neuroscience. One potentially useful application of this model system is in the study of the neural mechanism underlying spatial hearing in primate species, as the marmosets need to localize sounds to orient their head to events of interest and identify their vocalizing conspecifics that are not visible. However, interpretation of neurophysiological data on sound localization requires an understanding of perceptual abilities, and the sound localization behavior of marmosets has not been well studied. The present experiment measured sound localization acuity using an operant conditioning procedure in which marmosets were trained to discriminate changes in sound location in the horizontal (azimuth) or vertical (elevation) dimension. Our results showed that the minimum audible angle (MAA) for horizontal and vertical discrimination was 13.17° and 12.53°, respectively, for 2 to 32 kHz Gaussian noise. Removing the monaural spectral cues tended to increase the horizontal localization acuity (11.31°). Marmosets have larger horizontal MAA (15.54°) in the rear than the front. Removing the high-frequency (> 26 kHz) region of the head-related transfer function (HRTF) affected vertical acuity mildly (15.76°), but removing the first notch (12-26 kHz) region of HRTF substantially reduced the vertical acuity (89.01°). In summary, our findings indicate that marmosets' spatial acuity is on par with other species of similar head size and field of best vision, and they do not appear to use monaural spectral cues for horizontal discrimination but rely heavily on first notch region of HRTF for vertical discrimination.

Relative pitch representations and invariance to timbre

Information in speech and music is often conveyed through changes in fundamental frequency (f0), perceived by humans as "relative pitch". Relative pitch judgments are complicated by two facts. First, sounds can simultaneously vary in timbre due to filtering imposed by a vocal tract or instrument body. Second, relative pitch can be extracted in two ways: by measuring changes in constituent frequency components from one sound to another, or by estimating the f0 of each sound and comparing the estimates. We examined the effects of timbral differences on relative pitch judgments, and whether any invariance to timbre depends on whether judgments are based on constituent frequencies or their f0. Listeners performed up/down and interval discrimination tasks with pairs of spoken vowels, instrument notes, or synthetic tones, synthesized to be either harmonic or inharmonic. Inharmonic sounds lack a well-defined f0, such that relative pitch must be extracted from changes in individual frequencies. Pitch judgments were less accurate when vowels/instruments were different compared to when they were the same, and were biased by the associated timbre differences. However, this bias was similar for harmonic and inharmonic sounds, and was observed even in conditions where judgments of harmonic sounds were based on f0 representations. Relative pitch judgments are thus not invariant to timbre, even when timbral variation is naturalistic, and when such judgments are based on representations of f0.

Dog and human neural sensitivity to voicelikeness: A comparative fMRI study

Voice-sensitivity in the auditory cortex of a range of mammals has been proposed to be determined primarily by tuning to conspecific auditory stimuli, but recent human findings indicate a role for a more general tuning to voicelikeness. Vocal emotional valence, a central characteristic of vocalisations, has been linked to the same basic acoustic parameters across species. Comparative neuroimaging revealed that during voice perception, such acoustic parameters modulate emotional valence-sensitivity in auditory cortical regions in both family dogs and humans. To explore the role of voicelikeness in auditory emotional valence-sensitivity across species, here we constructed artificial emotional sounds in two sound categories: voice-like vs. sine-wave sounds, parametrically modulating two main acoustic parameters, f0 and call length. We hypothesised that if mammalian auditory systems are characterised by a general tuning to voicelikeness, voice-like sounds will be processed preferentially, and acoustic parameters for voice-like sounds will be processed differently than for sine-wave sounds - both in dogs and humans. We found cortical areas in both species that responded stronger to voice-like than to sine-wave stimuli, while there were no regions responding stronger to sine-wave sounds in either species. Additionally, we found that in bilateral primary and emotional valence-sensitive auditory regions of both species, the processing of voice-like and sine-wave sounds are modulated by f0 in opposite ways. These results reveal functional similarities between evolutionarily distant mammals for processing voicelikeness and its effect on processing basic acoustic cues of vocal emotions.

A radiofrequency coil to facilitate task-based fMRI of awake marmosets

BACKGROUND

The small common marmoset (Callithrix jacchus) is an ideal nonhuman primate for awake fMRI in ultra-high field small animal MRI scanners. However, it can often be challenging in task-based fMRI experiments to provide a robust stimulus within the MRI environment while using hardware (an RF coil and restraint system) that is compatible with awake imaging.

NEW METHOD

Here we present an RF coil and restraint system that permits unimpeded access to an awake marmoset's head subsequent to immobilization, thereby permitting the setup of peripheral devices and stimuli proximal to the head.

RESULTS

As an example application, an fMRI experiment probing whole-brain activation in response to marmoset vocalizations was conducted-this paradigm showed significant bilateral activation in the inferior colliculus, medial lateral geniculate nucleus, and auditory cortex.

COMPARISON WITH EXISTING METHOD(S)

The coil performance was evaluated and compared to a previously published restraint system with integrated RF coil. The image and temporal SNR were improved by up to 58 % and 27 %, respectively, in the peripheral cortex and by 30 % and 3 % in the centre of the brain. The restraint-system topology limited head motion to less than 100 µm of translation and 0.30° of rotation when measured over a 15-minute acquisition.

CONCLUSIONS

The proposed hardware solution provides a versatile approach to awake-marmoset imaging and, as demonstrated, can facilitate task-based fMRI.

Consonance and dissonance perception. A critical review of the historical sources, multidisciplinary findings, and main hypotheses

Revealed more than two millennia ago by Pythagoras, consonance and dissonance (C/D) are foundational concepts in music theory, perception, and aesthetics. The search for the biological, acoustical, and cultural factors that affect C/D perception has resulted in descriptive accounts inspired by arithmetic, musicological, psychoacoustical or neurobiological frameworks without reaching a consensus. Here, we review the key historical sources and modern multidisciplinary findings on C/D and integrate them into three main hypotheses: the vocal similarity hypothesis (VSH), the psychocultural hypothesis (PH), and the sensorimotor hypothesis (SH). By illustrating the hypotheses-related findings, we highlight their major conceptual, methodological, and terminological shortcomings. Trying to provide a unitary framework for C/D understanding, we put together multidisciplinary research on human and animal vocalizations, which converges to suggest that auditory roughness is associated with distress/danger and, therefore, elicits defensive behavioral reactions and neural responses that indicate aversion. We therefore stress the primacy of vocality and roughness as key factors in the explanation of C/D phenomenon, and we explore the (neuro)biological underpinnings of the attraction-aversion mechanisms that are triggered by C/D stimuli. Based on the reviewed evidence, while the aversive nature of dissonance appears as solidly rooted in the multidisciplinary findings, the attractive nature of consonance remains a somewhat speculative claim that needs further investigation. Finally, we outline future directions for empirical research in C/D, especially regarding cross-modal and cross-cultural approaches.

Cortical tracking of voice pitch in the presence of multiple speakers depends on selective attention

Voice pitch carries linguistic and non-linguistic information. Previous studies have described cortical tracking of voice pitch in clean speech, with responses reflecting both pitch strength and pitch value. However, pitch is also a powerful cue for auditory stream segregation, especially when competing streams have pitch differing in fundamental frequency, as is the case when multiple speakers talk simultaneously. We therefore investigated how cortical speech pitch tracking is affected in the presence of a second, task-irrelevant speaker. We analyzed human magnetoencephalography (MEG) responses to continuous narrative speech, presented either as a single talker in a quiet background or as a two-talker mixture of a male and a female speaker. In clean speech, voice pitch was associated with a right-dominant response, peaking at a latency of around 100 ms, consistent with previous electroencephalography and electrocorticography results. The response tracked both the presence of pitch and the relative value of the speaker’s fundamental frequency. In the two-talker mixture, the pitch of the attended speaker was tracked bilaterally, regardless of whether or not there was simultaneously present pitch in the speech of the irrelevant speaker. Pitch tracking for the irrelevant speaker was reduced: only the right hemisphere still significantly tracked pitch of the unattended speaker, and only during intervals in which no pitch was present in the attended talker’s speech. Taken together, these results suggest that pitch-based segregation of multiple speakers, at least as measured by macroscopic cortical tracking, is not entirely automatic but strongly dependent on selective attention.

Role of perceptual integration in pitch discrimination at high frequenciesa)

At very high frequencies, fundamental-frequency difference limens (F0DLs) for five-component harmonic complex tones can be better than predicted by optimal integration of information, assuming performance is limited by noise at the peripheral level, but are in line with predictions based on more central sources of noise. This study investigates whether there is a minimum number of harmonic components needed for such super-optimal integration effects and if harmonic range or inharmonicity affects this super-optimal integration. Results show super-optimal integration, even with two harmonic components and for most combinations of consecutive harmonic, but not inharmonic, components.

A Music Cognition–Guided Framework for Multi-pitch Estimation

As one of the most important subtasks of automatic music transcription (AMT), multi-pitch estimation (MPE) has been studied extensively for predicting the fundamental frequencies in the frames of audio recordings during the past decade. However, how to use music perception and cognition for MPE has not yet been thoroughly investigated. Motivated by this, this demonstrates how to effectively detect the fundamental frequency and the harmonic structure of polyphonic music using a cognitive framework. Inspired by cognitive neuroscience, an integration of the constant Q transform and a state-of-the-art matrix factorization method called shift-invariant probabilistic latent component analysis (SI-PLCA) are proposed to resolve the polyphonic short-time magnitude log-spectra for multiple pitch estimation and source-specific feature extraction. The cognitions of rhythm, harmonic periodicity and instrument timbre are used to guide the analysis of characterizing contiguous notes and the relationship between fundamental frequency and harmonic frequencies for detecting the pitches from the outcomes of SI-PLCA. In the experiment, we compare the performance of proposed MPE system to a number of existing state-of-the-art approaches (seven weak learning methods and four deep learning methods) on three widely used datasets (i.e. MAPS, BACH10 and TRIOS) in terms of F-measure ($${F}_{1}$$ F 1 ) values. The experimental results show that the proposed MPE method provides the best overall performance against other existing methods.

The Effects of Attention on the Syllable-Induced Prepulse Inhibition of the Startle Reflex and Cortical EEG Responses against Energetic or Informational Masking in Humans

Prepulse inhibition (PPI) is the reduction in the acoustic startle reflex (ASR) when the startling stimulus (pulse) is preceded by a weaker, non-starting stimulus. This can be enhanced by facilitating selective attention to the prepulse against a noise-masking background. On the other hand, the facilitation of selective attention to a target speech can release the target speech from masking, particularly from speech informational masking. It is not clear whether attentional regulation also affects PPI in this kind of auditory masking. This study used a speech syllable as the prepulse to examine whether the masker type and perceptual spatial attention can affect the PPI or the scalp EEG responses to the prepulse in healthy younger-adult humans, and whether the ERPs evoked by the prepulse can predict the PPI intensity of the ASR. The results showed that the speech masker produced a larger masking effect than the noise masker, and the perceptual spatial separation facilitated selective attention to the prepulse, enhancing both the N1 component of the prepulse syllable and the PPI of the ASR, particularly when the masker was speech. In addition, there was no significant correlation between the PPI and ERPs under any of the conditions, but the perceptual separation-induced PPI enhancement and ERP N1P2 peak-to-peak amplitude enhancement were correlated under the speech-masking condition. Thus, the attention-mediated PPI is useful for differentiating noise energetic masking and speech informational masking, and the perceptual separation-induced release of the prepulse from informational masking is more associated with attention-mediated early cortical unmasking processing than with energetic masking. However, the processes for the PPI of the ASR and the cortical responses to the prepulse are mediated by different neural mechanisms.

Signatures of cochlear processing in neuronal coding of auditory information

The vertebrate ear is endowed with remarkable perceptual capabilities. The faintest sounds produce vibrations of magnitudes comparable to those generated by thermal noise and can nonetheless be detected through efficient amplification of small acoustic stimuli. Two mechanisms have been proposed to underlie such sound amplification in the mammalian cochlea: somatic electromotility and active hair-bundle motility. These biomechanical mechanisms may work in concert to tune auditory sensitivity. In addition to amplitude sensitivity, the hearing system shows exceptional frequency discrimination allowing mammals to distinguish complex sounds with great accuracy. For instance, although the wide hearing range of humans encompasses frequencies from 20 Hz to 20 kHz, our frequency resolution extends to one-thirtieth of the interval between successive keys on a piano. In this article, we review the different cochlear mechanisms underlying sound encoding in the auditory system, with a particular focus on the frequency decomposition of sounds. The relation between peak frequency of activation and location along the cochlea - known as tonotopy - arises from multiple gradients in biophysical properties of the sensory epithelium. Tonotopic mapping represents a major organizational principle both in the peripheral hearing system and in higher processing levels and permits the spectral decomposition of complex tones. The ribbon synapses connecting sensory hair cells to auditory afferents and the downstream spiral ganglion neurons are also tuned to process periodic stimuli according to their preferred frequency. Though sensory hair cells and neurons necessarily filter signals beyond a few kHz, many animals can hear well beyond this range. We finally describe how the cochlear structure shapes the neural code for further processing in order to send meaningful information to the brain. Both the phase-locked response of auditory nerve fibers and tonotopy are key to decode sound frequency information and place specific constraints on the downstream neuronal network.

Music in the brain

Music is ubiquitous across human cultures - as a source of affective and pleasurable experience, moving us both physically and emotionally - and learning to play music shapes both brain structure and brain function. Music processing in the brain - namely, the perception of melody, harmony and rhythm - has traditionally been studied as an auditory phenomenon using passive listening paradigms. However, when listening to music, we actively generate predictions about what is likely to happen next. This enactive aspect has led to a more comprehensive understanding of music processing involving brain structures implicated in action, emotion and learning. Here we review the cognitive neuroscience literature of music perception. We show that music perception, action, emotion and learning all rest on the human brain's fundamental capacity for prediction - as formulated by the predictive coding of music model. This Review elucidates how this formulation of music perception and expertise in individuals can be extended to account for the dynamics and underlying brain mechanisms of collective music making. This in turn has important implications for human creativity as evinced by music improvisation. These recent advances shed new light on what makes music meaningful from a neuroscientific perspective.

Human discrimination and modeling of high-frequency complex tones shed light on the neural codes for pitch

Accurate pitch perception of harmonic complex tones is widely believed to rely on temporal fine structure information conveyed by the precise phase-locked responses of auditory-nerve fibers. However, accurate pitch perception remains possible even when spectrally resolved harmonics are presented at frequencies beyond the putative limits of neural phase locking, and it is unclear whether residual temporal information, or a coarser rate-place code, underlies this ability. We addressed this question by measuring human pitch discrimination at low and high frequencies for harmonic complex tones, presented either in isolation or in the presence of concurrent complex-tone maskers. We found that concurrent complex-tone maskers impaired performance at both low and high frequencies, although the impairment introduced by adding maskers at high frequencies relative to low frequencies differed between the tested masker types. We then combined simulated auditory-nerve responses to our stimuli with ideal-observer analysis to quantify the extent to which performance was limited by peripheral factors. We found that the worsening of both frequency discrimination and F0 discrimination at high frequencies could be well accounted for (in relative terms) by optimal decoding of all available information at the level of the auditory nerve. A Python package is provided to reproduce these results, and to simulate responses to acoustic stimuli from the three previously published models of the human auditory nerve used in our analyses.

Constructing the hierarchy of predictive auditory sequences in the marmoset brain

Our brains constantly generate predictions of sensory input that are compared with actual inputs, propagate the prediction-errors through a hierarchy of brain regions, and subsequently update the internal predictions of the world. However, the essential feature of predictive coding, the notion of hierarchical depth and its neural mechanisms, remains largely unexplored. Here, we investigated the hierarchical depth of predictive auditory processing by combining functional magnetic resonance imaging (fMRI) and high-density whole-brain electrocorticography (ECoG) in marmoset monkeys during an auditory local-global paradigm in which the temporal regularities of the stimuli were designed at two hierarchical levels. The prediction-errors and prediction updates were examined as neural responses to auditory mismatches and omissions. Using fMRI, we identified a hierarchical gradient along the auditory pathway: midbrain and sensory regions represented local, shorter-time-scale predictive processing followed by associative auditory regions, whereas anterior temporal and prefrontal areas represented global, longer-time-scale sequence processing. The complementary ECoG recordings confirmed the activations at cortical surface areas and further differentiated the signals of prediction-error and update, which were transmitted via putative bottom-up γ and top-down β oscillations, respectively. Furthermore, omission responses caused by absence of input, reflecting solely the two levels of prediction signals that are unique to the hierarchical predictive coding framework, demonstrated the hierarchical top-down process of predictions in the auditory, temporal, and prefrontal areas. Thus, our findings support the hierarchical predictive coding framework, and outline how neural networks and spatiotemporal dynamics are used to represent and arrange a hierarchical structure of auditory sequences in the marmoset brain.

An open-source tool for automated analysis of breathing behaviors in common marmosets and rodents

The respiratory system maintains homeostatic levels of oxygen (O2) and carbon dioxide (CO2) in the body through rapid and efficient regulation of breathing frequency and depth (tidal volume). The commonly used methods of analyzing breathing data in behaving experimental animals are usually subjective, laborious, and time-consuming. To overcome these hurdles, we optimized an analysis toolkit for the unsupervised study of respiratory activities in animal subjects. Using this tool, we analyzed breathing behaviors of the common marmoset (Callithrix jacchus), a New World non-human primate model. Using whole-body plethysmography in room air as well as acute hypoxic (10% O2) and hypercapnic (6% CO2) conditions, we describe breathing behaviors in awake, freely behaving marmosets. Our data indicate that marmosets' exposure to acute hypoxia decreased metabolic rate and increased sigh rate. However, the hypoxic condition did not augment ventilation. Hypercapnia, on the other hand, increased both the frequency and depth (i.e., tidal volume) of breathing.

MEG correlates of temporal regularity relevant to pitch perception in human auditory cortex

We recorded neural responses in human participants to three types of pitch-evoking regular stimuli at rates below and above the lower limit of pitch using magnetoencephalography (MEG). These bandpass filtered (1–4 kHz) stimuli were harmonic complex tones (HC), click trains (CT), and regular interval noise (RIN). Trials consisted of noise-regular-noise (NRN) or regular-noise-regular (RNR) segments in which the repetition rate (or fundamental frequency F0) was either above (250 Hz) or below (20 Hz) the lower limit of pitch. Neural activation was estimated and compared at the senor and source levels.

The pitch-relevant regular stimuli (F0 = 250 Hz) were all associated with marked evoked responses at around 140 ms after noise-to-regular transitions at both sensor and source levels. In particular, greater evoked responses to pitch-relevant stimuli than pitch-irrelevant stimuli (F0 = 20 Hz) were localized along the Heschl's sulcus around 140 ms. The regularity-onset responses for RIN were much weaker than for the other types of regular stimuli (HC, CT). This effect was localized over planum temporale, planum polare, and lateral Heschl's gyrus. Importantly, the effect of pitch did not interact with the stimulus type. That is, we did not find evidence to support different responses for different types of regular stimuli from the spatiotemporal cluster of the pitch effect (∼140 ms).

The current data demonstrate cortical sensitivity to temporal regularity relevant to pitch that is consistently present across different pitch-relevant stimuli in the Heschl's sulcus between Heschl's gyrus and planum temporale, both of which have been identified as a “pitch center” based on different modalities.

Effects of Cortical Cooling on Sound Processing in Auditory Cortex and Thalamus of Awake Marmosets

The auditory thalamus is the central nexus of bottom-up connections from the inferior colliculus and top-down connections from auditory cortical areas. While considerable efforts have been made to investigate feedforward processing of sounds in the auditory thalamus (medial geniculate body, MGB) of non-human primates, little is known about the role of corticofugal feedback in the MGB of awake non-human primates. Therefore, we developed a small, repositionable cooling probe to manipulate corticofugal feedback and studied neural responses in both auditory cortex and thalamus to sounds under conditions of normal and reduced cortical temperature. Cooling-induced increases in the width of extracellularly recorded spikes in auditory cortex were observed over the distance of several hundred micrometers away from the cooling probe. Cortical neurons displayed reduction in both spontaneous and stimulus driven firing rates with decreased cortical temperatures. In thalamus, cortical cooling led to increased spontaneous firing and either increased or decreased stimulus driven activity. Furthermore, response tuning to modulation frequencies of temporally modulated sounds and spatial tuning to sound source location could be altered (increased or decreased) by cortical cooling. Specifically, best modulation frequencies of individual MGB neurons could shift either toward higher or lower frequencies based on the vector strength or the firing rate. The tuning of MGB neurons for spatial location could both sharpen or widen. Elevation preference could shift toward higher or lower elevations and azimuth tuning could move toward ipsilateral or contralateral locations. Such bidirectional changes were observed in many parameters which suggests that the auditory thalamus acts as a filter that could be adjusted according to behaviorally driven signals from auditory cortex. Future work will have to delineate the circuit elements responsible for the observed effects.

Rabbits use both spectral and temporal cues to discriminate the fundamental frequency of harmonic complexes with missing fundamentals

The pitch of harmonic complex tones (HCTs) common in speech, music, and animal vocalizations plays a key role in the perceptual organization of sound. Unraveling the neural mechanisms of pitch perception requires animal models, but little is known about complex pitch perception by animals, and some species appear to use different pitch mechanisms than humans. Here, we tested rabbits' ability to discriminate the fundamental frequency (F0) of HCTs with missing fundamentals, using a behavioral paradigm inspired by foraging behavior in which rabbits learned to harness a spatial gradient in F0 to find the location of a virtual target within a room for a food reward. Rabbits were initially trained to discriminate HCTs with F0s in the range 400-800 Hz and with harmonics covering a wide frequency range (800-16,000 Hz) and then tested with stimuli differing in spectral composition to test the role of harmonic resolvability (experiment 1) or in F0 range (experiment 2) or in both F0 and spectral content (experiment 3). Together, these experiments show that rabbits can discriminate HCTs over a wide F0 range (200-1,600 Hz) encompassing the range of conspecific vocalizations and can use either the spectral pattern of harmonics resolved by the cochlea for higher F0s or temporal envelope cues resulting from interaction between unresolved harmonics for lower F0s. The qualitative similarity of these results to human performance supports the use of rabbits as an animal model for studies of pitch mechanisms, providing species differences in cochlear frequency selectivity and F0 range of vocalizations are taken into account.NEW & NOTEWORTHY Understanding the neural mechanisms of pitch perception requires experiments in animal models, but little is known about pitch perception by animals. Here we show that rabbits, a popular animal in auditory neuroscience, can discriminate complex sounds differing in pitch using either spectral cues or temporal cues. The results suggest that the role of spectral cues in pitch perception by animals may have been underestimated by predominantly testing low frequencies in the range of human voice.

Deep neural network models reveal interplay of peripheral coding and stimulus statistics in pitch perception

Perception is thought to be shaped by the environments for which organisms are optimized. These influences are difficult to test in biological organisms but may be revealed by machine perceptual systems optimized under different conditions. We investigated environmental and physiological influences on pitch perception, whose properties are commonly linked to peripheral neural coding limits. We first trained artificial neural networks to estimate fundamental frequency from biologically faithful cochlear representations of natural sounds. The best-performing networks replicated many characteristics of human pitch judgments. To probe the origins of these characteristics, we then optimized networks given altered cochleae or sound statistics. Human-like behavior emerged only when cochleae had high temporal fidelity and when models were optimized for naturalistic sounds. The results suggest pitch perception is critically shaped by the constraints of natural environments in addition to those of the cochlea, illustrating the use of artificial neural networks to reveal underpinnings of behavior.

Distinct neuron populations for simple and compound calls in the primary auditory cortex of awake marmosets

Marmosets are highly social non-human primates that live in families. They exhibit rich vocalization, but the neural basis underlying this complex vocal communication is largely unknown. Here we report the existence of specific neuron populations in marmoset A1 that respond selectively to distinct simple or compound calls made by conspecific marmosets. These neurons were spatially dispersed within A1 but distinct from those responsive to pure tones. Call-selective responses were markedly diminished when individual domains of the call were deleted or the domain sequence was altered, indicating the importance of the global rather than local spectral-temporal properties of the sound. Compound call-selective responses also disappeared when the sequence of the two simple-call components was reversed or their interval was extended beyond 1 s. Light anesthesia largely abolished call-selective responses. Our findings demonstrate extensive inhibitory and facilitatory interactions among call-evoked responses, and provide the basis for further study of circuit mechanisms underlying vocal communication in awake non-human primates.

Longitudinal auditory pathophysiology following mild blast-induced trauma

Blast-induced hearing difficulties affect thousands of veterans and civilians. The long-term impact of even a mild blast exposure on the central auditory system is hypothesized to contribute to lasting behavioral complaints associated with mild blast traumatic brain injury (bTBI). Although recovery from mild blast has been studied separately over brief or long time windows, few, if any, studies have investigated recovery longitudinally over short-term and longer-term (months) time windows. Specifically, many peripheral measures of auditory function either recover or exhibit subclinical deficits, masking deficits in processing complex, real-world stimuli that may recover differently. Thus, examining the acute time course and pattern of neurophysiological impairment using appropriate stimuli is critical to better understanding and intervening in bTBI-induced auditory system impairments. Here, we compared auditory brainstem response, middle-latency auditory-evoked potentials, and envelope following responses. Stimuli were clicks, tone pips, amplitude-modulated tones in quiet and in noise, and speech-like stimuli (iterated rippled noise pitch contours) in adult male rats subjected to mild blast and sham exposure over the course of 2 mo. We found that blast animals demonstrated drastic threshold increases and auditory transmission deficits immediately after blast exposure, followed by substantial recovery during the window of 7-14 days postblast, although with some deficits remaining even after 2 mo. Challenging conditions and speech-like stimuli can better elucidate mild bTBI-induced auditory deficit during this period. Our results suggest multiphasic recovery and therefore potentially different time windows for treatment, and deficits can be best observed using a small battery of sound stimuli.NEW & NOTEWORTHY Few studies on blast-induced hearing deficits go beyond simple sounds and sparsely track postexposure. Therefore, the recovery arc for potential therapies and real-world listening is poorly understood. Evidence suggested multiple recovery phases over 2 mo postexposure. Hearing thresholds largely recovered within 14 days and partially explained recovery. However, midlatency responses, responses to amplitude modulation in noise, and speech-like pitch sweeps exhibited extended changes, implying persistent central auditory deficits and the importance of subclinical threshold shifts.

Cortical hemisphere preference and brainstem ear asymmetry reflect experience-dependent functional modulation of pitch

Temporal attributes of pitch processing at cortical and subcortical levels are differentially weighted and well-coordinated. The question is whether language experience induces functional modulation of hemispheric preference complemented by brainstem ear symmetry for pitch processing. Brainstem frequency-following and cortical pitch responses were recorded concurrently from Mandarin and English participants. A Mandarin syllable with a rising pitch contour was presented to both ears with monaural stimulation. At the cortical level, left ear stimulation in the Chinese group revealed an experience-dependent response for pitch processing in the right hemisphere, consistent with a functionalaccount. The English group revealed a contralateral hemisphere preference consistent with a structuralaccount. At the brainstem level, Chinese participants showed a functional leftward ear asymmetry, whereas English were consistent with a structural account. Overall, language experience modulates both cortical hemispheric preference and brainstem ear asymmetry in a complementary manner to optimize processing of temporal attributes of pitch.

Infant Pitch and Timbre Discrimination in the Presence of Variation in the Other Dimension

Adult listeners perceive pitch with fine precision, with many adults capable of discriminating less than a 1 % change in fundamental frequency (F0). Although there is variability across individuals, this precise pitch perception is an ability ascribed to cortical functions that are also important for speech and music perception. Infants display neural immaturity in the auditory cortex, suggesting that pitch discrimination may improve throughout infancy. In two experiments, we tested the limits of F0 (pitch) and spectral centroid (timbre) perception in 66 infants and 31 adults. Contrary to expectations, we found that infants at both 3 and 7 months were able to reliably detect small changes in F0 in the presence of random variations in spectral content, and vice versa, to the extent that their performance matched that of adults with musical training and exceeded that of adults without musical training. The results indicate high fidelity of F0 and spectral-envelope coding in infants, implying that fully mature cortical processing is not necessary for accurate discrimination of these features. The surprising difference in performance between infants and musically untrained adults may reflect a developmental trajectory for learning natural statistical covariations between pitch and timbre that improves coding efficiency but results in degraded performance in adults without musical training when expectations for such covariations are violated.

Parallel and distributed encoding of speech across human auditory cortex

Speech perception is thought to rely on a cortical feedforward serial transformation of acoustic into linguistic representations. Using intracranial recordings across the entire human auditory cortex, electrocortical stimulation, and surgical ablation, we show that cortical processing across areas is not consistent with a serial hierarchical organization. Instead, response latency and receptive field analyses demonstrate parallel and distinct information processing in the primary and nonprimary auditory cortices. This functional dissociation was also observed where stimulation of the primary auditory cortex evokes auditory hallucination but does not distort or interfere with speech perception. Opposite effects were observed during stimulation of nonprimary cortex in superior temporal gyrus. Ablation of the primary auditory cortex does not affect speech perception. These results establish a distributed functional organization of parallel information processing throughout the human auditory cortex and demonstrate an essential independent role for nonprimary auditory cortex in speech processing.

Causal Relationship between the Right Auditory Cortex and Speech-Evoked Envelope-Following Response: Evidence from Combined Transcranial Stimulation and Electroencephalography

Speech-evoked envelope-following response (EFR) reflects brain encoding of speech periodicity that serves as a biomarker for pitch and speech perception and various auditory and language disorders. Although EFR is thought to originate from the subcortex, recent research illustrated a right-hemispheric cortical contribution to EFR. However, it is unclear whether this contribution is causal. This study aimed to establish this causality by combining transcranial direct current stimulation (tDCS) and measurement of EFR (pre- and post-tDCS) via scalp-recorded electroencephalography. We applied tDCS over the left and right auditory cortices in right-handed normal-hearing participants and examined whether altering cortical excitability via tDCS causes changes in EFR during monaural listening to speech syllables. We showed significant changes in EFR magnitude when tDCS was applied over the right auditory cortex compared with sham stimulation for the listening ear contralateral to the stimulation site. No such effect was found when tDCS was applied over the left auditory cortex. Crucially, we further observed a hemispheric laterality where aftereffect was significantly greater for tDCS applied over the right than the left auditory cortex in the contralateral ear condition. Our finding thus provides the first evidence that validates the causal relationship between the right auditory cortex and EFR.

Lemniscal Corticothalamic Feedback in Auditory Scene Analysis

Sound information is transmitted from the ear to central auditory stations of the brain via several nuclei. In addition to these ascending pathways there exist descending projections that can influence the information processing at each of these nuclei. A major descending pathway in the auditory system is the feedback projection from layer VI of the primary auditory cortex (A1) to the ventral division of medial geniculate body (MGBv) in the thalamus. The corticothalamic axons have small glutamatergic terminals that can modulate thalamic processing and thalamocortical information transmission. Corticothalamic neurons also provide input to GABAergic neurons of the thalamic reticular nucleus (TRN) that receives collaterals from the ascending thalamic axons. The balance of corticothalamic and TRN inputs has been shown to refine frequency tuning, firing patterns, and gating of MGBv neurons. Therefore, the thalamus is not merely a relay stage in the chain of auditory nuclei but does participate in complex aspects of sound processing that include top-down modulations. In this review, we aim (i) to examine how lemniscal corticothalamic feedback modulates responses in MGBv neurons, and (ii) to explore how the feedback contributes to auditory scene analysis, particularly on frequency and harmonic perception. Finally, we will discuss potential implications of the role of corticothalamic feedback in music and speech perception, where precise spectral and temporal processing is essential.

Context-dependent Plasticity and Strength of Subcortical Encoding of Musical Sounds Independently Underlie Pitch Discrimination for Music Melodies

Subcortical auditory nuclei contribute to pitch perception, but how subcortical sound encoding is related to pitch processing for music perception remains unclear. Conventionally, enhanced subcortical sound encoding is considered underlying superior pitch discrimination. However, associations between superior auditory perception and the context-dependent plasticity of subcortical sound encoding are also documented. Here, we explored the subcortical neural correlates to music pitch perception by analyzing frequency-following responses (FFRs) to musical sounds presented in a predictable context and a random context. We found that the FFR inter-trial phase-locking (ITPL) was negatively correlated with behavioral performances of discrimination of pitches in music melodies. It was also negatively correlated with the plasticity indices measuring the variability of FFRs to physically identical sounds between the two contexts. The plasticity indices were consistently positively correlated with pitch discrimination performances, suggesting the subcortical context-dependent plasticity underlying music pitch perception. Moreover, the raw FFR spectral strength was not significantly correlated with pitch discrimination performances. However, it was positively correlated with behavioral performances when the FFR ITPL was controlled by partial correlations, suggesting that the strength of subcortical sound encoding underlies music pitch perception. When the spectral strength was controlled by partial correlations, the negative ITPL-behavioral correlations were maintained. Furthermore, the FFR ITPL, the plasticity indices, and the FFR spectral strength were more correlated with pitch than with rhythm discrimination performances. These findings suggest that the context-dependent plasticity and the strength of subcortical encoding of musical sounds are independently and perhaps specifically associated with pitch perception for music melodies.

Inhibitory gating of coincidence-dependent sensory binding in secondary auditory cortex

Integration of multi-frequency sounds into a unified perceptual object is critical for recognizing syllables in speech. This "feature binding" relies on the precise synchrony of each component's onset timing, but little is known regarding its neural correlates. We find that multi-frequency sounds prevalent in vocalizations, specifically harmonics, preferentially activate the mouse secondary auditory cortex (A2), whose response deteriorates with shifts in component onset timings. The temporal window for harmonics integration in A2 was broadened by inactivation of somatostatin-expressing interneurons (SOM cells), but not parvalbumin-expressing interneurons (PV cells). Importantly, A2 has functionally connected subnetworks of neurons preferentially encoding harmonic over inharmonic sounds. These subnetworks are stable across days and exist prior to experimental harmonics exposure, suggesting their formation during development. Furthermore, A2 inactivation impairs performance in a discrimination task for coincident harmonics. Together, we propose A2 as a locus for multi-frequency integration, which may form the circuit basis for vocal processing.

Electro-Tactile Stimulation Enhances Cochlear-Implant Melody Recognition: Effects of Rhythm and Musical Training

OBJECTIVES

Electro-acoustic stimulation (EAS) enhances speech and music perception in cochlear-implant (CI) users who have residual low-frequency acoustic hearing. For CI users who do not have low-frequency acoustic hearing, tactile stimulation may be used in a similar fashion as residual low-frequency acoustic hearing to enhance CI performance. Previous studies showed that electro-tactile stimulation (ETS) enhanced speech recognition in noise and tonal language perception for CI listeners. Here, we examined the effect of ETS on melody recognition in both musician and nonmusician CI users.

DESIGN

Nine musician and eight nonmusician CI users were tested in a melody recognition task with or without rhythmic cues in three testing conditions: CI only (E), tactile only (T), and combined CI and tactile stimulation (ETS).

RESULTS

Overall, the combined electrical and tactile stimulation enhanced the melody recognition performance in CI users by 9% points. Two additional findings were observed. First, musician CI users outperformed nonmusicians CI users in melody recognition, but the size of the enhancement effect was similar between the two groups. Second, the ETS enhancement was significantly higher with nonrhythmic melodies than rhythmic melodies in both groups.

CONCLUSIONS

These findings suggest that, independent of musical experience, the size of the ETS enhancement depends on integration efficiency between tactile and auditory stimulation, and that the mechanism of the ETS enhancement is improved electric pitch perception. The present study supports the hypothesis that tactile stimulation can be used to improve pitch perception in CI users.

The impact of pitch on tempo-spatial accuracy and precision in intercepting a virtually moving ball

In two experiments, horizontal and vertical orientated sounds moved in parabolas. Participants had to touch a screen to indicate where and when a virtual moving ball would cross a visible line. We predicted that due to the sensitivity of the auditory system to temporal information, manipulations of pitch should affect temporal errors more than spatial errors. Stimuli were sound sources at five different pitches moving along a parabola produced through loudspeakers mounted around a touch screen. Results showed pitch effects on spatial constant and spatial variable errors when the parabola was horizontally oriented (Exp. 1), and on temporal constant errors in vertically oriented parabolas (Exp. 2). We conclude that temporal and spatial precision in interception tasks were affected differently by pitch manipulations and require consideration in future studies when assessing the impact of auditory information on catching virtually moving balls.

Functional neuroimaging in hearing research and audiology

The various methods of medical imaging are essential for many diagnostic issues in clinical routine, e.g., for the diagnostics and localisation of tumorous diseases, or for the clarification of other lesions in the central nervous system. In addition to these classical roles both positron emission tomography (PET) and magnetic resonance imaging (MRI) allow for the investigation of functional processes in the human brain, when used in a specific way. The last 25 years have seen great progress, especially with respect to functional MRI, in terms of the available experimental paradigms as well as the data analysis strategies, so that a directed investigation of neurophysiological correlates of psychoacoustic performance is possible. This covers fundamental measures of sound perception like loudness and pitch, specific audiological symptoms like tinnitus, which often accompanies hearing disorders, but it also includes experiments on speech perception or on virtual acoustic environments. One important aspect common to many auditory neuroimaging studies is the central question at what stage in the human auditory pathway the sensory coding of the incoming sound is transformed into a universal and context-dependent perceptual representation, which is the basis for what we hear. This overview summarises findings from the literature as well as a few studies from our lab, to discuss the possibilities and the limits of the adoption of functional neuroimaging methods in audiology. Up to this stage, most auditory neuroimaging studies have investigated basic processes in normal hearing listeners. However, the hitherto existing results suggest that the methods of auditory functional neuroimaging - possibly complemented by electrophysiological methods like EEG and MEG - have a great potential to contribute to a deeper understanding of the processes and the impact of hearing disorders.

Neurocognitive processing efficiency for discriminating human non-alarm rather than alarm scream calls

Across many species, scream calls signal the affective significance of events to other agents. Scream calls were often thought to be of generic alarming and fearful nature, to signal potential threats, with instantaneous, involuntary, and accurate recognition by perceivers. However, scream calls are more diverse in their affective signaling nature than being limited to fearfully alarming a threat, and thus the broader sociobiological relevance of various scream types is unclear. Here we used 4 different psychoacoustic, perceptual decision-making, and neuroimaging experiments in humans to demonstrate the existence of at least 6 psychoacoustically distinctive types of scream calls of both alarming and non-alarming nature, rather than there being only screams caused by fear or aggression. Second, based on perceptual and processing sensitivity measures for decision-making during scream recognition, we found that alarm screams (with some exceptions) were overall discriminated the worst, were responded to the slowest, and were associated with a lower perceptual sensitivity for their recognition compared with non-alarm screams. Third, the neural processing of alarm compared with non-alarm screams during an implicit processing task elicited only minimal neural signal and connectivity in perceivers, contrary to the frequent assumption of a threat processing bias of the primate neural system. These findings show that scream calls are more diverse in their signaling and communicative nature in humans than previously assumed, and, in contrast to a commonly observed threat processing bias in perceptual discriminations and neural processes, we found that especially non-alarm screams, and positive screams in particular, seem to have higher efficiency in speeded discriminations and the implicit neural processing of various scream types in humans.

Information diversity in individual auditory cortical neurons is associated with functionally distinct coordinated neuronal ensembles

Neuronal activity in auditory cortex is often highly synchronous between neighboring neurons. Such coordinated activity is thought to be crucial for information processing. We determined the functional properties of coordinated neuronal ensembles (cNEs) within primary auditory cortical (AI) columns relative to the contributing neurons. Nearly half of AI cNEs showed robust spectro-temporal receptive fields whereas the remaining cNEs showed little or no acoustic feature selectivity. cNEs can therefore capture either specific, time-locked information of spectro-temporal stimulus features or reflect stimulus-unspecific, less-time specific processing aspects. By contrast, we show that individual neurons can represent both of those aspects through membership in multiple cNEs with either high or absent feature selectivity. These associations produce functionally heterogeneous spikes identifiable by instantaneous association with different cNEs. This demonstrates that single neuron spike trains can sequentially convey multiple aspects that contribute to cortical processing, including stimulus-specific and unspecific information.

Speech compensation responses and sensorimotor adaptation to formant feedback perturbations

Control of speech formants is important for the production of distinguishable speech sounds and is achieved with both feedback and learned feedforward control. However, it is unclear whether the learning of feedforward control involves the mechanisms of feedback control. Speakers have been shown to compensate for unpredictable transient mid-utterance perturbations of pitch and loudness feedback, demonstrating online feedback control of these speech features. To determine whether similar feedback control mechanisms exist in the production of formants, responses to unpredictable vowel formant feedback perturbations were examined. Results showed similar within-trial compensatory responses to formant perturbations that were presented at utterance onset and mid-utterance. The relationship between online feedback compensation to unpredictable formant perturbations and sensorimotor adaptation to consistent formant perturbations was further examined. Within-trial online compensation responses were not correlated with across-trial sensorimotor adaptation. A detailed analysis of within-trial time course dynamics across trials during sensorimotor adaptation revealed that across-trial sensorimotor adaptation responses did not result from an incorporation of within-trial compensation response. These findings suggest that online feedback compensation and sensorimotor adaptation are governed by distinct neural mechanisms. These findings have important implications for models of speech motor control in terms of how feedback and feedforward control mechanisms are implemented.