Detecting Temporal Change in Dynamic Sounds: On the Role of Stimulus Duration, Speed, and Emotion

Musical rhythm effects on visual attention are non-rhythmical: evidence against metrical entrainment

The idea that external rhythms synchronize attention cross-modally has attracted much interest and scientific inquiry. Yet, whether associated attentional modulations are indeed rhythmical in that they spring from and map onto an underlying meter has not been clearly established. Here we tested this idea while addressing the shortcomings of previous work associated with confounding (i) metricality and regularity, (ii) rhythmic and temporal expectations or (iii) global and local temporal effects. We designed sound sequences that varied orthogonally (high/low) in metricality and regularity and presented them as task-irrelevant auditory background in four separate blocks. The participants' task was to detect rare visual targets occurring at a silent metrically aligned or misaligned temporal position. We found that target timing was irrelevant for reaction times and visual event-related potentials. High background regularity and to a lesser extent metricality facilitated target processing across metrically aligned and misaligned positions. Additionally, high regularity modulated auditory background frequencies in the EEG recorded over occipital cortex. We conclude that external rhythms, rather than synchronizing attention cross-modally, confer general, nontemporal benefits. Their predictability conserves processing resources that then benefit stimulus representations in other modalities.

Interactional synchrony: signals, mechanisms and benefits

Many group-living animals, humans included, occasionally synchronize their behavior with that of conspecifics. Social psychology and neuroscience have attempted to explain this phenomenon. Here we sought to integrate results around three themes: the stimuli, the mechanisms and the benefits of interactional synchrony. As regards stimuli, we asked what characteristics, apart from temporal regularity, prompt synchronization and found that stimulus modality and complexity are important. The high temporal resolution of the auditory system and the relevance of socio-emotional information endow auditory, multimodal, emotional and somewhat variable and adaptive sequences with particular synchronizing power. Looking at the mechanisms revealed that traditional perspectives emphasizing beat-based representations of others' signals conflict with more recent work investigating the perception of temporal regularity. Timing processes supported by striato-cortical loops represent any kind of repetitive interval sequence fairly automatically. Additionally, socio-emotional processes supported by posterior superior temporal cortex help endow such sequences with value motivating the extent of synchronizing. Synchronizing benefits arise from an increased predictability of incoming signals and include many positive outcomes ranging from basic information processing at the individual level to the bonding of dyads and larger groups.

Arousing the Sound: A Field Study on the Emotional Impact on Children of Arousing Sound Design and 3D Audio Spatialization in an Audio Story

Sound from media increases the immersion of the audience in the story, adding credibility to the narration but also generating emotions in the spectator. A study on children aged 9–13 years (N = 253), using an audio story, investigated the emotional impact of arousal vs. neutral treatment of sound and 3D vs. stereo mix spatialization. The emotional impact was measured combining three different measures: physiological (Electrodermal activity), self-report (pre-post exposition), and richness of mental images elicited by the story (using Think-aloud technique after exposition). Results showed higher emotional impact of the arousal and 3D audio conditions with different patterns according to the age of the participants and distinctive types of interaction when both variables were combined.

Cortical Microcircuit Mechanisms of Mismatch Negativity and Its Underlying Subcomponents

In the neocortex, neuronal processing of sensory events is significantly influenced by context. For instance, responses in sensory cortices are suppressed to repetitive or redundant stimuli, a phenomenon termed “stimulus-specific adaptation” (SSA). However, in a context in which that same stimulus is novel, or deviates from expectations, neuronal responses are augmented. This augmentation is termed “deviance detection” (DD). This contextual modulation of neural responses is fundamental for how the brain efficiently processes the sensory world to guide immediate and future behaviors. Notably, context modulation is deficient in some neuropsychiatric disorders such as schizophrenia (SZ), as quantified by reduced “mismatch negativity” (MMN), an electroencephalography waveform reflecting a combination of SSA and DD in sensory cortex. Although the role of NMDA-receptor function and other neuromodulatory systems on MMN is established, the precise microcircuit mechanisms of MMN and its underlying components, SSA and DD, remain unknown. When coupled with animal models, the development of powerful precision neurotechnologies over the past decade carries significant promise for making new progress into understanding the neurobiology of MMN with previously unreachable spatial resolution. Currently, rodent models represent the best tool for mechanistic study due to the vast genetic tools available. While quantifying human-like MMN waveforms in rodents is not straightforward, the “oddball” paradigms used to study it in humans and its underlying subcomponents (SSA/DD) are highly translatable across species. Here we summarize efforts published so far, with a focus on cortically measured SSA and DD in animals to maintain relevance to the classically measured MMN, which has cortical origins. While mechanistic studies that measure and contrast both components are sparse, we synthesize a potential set of microcircuit mechanisms from the existing rodent, primate, and human literature. While MMN and its subcomponents likely reflect several mechanisms across multiple brain regions, understanding fundamental microcircuit mechanisms is an important step to understand MMN as a whole. We hypothesize that SSA reflects adaptations occurring at synapses along the sensory-thalamocortical pathways, while DD depends on both SSA inherited from afferent inputs and resulting disinhibition of non-adapted neurons arising from the distinct physiology and wiring properties of local interneuronal subpopulations and NMDA-receptor function.

Interaction of Self-Regulation and Contextual Effects on Pre-attentive Auditory Processing: A Combined EEG/ECG Study

Environmental changes are not always within the focus of our attention, and sensitive reactions (i.e., quicker and stronger responses) can be essential for an organism's survival and adaptation. Here we report that neurophysiological responses to sound changes that are not in the focus of attention are related to both ambient acoustic contexts and regulation ability. We assessed electroencephalograph (EEG) mismatch negativity (MMN) latency and amplitude in response to sound changes in two contexts: ascending and descending pitch sequences while participants were instructed to attend to muted videos. Prolonged latency and increased amplitude of MMN at fronto-central region occurred in ascending pitch sequences relative to descending sequences. We also assessed how regulation related to the contextual effects on MMN. Reactions to changes in the ascending sequence were observed with the attention control (frontal EEG theta/beta ratio) indicating speed of reaction, and the autonomous regulation (heart-rate variability) indicating intensity of reaction. Moreover, sound changes in the ascending context were associated with more activation of anterior cingulate cortex and insula, suggesting arousal effects and regulation processes. These findings suggest that the relation between speed and intensity is not fixed and may be modified by contexts and self-regulation ability. Specifically, cortical and cardiovascular indicators of self-regulation may specify different aspects of response sensitivity in terms of speed and intensity.

Is the voice an auditory face? An ALE meta-analysis comparing vocal and facial emotion processing

This meta-analysis compares the brain structures and mechanisms involved in facial and vocal emotion recognition. Neuroimaging studies contrasting emotional with neutral (face: N = 76, voice: N = 34) and explicit with implicit emotion processing (face: N = 27, voice: N = 20) were collected to shed light on stimulus and goal-driven mechanisms, respectively. Activation likelihood estimations were conducted on the full data sets for the separate modalities and on reduced, modality-matched data sets for modality comparison. Stimulus-driven emotion processing engaged large networks with significant modality differences in the superior temporal (voice-specific) and the medial temporal (face-specific) cortex. Goal-driven processing was associated with only a small cluster in the dorsomedial prefrontal cortex for voices but not faces. Neither stimulus- nor goal-driven processing showed significant modality overlap. Together, these findings suggest that stimulus-driven processes shape activity in the social brain more powerfully than goal-driven processes in both the visual and the auditory domains. Yet, whereas faces emphasize subcortical emotional and mnemonic mechanisms, voices emphasize cortical mechanisms associated with perception and effortful stimulus evaluation (e.g. via subvocalization). These differences may be due to sensory stimulus properties and highlight the need for a modality-specific perspective when modeling emotion processing in the brain.

Emotional prosodic change detection in autism Spectrum disorder: an electrophysiological investigation in children and adults

Background

Autism spectrum disorder (ASD) is characterized by atypical behaviors in social environments and in reaction to changing events. While this dyad of symptoms is at the core of the pathology along with atypical sensory behaviors, most studies have investigated only one dimension. A focus on the sameness dimension has shown that intolerance to change is related to an atypical pre-attentional detection of irregularity. In the present study, we addressed the same process in response to emotional change in order to evaluate the interplay between alterations of change detection and socio-emotional processing in children and adults with autism.

Methods

Brain responses to neutral and emotional prosodic deviancies (mismatch negativity (MMN) and P3a, reflecting change detection and orientation of attention toward change, respectively) were recorded in children and adults with autism and in controls. Comparison of neutral and emotional conditions allowed distinguishing between general deviancy and emotional deviancy effects. Moreover, brain responses to the same neutral and emotional stimuli were recorded when they were not deviants to evaluate the sensory processing of these vocal stimuli.

Results

In controls, change detection was modulated by prosody: in children, this was characterized by a lateralization of emotional MMN to the right hemisphere, and in adults, by an earlier MMN for emotional deviancy than for neutral deviancy. In ASD, an overall atypical change detection was observed with an earlier MMN and a larger P3a compared to controls suggesting an unusual pre-attentional orientation toward any changes in the auditory environment. Moreover, in children with autism, deviancy detection depicted reduced MMN amplitude. In addition in children with autism, contrary to adults with autism, no modulation of the MMN by prosody was present and sensory processing of both neutral and emotional vocal stimuli appeared atypical.

Conclusions

Overall, change detection remains altered in people with autism. However, differences between children and adults with ASD evidence a trend toward normalization of vocal processing and of the automatic detection of emotion deviancy with age.

Temporal signatures of processing voiceness and emotion in sound

This study explored the temporal course of vocal and emotional sound processing. Participants detected rare repetitions in a stimulus stream comprising neutral and surprised non-verbal exclamations and spectrally rotated control sounds. Spectral rotation preserved some acoustic and emotional properties of the vocal originals. Event-related potentials elicited to unrepeated sounds revealed effects of voiceness and emotion. Relative to non-vocal sounds, vocal sounds elicited a larger centro-parietally distributed N1. This effect was followed by greater positivity to vocal relative to non-vocal sounds beginning with the P2 and extending throughout the recording epoch (N4, late positive potential) with larger amplitudes in female than in male listeners. Emotion effects overlapped with the voiceness effects but were smaller and differed topographically. Voiceness and emotion interacted only for the late positive potential, which was greater for vocal-emotional as compared with all other sounds. Taken together, these results point to a multi-stage process in which voiceness and emotionality are represented independently before being integrated in a manner that biases responses to stimuli with socio-emotional relevance.

Does emotion change auditory prediction and deviance detection?

In the last decades, a growing number of studies provided compelling evidence supporting the interplay of cognitive and affective processes. However, it remains to be clarified whether and how an emotional context affects the prediction and detection of change in unattended sensory events. In an event-related potential (ERP) study, we probed the modulatory role of pleasant, unpleasant and neutral visual contexts on the brain response to automatic detection of change in spectral (intensity) vs. temporal (duration) sound features. Twenty participants performed a passive auditory oddball task. Additionally, we tested the relationship between ERPs and self-reported mood. Participants reported more negative mood after the negative block. The P2 amplitude elicited by standards was increased in a positive context. Mismatch Negativity (MMN) amplitude was decreased in the negative relative to the neutral and positive contexts, and was associated with self-reported mood. These findings suggest that the detection of regularities in the auditory stream was facilitated in a positive context, whereas a negative visual context interfered with prediction error elicitation, through associated mood changes. Both ERP and behavioral effects highlight the intricate links between emotion, perception and cognitive processes.