TenseMusic: An automatic prediction model for musical tension

The perception of tension and release dynamics constitutes one of the essential aspects of music listening. However, modeling musical tension to predict perception of listeners has been a challenge to researchers. Seminal work demonstrated that tension is reported consistently by listeners and can be accurately predicted from a discrete set of musical features, combining them into a weighted sum of slopes reflecting their combined dynamics over time. However, previous modeling approaches lack an automatic pipeline for feature extraction that would make them widely accessible to researchers in the field. Here, we present TenseMusic: an open-source automatic predictive tension model that operates with a musical audio as the only input. Using state-of-the-art music information retrieval (MIR) methods, it automatically extracts a set of six features (i.e., loudness, pitch height, tonal tension, roughness, tempo, and onset frequency) to use as predictors for musical tension. The algorithm was optimized using Lasso regression to best predict behavioral tension ratings collected on 38 Western classical musical pieces. Its performance was then tested by assessing the correlation between the predicted tension and unseen continuous behavioral tension ratings yielding large mean correlations between ratings and predictions approximating r = .60 across all pieces. We hope that providing the research community with this well-validated open-source tool for predicting musical tension will motivate further work in music cognition and contribute to elucidate the neural and cognitive correlates of tension dynamics for various musical genres and cultures.

The development of oscillatory and aperiodic resting state activity is linked to a sensitive period in humans

Congenital blindness leads to profound changes in electroencephalographic (EEG) resting state activity. A well-known consequence of congenital blindness in humans is the reduction of alpha activity which seems to go together with increased gamma activity during rest. These results have been interpreted as indicating a higher excitatory/inhibitory (E/I) ratio in visual cortex compared to normally sighted controls. Yet it is unknown whether the spectral profile of EEG during rest would recover if sight were restored. To test this question, the present study evaluated periodic and aperiodic components of the EEG resting state power spectrum. Previous research has linked the aperiodic components, which exhibit a power-law distribution and are operationalized as a linear fit of the spectrum in log-log space, to cortical E/I ratio. Moreover, by correcting for the aperiodic components from the power spectrum, a more valid estimate of the periodic activity is possible. Here we analyzed resting state EEG activity from two studies involving (1) 27 permanently congenitally blind adults (CB) and 27 age-matched normally sighted controls (MCB); (2) 38 individuals with reversed blindness due to bilateral, dense, congenital cataracts (CC) and 77 age-matched sighted controls (MCC). Based on a data driven approach, aperiodic components of the spectra were extracted for the low frequency (Lf-Slope 1.5 to 19.5 Hz) and high frequency (Hf-Slope 20 to 45 Hz) range. The Lf-Slope of the aperiodic component was significantly steeper (more negative slope), and the Hf-Slope of the aperiodic component was significantly flatter (less negative slope) in CB and CC participants compared to the typically sighted controls. Alpha power was significantly reduced, and gamma power was higher in the CB and the CC groups. These results suggest a sensitive period for the typical development of the spectral profile during rest and thus likely an irreversible change in the E/I ratio in visual cortex due to congenital blindness. We speculate that these changes are a consequence of impaired inhibitory circuits and imbalanced feedforward and feedback processing in early visual areas of individuals with a history of congenital blindness.

Explaining flexible continuous speech comprehension from individual motor rhythms

When speech is too fast, the tracking of the acoustic signal along the auditory pathway deteriorates, leading to suboptimal speech segmentation and decoding of speech information. Thus, speech comprehension is limited by the temporal constraints of the auditory system. Here we ask whether individual differences in auditory-motor coupling strength in part shape these temporal constraints. In two behavioural experiments, we characterize individual differences in the comprehension of naturalistic speech as function of the individual synchronization between the auditory and motor systems and the preferred frequencies of the systems. Obviously, speech comprehension declined at higher speech rates. Importantly, however, both higher auditory-motor synchronization and higher spontaneous speech motor production rates were predictive of better speech-comprehension performance. Furthermore, performance increased with higher working memory capacity (digit span) and higher linguistic, model-based sentence predictability-particularly so at higher speech rates and for individuals with high auditory-motor synchronization. The data provide evidence for a model of speech comprehension in which individual flexibility of not only the motor system but also auditory-motor synchronization may play a modulatory role.

Dynamics of Functional Networks for Syllable and Word-Level Processing

Speech comprehension requires the ability to temporally segment the acoustic input for higher-level linguistic analysis. Oscillation-based approaches suggest that low-frequency auditory cortex oscillations track syllable-sized acoustic information and therefore emphasize the relevance of syllabic-level acoustic processing for speech segmentation. How syllabic processing interacts with higher levels of speech processing, beyond segmentation, including the anatomical and neurophysiological characteristics of the networks involved, is debated. In two MEG experiments, we investigate lexical and sublexical word-level processing and the interactions with (acoustic) syllable processing using a frequency-tagging paradigm. Participants listened to disyllabic words presented at a rate of 4 syllables/s. Lexical content (native language), sublexical syllable-to-syllable transitions (foreign language), or mere syllabic information (pseudo-words) were presented. Two conjectures were evaluated: (i) syllable-to-syllable transitions contribute to word-level processing; and (ii) processing of words activates brain areas that interact with acoustic syllable processing. We show that syllable-to-syllable transition information compared to mere syllable information, activated a bilateral superior, middle temporal and inferior frontal network. Lexical content resulted, additionally, in increased neural activity. Evidence for an interaction of word- and acoustic syllable-level processing was inconclusive. Decreases in syllable tracking (cerebroacoustic coherence) in auditory cortex and increases in cross-frequency coupling between right superior and middle temporal and frontal areas were found when lexical content was present compared to all other conditions; however, not when conditions were compared separately. The data provide experimental insight into how subtle and sensitive syllable-to-syllable transition information for word-level processing is.

Musical Sophistication and Speech Auditory-Motor Coupling: Easy Tests for Quick Answers

Musical training enhances auditory-motor cortex coupling, which in turn facilitates music and speech perception. How tightly the temporal processing of music and speech are intertwined is a topic of current research. We investigated the relationship between musical sophistication (Goldsmiths Musical Sophistication index, Gold-MSI) and spontaneous speech-to-speech synchronization behavior as an indirect measure of speech auditory-motor cortex coupling strength. In a group of participants (n = 196), we tested whether the outcome of the spontaneous speech-to-speech synchronization test (SSS-test) can be inferred from self-reported musical sophistication. Participants were classified as high (HIGHs) or low (LOWs) synchronizers according to the SSS-test. HIGHs scored higher than LOWs on all Gold-MSI subscales (General Score, Active Engagement, Musical Perception, Musical Training, Singing Skills), but the Emotional Attachment scale. More specifically, compared to a previously reported German-speaking sample, HIGHs overall scored higher and LOWs lower. Compared to an estimated distribution of the English-speaking general population, our sample overall scored lower, with the scores of LOWs significantly differing from the normal distribution, with scores in the ∼30th percentile. While HIGHs more often reported musical training compared to LOWs, the distribution of training instruments did not vary across groups. Importantly, even after the highly correlated subscores of the Gold-MSI were decorrelated, particularly the subscales Musical Perception and Musical Training allowed to infer the speech-to-speech synchronization behavior. The differential effects of musical perception and training were observed, with training predicting audio-motor synchronization in both groups, but perception only in the HIGHs. Our findings suggest that speech auditory-motor cortex coupling strength can be inferred from training and perceptual aspects of musical sophistication, suggesting shared mechanisms involved in speech and music perception.

Preferred auditory temporal processing regimes and auditory-motor synchronization

Decoding the rich temporal dynamics of complex sounds such as speech is constrained by the underlying neuronal-processing mechanisms. Oscillatory theories suggest the existence of one optimal perceptual performance regime at auditory stimulation rates in the delta to theta range (< 10 Hz), but reduced performance in the alpha range (10-14 Hz) is controversial. Additionally, the widely discussed motor system contribution to timing remains unclear. We measured rate discrimination thresholds between 4 and 15 Hz, and auditory-motor coupling strength was estimated through a behavioral auditory-motor synchronization task. In a Bayesian model comparison, high auditory-motor synchronizers showed a larger range of constant optimal temporal judgments than low synchronizers, with performance decreasing in the alpha range. This evidence for optimal processing in the theta range is consistent with preferred oscillatory regimes in auditory cortex that compartmentalize stimulus encoding and processing. The findings suggest, remarkably, that increased auditory-motor synchronization might extend such an optimal range towards faster rates.

Data-Driven Classification of Spectral Profiles Reveals Brain Region-Specific Plasticity in Blindness

Congenital blindness has been shown to result in behavioral adaptation and neuronal reorganization, but the underlying neuronal mechanisms are largely unknown. Brain rhythms are characteristic for anatomically defined brain regions and provide a putative mechanistic link to cognitive processes. In a novel approach, using magnetoencephalography resting state data of congenitally blind and sighted humans, deprivation-related changes in spectral profiles were mapped to the cortex using clustering and classification procedures. Altered spectral profiles in visual areas suggest changes in visual alpha-gamma band inhibitory-excitatory circuits. Remarkably, spectral profiles were also altered in auditory and right frontal areas showing increased power in theta-to-beta frequency bands in blind compared with sighted individuals, possibly related to adaptive auditory and higher cognitive processing. Moreover, occipital alpha correlated with microstructural white matter properties extending bilaterally across posterior parts of the brain. We provide evidence that visual deprivation selectively modulates spectral profiles, possibly reflecting structural and functional adaptation.

Speaking rhythmically can shape hearing

Evidence suggests that temporal predictions arising from the motor system can enhance auditory perception. However, in speech perception, we lack evidence of perception being modulated by production. Here we show a behavioural protocol that captures the existence of such auditory-motor interactions. Participants performed a syllable discrimination task immediately after producing periodic syllable sequences. Two speech rates were explored: a 'natural' (individually preferred) and a fixed 'non-natural' (2 Hz) rate. Using a decoding approach, we show that perceptual performance is modulated by the stimulus phase determined by a participant's own motor rhythm. Remarkably, for 'natural' and 'non-natural' rates, this finding is restricted to a subgroup of the population with quantifiable auditory-motor coupling. The observed pattern is compatible with a neural model assuming a bidirectional interaction of auditory and speech motor cortices. Crucially, the model matches the experimental results only if it incorporates individual differences in the strength of the auditory-motor connection.

Post-training Load-Related Changes of Auditory Working Memory - An EEG Study

Working memory (WM) refers to the temporary retention and manipulation of information, and its capacity is highly susceptible to training. Yet, the neural mechanisms that allow for increased performance under demanding conditions are not fully understood. We expected that post-training efficiency in WM performance modulates neural processing during high load tasks. We tested this hypothesis, using electroencephalography (EEG) (N = 39), by comparing source space spectral power of healthy adults performing low and high load auditory WM tasks. Prior to the assessment, participants either underwent a modality-specific auditory WM training, or a modality-irrelevant tactile WM training, or were not trained (active control). After a modality-specific training participants showed higher behavioral performance, compared to the control. EEG data analysis revealed general effects of WM load, across all training groups, in the theta-, alpha-, and beta-frequency bands. With increased load theta-band power increased over frontal, and decreased over parietal areas. Centro-parietal alpha-band power and central beta-band power decreased with load. Interestingly, in the high load condition a tendency toward reduced beta-band power in the right medial temporal lobe was observed in the modality-specific WM training group compared to the modality-irrelevant and active control groups. Our finding that WM processing during the high load condition changed after modality-specific WM training, showing reduced beta-band activity in voice-selective regions, possibly indicates a more efficient maintenance of task-relevant stimuli. The general load effects suggest that WM performance at high load demands involves complementary mechanisms, combining a strengthening of task-relevant and a suppression of task-irrelevant processing.

The effects of selective attention and speech acoustics on neural speech-tracking in a multi-talker scene

Attending to one speaker in multi-speaker situations is challenging. One neural mechanism proposed to underlie the ability to attend to a particular speaker is phase-locking of low-frequency activity in auditory cortex to speech's temporal envelope ("speech-tracking"), which is more precise for attended speech. However, it is not known what brings about this attentional effect, and specifically if it reflects enhanced processing of the fine structure of attended speech. To investigate this question we compared attentional effects on speech-tracking of natural versus vocoded speech which preserves the temporal envelope but removes the fine structure of speech. Pairs of natural and vocoded speech stimuli were presented concurrently and participants attended to one stimulus and performed a detection task while ignoring the other stimulus. We recorded magnetoencephalography (MEG) and compared attentional effects on the speech-tracking response in auditory cortex. Speech-tracking of natural, but not vocoded, speech was enhanced by attention, whereas neural tracking of ignored speech was similar for natural and vocoded speech. These findings suggest that the more precise speech-tracking of attended natural speech is related to processing its fine structure, possibly reflecting the application of higher-order linguistic processes. In contrast, when speech is unattended its fine structure is not processed to the same degree and thus elicits less precise speech-tracking more similar to vocoded speech.