Earlier timbre processing of instrumental tones compared to equally complex spectrally rotated sounds as revealed by the mismatch negativity

Seeking the neural representation of statistical properties in print during implicit processing of visual words

Statistical learning (SL) plays a key role in literacy acquisition. Studies have increasingly revealed the influence of distributional statistical properties of words on visual word processing, including the effects of word frequency (lexical level) and mappings between orthography, phonology, and semantics (sub-lexical level). However, there has been scant evidence to directly confirm that the statistical properties contained in print can be directly characterized by neural activities. Using time-resolved representational similarity analysis (RSA), the present study examined neural representations of different types of statistical properties in visual word processing. From the perspective of predictive coding, an equal probability sequence with low built-in prediction precision and three oddball sequences with high built-in prediction precision were designed with consistent and three types of inconsistent (orthographically inconsistent, orthography-to-phonology inconsistent, and orthography-to-semantics inconsistent) Chinese characters as visual stimuli. In the three oddball sequences, consistent characters were set as the standard stimuli (probability of occurrence p = 0.75) and three types of inconsistent characters were set as deviant stimuli (p = 0.25), respectively. In the equal probability sequence, the same consistent and inconsistent characters were presented randomly with identical occurrence probability (p = 0.25). Significant neural representation activities of word frequency were observed in the equal probability sequence. By contrast, neural representations of sub-lexical statistics only emerged in oddball sequences where short-term predictions were shaped. These findings reveal that the statistical properties learned from long-term print environment continues to play a role in current word processing mechanisms and these mechanisms can be modulated by short-term predictions.

Neural correlates of statistical learning in developmental dyslexia: An electroencephalography study

The human brain extracts statistical regularities from the surrounding environment in a process called statistical learning. Behavioural evidence suggests that developmental dyslexia affects statistical learning. However, surprisingly few studies have assessed how developmental dyslexia affects the neural processing underlying this type of learning. We used electroencephalography to explore the neural correlates of an important aspect of statistical learning - sensitivity to transitional probabilities - in individuals with developmental dyslexia. Adults diagnosed with developmental dyslexia (n = 17) and controls (n = 19) were exposed to a continuous stream of sound triplets. Every so often, a triplet ending had a low transitional probability given the triplet's first two sounds ("statistical deviants"). Furthermore, every so often a triplet ending was presented from a deviant location ("acoustic deviants"). We examined mismatch negativity elicited by statistical deviants (sMMN), and MMN elicited by location deviants (i.e., acoustic changes). Acoustic deviants elicited a MMN which was larger in the control group than in the developmental dyslexia group. Statistical deviants elicited a small, yet significant, sMMN in the control group, but not in the developmental dyslexia group. However, the difference between the groups was not significant. Our findings indicate that the neural mechanisms underlying pre-attentive acoustic change detection and implicit statistical auditory learning are both affected in developmental dyslexia.

A Review of Research on the Neurocognition for Timbre Perception

As one of the basic elements in acoustic events, timbre influences the brain collectively with other factors such as pitch and loudness. Research on timbre perception involve interdisciplinary fields, including physical acoustics, auditory psychology, neurocognitive science and music theory, etc. From the perspectives of psychology and physiology, this article summarizes the features and functions of timbre perception as well as their correlation, among which the multi-dimensional scaling modeling methods to define timbre are the focus; the neurocognition and perception of timbre (including sensitivity, adaptability, memory capability, etc.) are outlined; related experiment findings (by using EEG/ERP, fMRI, etc.) on the deeper level of timbre perception in terms of neural cognition are summarized. In the meantime, potential problems in the process of experiments on timbre perception and future possibilities are also discussed. Thought sorting out the existing research contents, methods and findings of timbre perception, this article aims to provide heuristic guidance for researchers in related fields of timbre perception psychology, physiology and neural mechanism. It is believed that the study of timbre perception will be essential in various fields in the future, including neuroaesthetics, psychological intervention, artistic creation, rehabilitation, etc.

Do categorical representations modulate early automatic visual processing? A visual mismatch-negativity study

Perceptual categorization is an important cognitive function. In the auditory domain, categorization already occurs within the first 200 ms of information processing, as indexed by the mismatch negativity. Here, we assessed the characteristics of the visual mismatch negativity (vMMN) elicited during the categorization of previously unknown visual stimuli. To examine this, we used five-dot patterns with characteristics that allow for the formation of categories through rotation and reflection but not through other physical properties. To assess whether or not between-category and within-category vMMN differ in amplitude, the data was analyzed with the Bayesian approach. We observed that both between-category and within-category deviants elicited a vMMN, but that both vMMNs were comparable in magnitude. This finding suggests that abstract categorical representations are not always automatically processed at early visual stages and demonstrates limitations of generalization from the auditory domain to visual domain.

Emotional Connotations of Musical Instrument Timbre in Comparison With Emotional Speech Prosody: Evidence From Acoustics and Event-Related Potentials

Music and speech both communicate emotional meanings in addition to their domain-specific contents. But it is not clear whether and how the two kinds of emotional meanings are linked. The present study is focused on exploring the emotional connotations of musical timbre of isolated instrument sounds through the perspective of emotional speech prosody. The stimuli were isolated instrument sounds and emotional speech prosody categorized by listeners into anger, happiness and sadness, respectively. We first analyzed the timbral features of the stimuli, which showed that relations between the three emotions were relatively consistent in those features for speech and music. The results further echo the size-code hypothesis in which different sound timbre indicates different body size projections. Then we conducted an ERP experiment using a priming paradigm with isolated instrument sounds as primes and emotional speech prosody as targets. The results showed that emotionally incongruent instrument-speech pairs triggered a larger N400 response than emotionally congruent pairs. Taken together, this is the first study to provide evidence that the timbre of simple and isolated musical instrument sounds can convey emotion in a way similar to emotional speech prosody.

Cortical Sensitivity to Guitar Note Patterns: EEG Entrainment to Repetition and Key

Music is ubiquitous throughout recent human culture, and many individual's have an innate ability to appreciate and understand music. Our appreciation of music likely emerges from the brain's ability to process a series of repeated complex acoustic patterns. In order to understand these processes further, cortical responses were measured to a series of guitar notes presented with a musical pattern or without a pattern. ERP responses to individual notes were measured using a 24 electrode Bluetooth mobile EEG system (Smarting mBrainTrain) while 13 healthy non-musicians listened to structured (i.e., within musical keys and with repetition) or random sequences of guitar notes for 10 min each. We demonstrate an increased amplitude to the ERP that appears ~200 ms to notes presented within the musical sequence. This amplitude difference between random notes and patterned notes likely reflects individual's cortical sensitivity to guitar note patterns. These amplitudes were compared to ERP responses to a rare note embedded within a stream of frequent notes to determine whether the sensitivity to complex musical structure overlaps with the sensitivity to simple irregularities reflected in traditional auditory oddball experiments. Response amplitudes to the negative peak at ~175 ms are statistically correlated with the mismatch negativity (MMN) response measured to a rare note presented among a series of frequent notes (i.e., in a traditional oddball sequence), but responses to the subsequent positive peak at ~200 do not show a statistical relationship with the P300 response. Thus, the sensitivity to musical structure identified to 4 Hz note patterns appears somewhat distinct from the sensitivity to statistical regularities reflected in the traditional "auditory oddball" sequence. Overall, we suggest that this is a promising approach to examine individual's sensitivity to complex acoustic patterns, which may overlap with higher level cognitive processes, including language.

When speech enhances Spatial Musical Association of Response Codes: Joint spatial associations of pitch and timbre in nonmusicians

Previous studies have shown that the effect of the Spatial Musical Association of Response Codes (SMARC) depends on various features, such as task conditions (whether pitch height is implicit or explicit), response dimension (horizontal vs. vertical), presence or absence of a reference tone, and former musical training of the participants. In the present study, we investigated the effects of pitch range and timbre: in particular, how timbre (piano vs. vocal) contributes to the horizontal and vertical SMARC effect in nonmusicians under varied pitch range conditions. Nonmusicians performed a timbre judgement task in which the pitch range was either small (6 or 8 semitone steps) or large (9 or 12 semitone steps) in a horizontal and a vertical response setting. For piano sounds, SMARC effects were observed in all conditions. For the vocal sounds, in contrast, SMARC effects depended on pitch range. We concluded that the occurrence of the SMARC effect, especially in horizontal response settings, depends on the interaction of the timbre (vocal and piano) and pitch range if vocal and instrumental sounds are combined in one experiment: the human voice enhances the attention, both to the vocal and the instrumental sounds.

Location negative priming effects in children with developmental dyslexia: An event-related potential study

As the reading process is inseparable from working memory, inhibition, and other higher cognitive processes, the deep cognitive processing defects that are associated with dyslexia may be due to defective distraction inhibition systems. In this study, we used event-related potential technology to explore the source of negative priming effects in children with developmental dyslexia and in a group of healthy children for comparison. We found that the changes in the average response times in the negative priming and control conditions were consistent across the two groups, while the negative priming effects differed significantly between the groups. The magnitude of the negative priming effect was significantly different between the two groups, with the magnitude being significantly higher in the control group than it was in the developmental dyslexia group. These results indicate that there are deficits in distraction inhibition in children with developmental dyslexia. In terms of the time course of processing, inhibition deficits in the dyslexia group appeared during early-stage cognition selection and lasted through the response selection phase. Regarding the cerebral cortex locations, early-stage cognition selection was mainly located in the parietal region, while late-stage response selection was mainly located in the frontal and central regions. The results of our study may help further our understanding of the intrinsic causes of developmental dyslexia.

Pre-attentive dysfunction of musical processing in major depressive disorder: A mismatch negativity study

BACKGROUND

Deficits of pre-attentive information processing have been frequently found in patients with major depressive disorder, nevertheless the results are quite inconsistent due to clinical heterogeneity and methodological difference. Cognitive processing of music is a useful tool for investigating human cognition and its underlying brain mechanisms. Although general auditory processing and perception of musical sound are hampered in patients with MDD, whether the deficits in musical processing begin from pre-attentive stage is not well investigated yet. The present study aimed to investigate the MMN of musical sound in patients with MDD.

METHOD

MMN responses to different musical features were compared in 20 patients with MDD and 20 age-matched healthy controls. The multi-feature paradigm was used to examine automatic change detection of six different musical sound features (pitch, timbre, location, intensity, slide, rhythm) in a complex musical context. Severity of depression and co-morbid anxiety were evaluated using the Hamilton Rating Scale of Depression (HRSD-17) and the Hamilton Anxiety Rating Scale (HAMA).

RESULTS

MMNs were obtained with all deviants. The timbre-MMN was significantly larger in MDD patients than in healthy controls, while the other deviants (pitch, location, intensity, slide and rhythm) elicited similar MMN across groups. For MDD patients, the amplitudes and latencies of MMNs did not correlate with severity of depression or co-morbid anxiety.

LIMITATIONS

The sample size in this study is relatively small.

CONCLUSION

Patients with MDD do not perform at the same level as controls in automatic change detection of timbre. This dysfunction is considered to be a trait-dependent feature of MDD.

Nonverbal auditory working memory: Can music indicate the capacity?

Different working memory (WM) mechanisms that underlie words, tones, and timbres have been proposed in previous studies. In this regard, the present study developed a WM test with nonverbal sounds and compared it to the conventional verbal WM test. A total of twenty-five, non-music major, right-handed college students were presented with four different types of sounds (words, syllables, pitches, timbres) that varied from two to eight digits in length. Both accuracy and oxygenated hemoglobin (oxyHb) were measured. The results showed significant effects of number of targets on accuracy and sound type on oxyHb. A further analysis showed prefrontal asymmetry with pitch being processed by the right hemisphere (RH) and timbre by the left hemisphere (LH). These findings suggest a potential for employing musical sounds (i.e., pitch and timbre) as a complementary stimuli for conventional nonverbal WM tests, which can additionally examine its asymmetrical roles in the prefrontal regions.

The Mismatch Negativity: An Indicator of Perception of Regularities in Music

This paper reviews music research using Mismatch Negativity (MMN). MMN is a deviation-specific component of auditory event-related potential (EPR), which detects a deviation between a sound and an internal representation (e.g., memory trace). Recent studies have expanded the notion and the paradigms of MMN to higher-order music processing such as those involving short melodies, harmony chord, and music syntax. In this vein, we firstly reviewed the evolution of MMN from sound to music and then mainly compared the differences of MMN features between musicians and nonmusicians, followed by the discussion of the potential roles of the training effect and the natural exposure in MMN. Since MMN can serve as an index of neural plasticity, it thus can be widely used in clinical and other applied areas, such as detecting music preference in newborns or assessing wholeness of central auditory system of hearing illness. Finally, we pointed out some open questions and further directions. Current music perception research using MMN has mainly focused on relatively low hierarchical structure of music perception. To fully understand the neural substrates underlying processing of regularities in music, it is important and beneficial to combine MMN with other experimental paradigms such as early right-anterior negativity (ERAN).