Pathways to seeing music: enhanced structural connectivity in colored-music synesthesia

Creative tempo: Spatiotemporal dynamics of the default mode network in improvisational musicians

The intrinsic dynamics of human brain activity display a recurring pattern of anti-correlated activity between the default mode network (DMN), associated with internal processing and mentation, and task positive regions, associated with externally directed attention. In human functional magnetic resonance imaging (fMRI) data, this anti-correlated pattern is detectable on the infraslow timescale (<0.1 Hz) as a quasi-periodic pattern (QPP). While the DMN is implicated in creativity and musicality in traditional time-averaged functional connectivity studies, no one has yet explored how creative training may alter dynamic spatiotemporal patterns involving the DMN such as QPPs. In the present study, we compare the outputs of two QPP detection approaches, sliding window algorithm and complex principal components analysis (cPCA). We apply both methods to an existing dataset of musicians captured with resting state fMRI, grouped as either classical, improvisational, or minimally trained non-musicians. The original time-averaged functional connectivity (FC) analysis of this dataset used improvisation as a proxy for creative thinking and found that the DMN and visual networks (VIS) display higher connectivity in improvisational musicians. We expand upon this dataset's original study and find that QPP analysis detects convergent results at the group level with both methods. In improvisational musicians, dynamic functional correlation in the group-averaged QPP was found to be increased between the DMN-VIS and DMN-FPN for both the QPP algorithm and complex principal components analysis (cPCA) methods. Additionally, we found an unexpected increase in FC in the group-averaged QPP between the dorsal attention network and amygdala in improvisational musicians; this result was not reported in the original seed-based study of this dataset. The current study represents a novel application of two dynamic FC detection methods with results that replicate and expand upon previous seed-based FC findings. The results show the robustness of both the QPP phenomenon and its detection methods. This study also demonstrates the value of dynamic FC methods in reproducing seed-based findings and their promise in detecting group-wise or individual differences that may be missed by traditional seed-based resting state fMRI studies.

Action, emotion, and music-colour synaesthesia: an examination of sensorimotor and emotional responses in synaesthetes and non-synaesthetes

Synaesthesia has been conceptualised as a joining of sensory experiences. Taking a holistic, embodied perspective, we investigate in this paper the role of action and emotion, testing hypotheses related to (1) changes to action-related qualities of a musical stimulus affect the resulting synaesthetic experience; (2) a comparable relationship exists between music, sensorimotor and emotional responses in synaesthetes and the general population; and (3) sensorimotor responses are more strongly associated with synaesthesia than emotion. 29 synaesthetes and 33 non-synaesthetes listened to 12 musical excerpts performed on a musical instrument they had first-hand experience playing, an instrument never played before, and a deadpan performance generated by notation software, i.e., a performance without expression. They evaluated the intensity of their experience of the music using a list of dimensions that relate to sensorimotor, emotional or synaesthetic sensations. Results demonstrated that the intensity of listeners' responses was most strongly influenced by whether or not music is performed by a human, more so than familiarity with a particular instrument. Furthermore, our findings reveal a shared relationship between emotional and sensorimotor responses among both synaesthetes and non-synaesthetes. Yet it was sensorimotor intensity that was shown to be fundamentally associated with the intensity of the synaesthetic response. Overall, the research argues for, and gives first evidence of a key role of action in shaping the experiences of music-colour synaesthesia.

fNIRS-Based Dynamic Functional Connectivity Reveals the Innate Musical Sensing Brain Networks in Preterm Infants

Humans have the ability to appreciate and create music. However, why and how humans have this distinctive ability to perceive music remains unclear. Additionally, the investigation of the innate perceiving skill in humans is compounded by the fact that we have been actively and passively exposed to auditory stimuli or have systematically learnt music after birth. Therefore, to explore the innate musical perceiving ability, infants with preterm birth may be the most suitable population. In this study, the auditory brain networks were explored using dynamic functional connectivity-based reliable component analysis (RCA) in preterm infants during music listening. The brain activation was captured by portable functional near-infrared spectroscopy (fNIRS) to simulate a natural environment for preterm infants. The components with the maximum inter-subject correlation were extracted. The generated spatial filters identified the shared spatial structural features of functional brain connectivity across subjects during listening to the common music, exhibiting a functional synchronization between the right temporal region and the frontal and motor cortex, and synchronization between the bilateral temporal regions. The specific pattern is responsible for the functions involving music comprehension, emotion generation, language processing, memory, and sensory. The fluctuation of the extracted components and the phase variation demonstrates the interactions between the extracted brain networks to encode musical information. These results are critically important for our understanding of the underlying mechanisms of the innate perceiving skills at early ages of human during naturalistic music listening.

The Benefits of Music Listening for Induced State Anxiety: Behavioral and Physiological Evidence

BACKGROUND

Some clinical studies have indicated that neutral and happy music may relieve state anxiety. However, the brain mechanisms by which these effective interventions in music impact state anxiety remain unknown.

METHODS

In this study, we selected music with clinical effects for therapy, and 62 subjects were included using the evoked anxiety paradigm. After evoking anxiety with a visual stimulus, all subjects were randomly divided into three groups (listening to happy music, neutral music and a blank stimulus), and EEG signals were acquired.

RESULTS

We found that different emotional types of music might have different mechanisms in state anxiety interventions. Neutral music had the effect of alleviating state anxiety. The brain mechanisms supported that neutral music ameliorating state anxiety was associated with decreased power spectral density of the occipital lobe and increased brain functional connectivity between the occipital lobe and frontal lobe. Happy music also had the effect of alleviating state anxiety, and the brain mechanism was associated with enhanced brain functional connectivity between the occipital lobe and right temporal lobe.

CONCLUSIONS

This study may be important for a deep understanding of the mechanisms associated with state anxiety music interventions and may further contribute to future clinical treatment using nonpharmaceutical interventions.

Spatiospectral brain networks reflective of improvisational experience

Musical improvisers are trained to categorize certain musical structures into functional classes, which is thought to facilitate improvisation. Using a novel auditory oddball paradigm (Goldman et al., 2020) which enables us to disassociate a deviant (i.e. musical chord inversion) from a consistent functional class, we recorded scalp EEG from a group of musicians who spanned a range of improvisational and classically trained experience. Using a spatiospectral based inter and intra network connectivity analysis, we found that improvisers showed a variety of differences in connectivity within and between large-scale cortical networks compared to classically trained musicians, as a function of deviant type. Inter-network connectivity in the alpha band, for a time window leading up to the behavioural response, was strongly linked to improvisation experience, with the default mode network acting as a hub. Spatiospectral networks post response were substantially different between improvisers and classically trained musicians, with greater inter-network connectivity (specific to the alpha and beta bands) seen in improvisers whereas those with more classical training had largely reduced inter-network activity (mostly in the gamma band). More generally, we interpret our findings in the context of network-level correlates of expectation violation as a function of subject expertise, and we discuss how these may generalize to other and more ecologically valid scenarios.

Enhanced White Matter Fiber Tracts in Advanced Jazz Improvisers

Human cognition and behavior arise from neuronal interactions over brain structural networks. These neuronal interactions cause changes in structural networks over time. How a creative activity such as musical improvisation performance changes the brain structure is largely unknown. In this diffusion magnetic resonance imaging study, we examined the brain’s white matter fiber properties in previously identified functional networks and compared the findings between advanced jazz improvisers and non-musicians. We found that, for advanced improvisers compared with non-musicians, the normalized quantitative anisotropy (NQA) is elevated in the lateral prefrontal areas and supplementary motor area, and the underlying white matter fiber tracts connecting these areas. This enhancement of the diffusion anisotropy along the fiber pathway connecting the lateral prefrontal and supplementary motor is consistent with the functional networks during musical improvisation tasks performed by expert jazz improvisers. These findings together suggest that experts’ creative skill is associated with the task-relevant, long-timescale brain structural network changes, in support of related cognitive underpinnings.

Audiovisual structural connectivity in musicians and non-musicians: a cortical thickness and diffusion tensor imaging study

Our sensory systems provide complementary information about the multimodal objects and events that are the target of perception in everyday life. Professional musicians' specialization in the auditory domain is reflected in the morphology of their brains, which has distinctive characteristics, particularly in areas related to auditory and audio-motor activity. Here, we combined diffusion tensor imaging (DTI) with a behavioral measure of visually induced gain in pitch discrimination, and we used measures of cortical thickness (CT) correlations to assess how auditory specialization and musical expertise are reflected in the structural architecture of white and grey matter relevant to audiovisual processing. Across all participants (n = 45), we found a correlation (p < 0.001) between reliance on visual cues in pitch discrimination and the fractional anisotropy (FA) in the left inferior fronto-occipital fasciculus (IFOF), a structure connecting visual and auditory brain areas. Group analyses also revealed greater cortical thickness correlation between visual and auditory areas in non-musicians (n = 28) compared to musicians (n = 17), possibly reflecting musicians' auditory specialization (FDR < 10%). Our results corroborate and expand current knowledge of functional specialization with a specific focus on audition, and highlight the fact that perception is essentially multimodal while uni-sensory processing is a specialized task.

Synaesthesia and autism: Different developmental outcomes from overlapping mechanisms?

Synaesthesia, a mixing of the senses, is more common in individuals with autism. Here, we review the evidence for the association between synaesthesia and autism with regard to their genetic background, brain connectivity, perception, cognitive mechanisms and their contribution to exceptional talents. Currently, the overlap between synaesthesia and autism is established most convincingly at the level of alterations in sensory sensitivity and perception, with synaesthetes showing autism-like profiles of sensory sensitivity and a bias towards details in perception. Shared features may include a predominance of local over global connectivity in the brain. When autism and synaesthesia co-occur in the same individual, the chance of developing heightened cognitive and memory abilities is increased. We discuss how the same theoretical models could potentially explain both conditions. Given the evidence, we believe the phenotypical overlap between autism and synaesthesia has been established clearly enough to invite future research to confirm overlapping mechanisms.

Reduced perceptual narrowing in synesthesia

Synesthesia is a neurologic trait in which specific inducers, such as sounds, automatically elicit additional idiosyncratic percepts, such as color (thus "colored hearing"). One explanation for this trait-and the one tested here-is that synesthesia results from unusually weak pruning of cortical synaptic hyperconnectivity during early perceptual development. We tested the prediction from this hypothesis that synesthetes would be superior at making discriminations from nonnative categories that are normally weakened by experience-dependent pruning during a critical period early in development-namely, discrimination among nonnative phonemes (Hindi retroflex /d̪a/ and dental /ɖa/), among chimpanzee faces, and among inverted human faces. Like the superiority of 6-mo-old infants over older infants, the synesthetic groups were significantly better than control groups at making all the nonnative discriminations across five samples and three testing sites. The consistent superiority of the synesthetic groups in making discriminations that are normally eliminated during infancy suggests that residual cortical connectivity in synesthesia supports changes in perception that extend beyond the specific synesthetic percepts, consistent with the incomplete pruning hypothesis.

Echoes from the past: synaesthetic colour associations reflect childhood gender stereotypes

Grapheme-colour synaesthesia is a neurological phenomenon in which linguistic symbols evoke consistent colour sensations. Synaesthesia is believed to be influenced by both genetic and environmental factors, but how these factors interact to create specific associations in specific individuals is poorly understood. In this paper, we show that a grapheme-colour association in adult synaesthetes can be traced to a particular environmental effect at a particular moment in childhood. We propose a model in which specific grapheme-colour associations are 'locked in' during development in children predisposed to become synaesthetes, whereas grapheme-colour associations remain flexible in non-synaesthetes. We exploit Western gender-colour stereotypes to test our model: we found that young girls in general tend to associate their first initial with the colour pink. Consistent with our model, adult female synaesthetes are influenced by their childhood environment: they associate their first initial with pink. Adult female non-synaesthetes do not show this bias. Instead, in our study, non-synaesthetes tended to associate their first initial with their current favourite colour. The results thus support the 'locking in' model of synaesthesia, suggesting that synaesthetic associations can be used as a 'time capsule', revealing childhood influences on adult linguistic associations. Grapheme-colour synaesthesia may thus offer an extraordinary opportunity to study linguistic development. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Stochastic resonance model of synaesthesia

In synaesthesia, stimulation of one sensory modality evokes additional experiences in another modality (e.g. sounds evoking colours). Along with these cross-sensory experiences, there are several cognitive and perceptual differences between synaesthetes and non-synaesthetes. For example, synaesthetes demonstrate enhanced imagery, increased cortical excitability and greater perceptual sensitivity in the concurrent modality. Previous models suggest that synaesthesia results from increased connectivity between corresponding sensory regions or disinhibited feedback from higher cortical areas. While these models explain how one sense can evoke qualitative experiences in another, they fail to predict the broader phenotype of differences observed in synaesthetes. Here, we propose a novel model of synaesthesia based on the principles of stochastic resonance. Specifically, we hypothesize that synaesthetes have greater neural noise in sensory regions, which allows pre-existing multisensory pathways to elicit supra-threshold activation (i.e. synaesthetic experiences). The strengths of this model are (a) it predicts the broader cognitive and perceptual differences in synaesthetes, (b) it provides a unified framework linking developmental and induced synaesthesias, and (c) it explains why synaesthetic associations are inconsistent at onset but stabilize over time. We review research consistent with this model and propose future studies to test its limits. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Autistic traits, resting-state connectivity, and absolute pitch in professional musicians: shared and distinct neural features

Background

Recent studies indicate increased autistic traits in musicians with absolute pitch and a higher proportion of absolute pitch in people with autism. Theoretical accounts connect both of these with shared neural principles of local hyper- and global hypoconnectivity, enhanced perceptual functioning, and a detail-focused cognitive style. This is the first study to investigate absolute pitch proficiency, autistic traits, and brain correlates in the same study.

Sample and methods

Graph theoretical analysis was conducted on resting-state (eyes closed and eyes open) EEG connectivity (wPLI, weighted phase lag index) matrices obtained from 31 absolute pitch (AP) and 33 relative pitch (RP) professional musicians. Small-worldness, global clustering coefficient, and average path length were related to autistic traits, passive (tone identification) and active (pitch adjustment) absolute pitch proficiency, and onset of musical training using Welch two-sample tests, correlations, and general linear models.

Results

Analyses revealed increased path length (delta 2–4 Hz), reduced clustering (beta 13–18 Hz), reduced small-worldness (gamma 30–60 Hz), and increased autistic traits for AP compared to RP. Only clustering values (beta 13–18 Hz) were predicted by both AP proficiency and autistic traits. Post hoc single connection permutation tests among raw wPLI matrices in the beta band (13–18 Hz) revealed widely reduced interhemispheric connectivity between bilateral auditory-related electrode positions along with higher connectivity between F7–F8 and F8–P9 for AP. Pitch-naming ability and pitch adjustment ability were predicted by path length, clustering, autistic traits, and onset of musical training (for pitch adjustment) explaining 44% and 38% of variance, respectively.

Conclusions

Results show both shared and distinct neural features between AP and autistic traits. Differences in the beta range were associated with higher autistic traits in the same population. In general, AP musicians exhibit a widely underconnected brain with reduced functional integration and reduced small-world property during resting state. This might be partly related to autism-specific brain connectivity, while differences in path length and small-worldness reflect other ability-specific influences. This is further evidenced for different pathways in the acquisition and development of absolute pitch, likely influenced by both genetic and environmental factors and their interaction.

Color, Music, and Emotion: Bach to the Blues

When people make cross-modal matches from classical music to colors, they choose colors whose emotional associations fit the emotional associations of the music, supporting the emotional mediation hypothesis. We further explored this result with a large, diverse sample of 34 musical excerpts from different genres, including Blues, Salsa, Heavy metal, and many others, a broad sample of 10 emotion-related rating scales, and a large range of 15 rated music-perceptual features. We found systematic music-to-color associations between perceptual features of the music and perceptual dimensions of the colors chosen as going best/worst with the music (e.g., loud, punchy, distorted music was generally associated with darker, redder, more saturated colors). However, these associations were also consistent with emotional mediation (e.g., agitated-sounding music was associated with agitated-looking colors). Indeed, partialling out the variance due to emotional content eliminated all significant cross-modal correlations between lower level perceptual features. Parallel factor analysis (Parafac, a type of factor analysis that encompasses individual differences) revealed two latent affective factors- arousal and valence -which mediated lower level correspondences in music-to-color associations. Participants thus appear to match music to colors primarily in terms of common, mediating emotional associations.

Music-colour synaesthesia: Concept, context and qualia

This review provides a commentary on coloured-hearing arising on hearing music: music-colour synaesthesia. Although traditionally explained by the hyperconnectivity theory (Ramachandran & Hubbard, 2001a) and the disinhibited feedback theory (Grossenbacher & Lovelace, 2001) as a purely perceptual phenomenon, the review of eight coloured-hearing neuroimaging studies shows that it may not be assumed that these explanations are directly translatable to music-colour synaesthesia. The concept of 'ideaesthesia' (Nikolić, 2009) and the role of conceptual and semantic inducers challenge the likelihood of a single mechanism underlying the cause of synaesthesia and argue for a move away from a purely sensory to sensory explanation. Finally, music-colour synaesthesia forms a challenge for established philosophical theories and the position of synaesthesia is considered within the larger context of musical qualia.

Magnetic resonance imaging does not reveal structural alterations in the brain of grapheme-color synesthetes

Several publications have reported structural changes in the brain of synesthetes compared to controls, either local differences or differences in connectivity. In the present study, we pursued this quest for structural brain differences that might support the subjective experience of synesthesia. In particular, for the first time in this field, we investigated brain folding in comparing 45 sulcal shapes in each hemisphere of control and grapheme-color synesthete populations. To overcome flaws relative to data interpretation based only on p-values, common in the synesthesia literature, we report confidence intervals of effect sizes. Moreover, our statistical maps are displayed without introducing the classical, but misleading, p-value level threshold. We adopt such a methodological procedure to facilitate appropriate data interpretation and promote the "New Statistics" approach. Based on structural or diffusion magnetic resonance imaging data, we did not find any strong cerebral anomaly, in sulci, tissue volume, tissue density or fiber organization that could support synesthetic color experience. Finally, by sharing our complete datasets, we strongly support the multi-center construction of a sufficient large dataset repository for detecting, if any, subtle brain differences that may help understanding how a subjective experience, such as synesthesia, is mentally constructed.

Top-down signal transmission and global hyperconnectivity in auditory-visual synesthesia: Evidence from a functional EEG resting-state study

Auditory-visual (AV) synesthesia is a rare phenomenon in which an auditory stimulus induces a "concurrent" color sensation. Current neurophysiological models of synesthesia mainly hypothesize "hyperconnected" and "hyperactivated" brains, but differ in the directionality of signal transmission. The two-stage model proposes bottom-up signal transmission from inducer- to concurrent- to higher-order brain areas, whereas the disinhibited feedback model postulates top-down signal transmission from inducer- to higher-order- to concurrent brain areas. To test the different models of synesthesia, we estimated local current density, directed and undirected connectivity patterns in the intracranial space during 2 min of resting-state (RS) EEG in 11 AV synesthetes and 11 nonsynesthetes. AV synesthetes demonstrated increased parietal theta, alpha, and lower beta current density compared to nonsynesthetes. Furthermore, AV synesthetes were characterized by increased top-down signal transmission from the superior parietal lobe to the left color processing area V4 in the upper beta frequency band. Analyses of undirected connectivity revealed a global, synesthesia-specific hyperconnectivity in the alpha frequency band. The involvement of the superior parietal lobe even during rest is a strong indicator for its key role in AV synesthesia. By demonstrating top-down signal transmission in AV synesthetes, we provide direct support for the disinhibited feedback model of synesthesia. Finally, we suggest that synesthesia is a consequence of global hyperconnectivity. Hum Brain Mapp 39:522-531, 2018. © 2017 Wiley Periodicals, Inc.

Decreased integrity of the fronto-temporal fibers of the left inferior occipito-frontal fasciculus associated with auditory verbal hallucinations in schizophrenia

Auditory verbal hallucinations (AVH) have been proposed to result from altered connectivity between frontal speech production regions and temporal speech perception regions. Whilst the dorsal language pathway, serviced by the arcuate fasciculus, has been extensively studied in relation to AVH, the ventral language pathway, serviced by the inferior occipito-frontal fasciculus (IOFF) has been rarely studied in relation to AVH. This study examined whether structural changes in anatomically defined subregions of the IOFF were associated with AVH in patients with schizophrenia. Diffusion tensor imaging scans and clinical data were obtained from the Australian Schizophrenia Research Bank for 113 schizophrenia patients, of whom 39 had lifetime experience of AVH (18 had current AVH, 21 had remitted AVH), 74 had no lifetime experience of AVH, and 40 healthy controls. Schizophrenia patients with a lifetime experience of AVH exhibited reduced fractional anisotropy (FA) in the fronto-temporal fibers of the left IOFF compared to both healthy controls and schizophrenia patients without AVH. In contrast, structural abnormalities in the temporal and occipital regions of the IOFF were observed bilaterally in both patient groups, relative to the healthy controls. These results suggest that while changes in the structural integrity of the bilateral IOFF are associated with schizophrenia per se, integrity reductions in the fronto-temporal fibers of the left IOFF may be specifically associated with AVH.

Is Synaesthesia More Prevalent in Autism Spectrum Conditions? Only Where There Is Prodigious Talent

Savant syndrome is a condition where prodigious talent co-occurs with developmental difficulties such as autism spectrum conditions (ASC). To better understand savant skills, we previously proposed a link with synaesthesia: that savant syndrome may arise in ASC individuals who also happen to have synaesthesia. A second, unrelated claim is that people with autism may have higher rates of synaesthesia. Here we ask whether synaesthesia is indeed found more often in autism per se, or only in cases where autism co-occurs with savant skills. People with autism in previous studies when tested for synaesthesia were not differentiated into those with and without savant abilities. Here we tested three groups: people with autism who also have savant skills (), people with autism without savant skills (), and controls without autism (). We used a validated test to diagnose grapheme–colour synaesthesia. Results show a significantly higher prevalence of synaesthesia in people with ASC, but only those who also have savant skills. This suggests that synaesthesia in autism is linked to those with savant abilities rather than autism per se. We discuss the role of synaesthesia in the development of prodigious talent.

Synesthesia strengthens sound-symbolic cross-modal correspondences

Synesthesia is a phenomenon in which an experience in one domain is accompanied by an involuntary secondary experience in another, unrelated domain; in classical synesthesia, these associations are arbitrary and idiosyncratic. Cross-modal correspondences refer to universal associations between seemingly unrelated sensory features, e.g., auditory pitch and visual size. Some argue that these phenomena form a continuum, with classical synesthesia being an exaggeration of universal cross-modal correspondences, whereas others contend that the two are quite different, since cross-modal correspondences are non-arbitrary, non-idiosyncratic, and do not involve secondary experiences. Here, we used the implicit association test to compare synesthetes' and non-synesthetes' sensitivity to cross-modal correspondences. We tested the associations between auditory pitch and visual elevation, auditory pitch and visual size, and sound-symbolic correspondences between auditory pseudowords and visual shapes. Synesthetes were more sensitive than non-synesthetes to cross-modal correspondences involving sound-symbolic, but not low-level sensory, associations. We conclude that synesthesia heightens universally experienced cross-modal correspondences, but only when these involve sound symbolism. This is only partly consistent with the idea of a continuum between synesthesia and cross-modal correspondences, but accords with the idea that synesthesia is a high-level, post-perceptual phenomenon, with spillover of the abilities of synesthetes into domains outside their synesthesias. To our knowledge, this is the first demonstration that synesthetes, relative to non-synesthetes, experience stronger cross-modal correspondences outside their synesthetic domains.

The Merit of Synesthesia for Consciousness Research

Synesthesia is a phenomenon in which additional perceptual experiences are elicited by sensory stimuli or cognitive concepts. Synesthetes possess a unique type of phenomenal experiences not directly triggered by sensory stimulation. Therefore, for better understanding of consciousness it is relevant to identify the mental and physiological processes that subserve synesthetic experience. In the present work we suggest several reasons why synesthesia has merit for research on consciousness. We first review the research on the dynamic and rapidly growing field of the studies of synesthesia. We particularly draw attention to the role of semantics in synesthesia, which is important for establishing synesthetic associations in the brain. We then propose that the interplay between semantics and sensory input in synesthesia can be helpful for the study of the neural correlates of consciousness, especially when making use of ambiguous stimuli for inducing synesthesia. Finally, synesthesia-related alterations of brain networks and functional connectivity can be of merit for the study of consciousness.

Hearing colors: an example of brain plasticity

Sensory substitution devices (SSDs) are providing new ways for improving or replacing sensory abilities that have been lost due to disease or injury, and at the same time offer unprecedented opportunities to address how the nervous system could lead to an augmentation of its capacities. In this work we have evaluated a color-blind subject using a new visual-to-auditory SSD device called “Eyeborg”, that allows colors to be perceived as sounds. We used a combination of neuroimaging techniques including Functional Magnetic Resonance Imaging (fMRI), Diffusion Tensor Imaging (DTI) and proton Magnetic Resonance Spectroscopy (¹H-MRS) to study potential brain plasticity in this subject. Our results suggest that after 8 years of continuous use of this device there could be significant adaptive and compensatory changes within the brain. In particular, we found changes in functional neural patterns, structural connectivity and cortical topography at the visual and auditive cortex of the Eyeborg user in comparison with a control population. Although at the moment we cannot claim that the continuous use of the Eyeborg is the only reason for these findings, our results may shed further light on potential brain changes associated with the use of other SSDs. This could help to better understand how the brain adapts to several pathologies and uncover adaptive resources such as cross-modal representations. We expect that the precise understanding of these changes will have clear implications for rehabilitative training, device development and for more efficient programs for people with disabilities.

A critical review of the neuroimaging literature on synesthesia

Synesthesia refers to additional sensations experienced by some people for specific stimulations, such as the systematic arbitrary association of colors to letters for the most studied type. Here, we review all the studies (based mostly on functional and structural magnetic resonance imaging) that have searched for the neural correlates of this subjective experience, as well as structural differences related to synesthesia. Most differences claimed for synesthetes are unsupported, due mainly to low statistical power, statistical errors, and methodological limitations. Our critical review therefore casts some doubts on whether any neural correlate of the synesthetic experience has been established yet. Rather than being a neurological condition (i.e., a structural or functional brain anomaly), synesthesia could be reconsidered as a special kind of childhood memory, whose signature in the brain may be out of reach with present brain imaging techniques.

Cross-modal, bidirectional priming in grapheme-color synesthesia

Grapheme-color synesthetes perceive achromatic graphemes to be inherently colored. In this study grapheme-color synesthetes and non-synesthetes discriminated (1) the color of visual targets presented along with aurally presented digit primes, and (2) the identity of aurally presented digit targets presented with visual color primes. Reaction times to visual color targets were longer when the color of the target was incongruent with the synesthetic percept reported for the prime. Likewise, discriminating aurally presented digit targets took longer when the color of the prime was incongruent with the synesthetic percept for the target. These priming effects were absent in non-synesthetes. We conclude that binding between digits and colors in grapheme-color synesthetes can occur bidirectionally across senses. The results are in line with the idea that synesthesia is the result of linking inducing stimuli (e.g. digits) to synesthetic percepts (colors) at an abstract - supra-modal - conceptual level of processing.

Beyond visual imagery: how modality-specific is enhanced mental imagery in synesthesia?

Synesthesia based in visual modalities has been associated with reports of vivid visual imagery. We extend this finding to consider whether other forms of synesthesia are also associated with enhanced imagery, and whether this enhancement reflects the modality of synesthesia. We used self-report imagery measures across multiple sensory modalities, comparing synesthetes' responses (with a variety of forms of synesthesia) to those of non-synesthete matched controls. Synesthetes reported higher levels of visual, auditory, gustatory, olfactory and tactile imagery and a greater level of imagery use. Furthermore, their reported enhanced imagery is restricted to the modalities involved in the individual's synesthesia. There was also a relationship between the number of forms of synesthesia an individual has, and the reported vividness of their imagery, highlighting the need for future research to consider the impact of multiple forms of synesthesia. We also recommend the use of behavioral measures to validate these self-report findings.

Linking neocortical, cognitive, and genetic variability in autism with alterations of brain plasticity: the Trigger-Threshold-Target model

The phenotype of autism involves heterogeneous adaptive traits (strengths vs. disabilities), different domains of alterations (social vs. non-social), and various associated genetic conditions (syndromic vs. nonsyndromic autism). Three observations suggest that alterations in experience-dependent plasticity are an etiological factor in autism: (1) the main cognitive domains enhanced in autism are controlled by the most plastic cortical brain regions, the multimodal association cortices; (2) autism and sensory deprivation share several features of cortical and functional reorganization; and (3) genetic mutations and/or environmental insults involved in autism all appear to affect developmental synaptic plasticity, and mostly lead to its upregulation. We present the Trigger-Threshold-Target (TTT) model of autism to organize these findings. In this model, genetic mutations trigger brain reorganization in individuals with a low plasticity threshold, mostly within regions sensitive to cortical reallocations. These changes account for the cognitive enhancements and reduced social expertise associated with autism. Enhanced but normal plasticity may underlie non-syndromic autism, whereas syndromic autism may occur when a triggering mutation or event produces an altered plastic reaction, also resulting in intellectual disability and dysmorphism in addition to autism. Differences in the target of brain reorganization (perceptual vs. language regions) account for the main autistic subgroups. In light of this model, future research should investigate how individual and sex-related differences in synaptic/regional brain plasticity influence the occurrence of autism.

Not on speaking terms: hallucinations and structural network disconnectivity in schizophrenia

Auditory verbal hallucinations (AVH) in schizophrenia have previously been associated with functional deficiencies in language networks, specifically with functional disconnectivity in fronto-temporal connections in the left hemisphere and in interhemispheric connections between frontal regions. Here, we investigate whether AVH are accompanied by white matter abnormalities in tracts connecting the frontal, parietal and temporal lobes, also engaged during language tasks. We combined diffusion tensor imaging with tract-based spatial statistics and found white matter abnormalities in patients with schizophrenia as compared with healthy controls. The patients showed reduced fractional anisotropy bilaterally: in the anterior thalamic radiation (ATR), body of the corpus callosum (forceps minor), cingulum, temporal part of the superior longitudinal fasciculus (SLF) and a small area in the inferior fronto-occipital fasciculus (IFOF); and in the right hemisphere: in the visual cortex, forceps major, body of the corpus callosum (posterior parts) and inferior parietal cortex. Compared to patients without current hallucinations, patients with hallucinations revealed decreased fractional anisotropy in the left IFOF, uncinate fasciculus, arcuate fasciculus with SLF, corpus callosum (posterior parts-forceps major), cingulate, corticospinal tract and ATR. The severity of hallucinations correlated negatively with white matter integrity in tracts connecting the left frontal lobe with temporal regions (uncinate fasciculus, IFOF, cingulum, arcuate fasciculus anterior and long part and superior long fasciculus frontal part) and in interhemispheric connections (anterior corona radiata). These findings support the hypothesis that hallucinations in schizophrenia are accompanied by a complex pattern of white matter alterations that negatively affect the language, emotion and attention/perception networks.

Combined structural and functional imaging reveals cortical deactivations in grapheme-color synaesthesia

Synaesthesia is a heritable condition in which particular stimuli generate specific and consistent sensory percepts or associations in another modality or processing stream. Functional neuroimaging studies have identified potential correlates of these experiences, including, in some but not all cases, the hyperactivation of visuotemporal areas and of parietal areas thought to be involved in perceptual binding. Structural studies have identified a similarly variable spectrum of differences between synaesthetes and controls. However, it remains unclear the extent to which these neural correlates reflect the synaesthetic experience itself or additional phenotypes associated with the condition. Here, we acquired both structural and functional neuroimaging data comparing thirteen grapheme-color synaesthetes with eleven non-synaesthetes. Using voxel-based morphometry and diffusion tensor imaging, we identify a number of clusters of increased volume of gray matter, of white matter or of increased fractional anisotropy in synaesthetes vs. controls. To assess the possible involvement of these areas in the synaesthetic experience, we used nine areas of increased gray matter volume as regions of interest in an fMRI experiment that characterized the contrast in response to stimuli which induced synaesthesia (i.e., letters) vs. those which did not (non-meaningful symbols). Four of these areas showed sensitivity to this contrast in synaesthetes but not controls. Unexpectedly, in two of them, in left lateral occipital cortex and in postcentral gyrus, the letter stimuli produced a strong negative BOLD signal in synaesthetes. An additional whole-brain fMRI analysis identified 14 areas, three of which were driven mainly by a negative BOLD response to letters in synaesthetes. Our findings suggest that cortical deactivations may be involved in the conscious experience of internally generated synaesthetic percepts.

Serotonergic hyperactivity as a potential factor in developmental, acquired and drug-induced synesthesia

Though synesthesia research has seen a huge growth in recent decades, and tremendous progress has been made in terms of understanding the mechanism and cause of synesthesia, we are still left mostly in the dark when it comes to the mechanistic commonalities (if any) among developmental, acquired and drug-induced synesthesia. We know that many forms of synesthesia involve aberrant structural or functional brain connectivity. Proposed mechanisms include direct projection and disinhibited feedback mechanisms, in which information from two otherwise structurally or functionally separate brain regions mix. We also know that synesthesia sometimes runs in families. However, it is unclear what causes its onset. Studies of psychedelic drugs, such as psilocybin, LSD and mescaline, reveal that exposure to these drugs can induce synesthesia. One neurotransmitter suspected to be central to the perceptual changes is serotonin. Excessive serotonin in the brain may cause many of the characteristics of psychedelic intoxication. Excessive serotonin levels may also play a role in synesthesia acquired after brain injury. In brain injury sudden cell death floods local brain regions with serotonin and glutamate. This neurotransmitter flooding could perhaps result in unusual feature binding. Finally, developmental synesthesia that occurs in individuals with autism may be a result of alterations in the serotonergic system, leading to a blockage of regular gating mechanisms. I conclude on these grounds that one commonality among at least some cases of acquired, developmental and drug-induced synesthesia may be the presence of excessive levels of serotonin, which increases the excitability and connectedness of sensory brain regions.

Misophonia: physiological investigations and case descriptions

Misophonia is a relatively unexplored chronic condition in which a person experiences autonomic arousal (analogous to an involuntary "fight-or-flight" response) to certain innocuous or repetitive sounds such as chewing, pen clicking, and lip smacking. Misophonics report anxiety, panic, and rage when exposed to trigger sounds, compromising their ability to complete everyday tasks and engage in healthy and normal social interactions. Across two experiments, we measured behavioral and physiological characteristics of the condition. Interviews (Experiment 1) with misophonics showed that the most problematic sounds are generally related to other people's behavior (pen clicking, chewing sounds). Misophonics are however not bothered when they produce these "trigger" sounds themselves, and some report mimicry as a coping strategy. Next, (Experiment 2) we tested the hypothesis that misophonics' subjective experiences evoke an anomalous physiological response to certain auditory stimuli. Misophonic individuals showed heightened ratings and skin conductance responses (SCRs) to auditory, but not visual stimuli, relative to a group of typically developed controls, supporting this general viewpoint and indicating that misophonia is a disorder that produces distinct autonomic effects not seen in typically developed individuals.