Coordinated neural, behavioral, and phenomenological changes in perceptual plasticity through overtraining of synesthetic associations

Synesthetic Correspondence: An Overview

Intramodal and cross-modal perceptual grouping based on the spatial proximity and temporal closeness between multiple sensory stimuli, as an operational principle has built a coherent and meaningful representation of the multisensory event/object. To implement and investigate the cross-modal perceptual grouping, researchers have employed excellent paradigms of spatial/temporal ventriloquism and cross-modal dynamic capture and have revealed the conditional constraints as well as the functional facilitations among various correspondence of sensory properties, with featured behavioral evidence, computational framework as well as brain oscillation patterns. Typically, synesthetic correspondence as a special type of cross-modal correspondence can shape the efficiency and effect-size of cross-modal interaction. For example, factors such as pitch/loudness in the auditory dimension with size/brightness in the visual dimension could modulate the strength of the cross-modal temporal capture. The empirical behavioral findings, as well as psychophysical and neurophysiological evidence to address the cross-modal perceptual grouping and synesthetic correspondence, were summarized in this review. Finally, the potential applications (such as artificial synesthesia device) and how synesthetic correspondence interface with semantics (sensory linguistics), as well as the promising research questions in this field have been discussed.

The chemical induction of synaesthesia

OBJECTIVE

Preliminary research suggests that experiences resembling synaesthesia are frequently reported under the influence of a diverse range of chemical substances although the incidence, chemical specificity, and characteristics of these effects are poorly understood.

METHODS

Here we surveyed recreational drug users and self-reported developmental synaesthetes regarding their use of 28 psychoactive drugs from 12 different drug classes and whether they had experienced synaesthesia under the influence of these substances.

RESULTS

The drug class of tryptamines exhibited the highest incidence rates of drug-induced synaesthesia in controls and induction rates of novel forms of synaesthesia in developmental synaesthetes. Induction incidence rates in controls were strongly correlated with the corresponding induction and enhancement rates in developmental synaesthetes. In addition, the use of lysergic acid diethylamide (LSD) was the strongest predictor of drug-induced synaesthesia in both controls and developmental synaesthetes. Clear evidence was observed for a clustering of synaesthesia-induction rates as a function of drug class in both groups, denoting non-random incidence rates within drug classes. Sound-colour synaesthesia was the most commonly observed type of induced synaesthesia. Further analyses suggest the presence of synaesthesia-prone individuals, who were more likely to experience drug-induced synaesthesia with multiple drugs.

CONCLUSIONS

These data corroborate the hypothesized link between drug-induced synaesthesia and serotoninergic activity, but also suggest the possibility of alternative neurochemical pathways involved in the induction of synaesthesia. They further imply that the induction and modulation of synaesthesia in controls and developmental synaesthetes share overlapping mechanisms and that certain individuals may be more susceptible to experiencing induced synaesthesia with different drugs.

Cognitive and brain correlates of acquired number-colour synaesthetic-like associations

Synaesthesia is a condition in which one sensory dimension triggers another sensation. The exact contribution of genetic and environmental factors in synaesthesia is not yet fully understood. Most synaesthesia phenotypes involve associations in which the synaesthetic inducer constitutes some form of linguistic/conceptual information acquired during the course of development (e.g., digits, letters, and names of months). To study the role of learning in synaesthesia, we induced synaesthesia by training and examined the behavioural and brain correlates of number-colour associations. We took advantage of the well-known behavioural and neural signature of numbers and used number symbols as inducers. Short (2 weeks) and long (4 weeks) training protocols were conducted with two different groups. Task-related BOLD response was acquired while participants performed Stroop tasks requiring naming colours while ignoring the stimuli (i.e., number symbols, dots, words). If the arbitrary association involving number-colour is automatic, the irrelevant dimension (i.e., numbers) would interfere with the colour response. In addition, if number-colour associations are transferred to linguistic and non-symbolic representations, the passive viewing of stimuli (i.e., words and dots) would disrupt colour naming. Behavioural findings showed automatic associations as both training protocols elicited reliable congruency effects for all stimulus dimensions. Congruency effects following both training protocols produced reliable brain activations in various cortical sites involved in number and in cognitive control. The behavioural and brain patterns reported here support the role of learning in the brain correlates of developmental synaesthesia and provide the first evidence that automatic associations involving different magnitude dimensions can be acquired.

Enhanced perception and memory: Insights from synesthesia and expertise

Memory research has identified many strategies to enhance memory. However, natural foundations of enhanced memory are vastly underexplored. Interestingly, numerous studies show that synesthesia is associated with enhanced memory performance. Although it has been hypothesized for years that wider changes in visual perception are closely linked with enhanced memory functions in synesthesia, the hypothesis has never been directly put to the test. Here, we investigated whether visual perceptual abilities in synesthesia are linked with higher memory performance by comparing synesthetes who experience colors for letters with non-synesthetic color experts and non-synesthetic individuals from the more general population. Our results showed that synesthesia and expertise share a common profile of enhanced visual perceptual ability and memory in contrast to non-synesthetic individuals from the more general population. Overall, our findings suggest that visual perception and visual memory are more closely connected than previously thought.

The time course of synaesthetic colour perception

Grapheme-colour synaesthesia is a neurodevelopmental condition wherein perception of numbers and letters consistently and involuntarily elicits concurrent experiences of colour photisms. Accumulating evidence suggests that heterogeneity in the visuospatial phenomenology of synaesthesia is attributable to the operation of top-down processes underlying photisms experienced as representations in associator synaesthetes and bottom-up processes subserving photisms experienced as spatially localized in projector synaesthetes. An untested corollary of this hypothesis is that bottom-up mechanisms will actuate earlier photism perception in projector than associator synaesthetes. We tested this prediction in a pre-registered study wherein associators and projectors completed adaptive temporal order judgement tasks for graphemes, colours, and photisms. In corroboration of the hypothesis of differential photism access across subtypes, projectors displayed lower photism colour thresholds than associators whereas the two subtypes did not significantly differ in veridical colour thresholds. Synaesthetes did not differ in grapheme or colour thresholds relative to non-synaesthete controls. These results are consistent with the proposal of differential neural mechanisms underlying photism perception in subtypes of grapheme-colour synaesthesia and warrant renewed attention to heterogeneity in the mechanisms and phenomenology of this condition.

Hearing what you see: Distinct excitatory and disinhibitory mechanisms contribute to visually-evoked auditory sensations

Visual motion or flashing lights can evoke auditory sensations in some people. This large-scale internet study aimed to validate a combined subjective/objective test of the genuineness of this putative form of synaesthesia (visually-evoked auditory response, vEAR). Correlations were measured between each individual's ratings of the vividness of auditory sensations evoked by a series of looping videos, and measurement of the videos' physical low-level motion energy, calculated using Adelson and Bergen's (1985) computational model of low-level visual motion processing. The strength of this association for each individual provided a test of how strongly subjective vEAR was driven by objective motion energy ('ME-sensitivity'). A second aim was to infer whether vEAR depends on cortical excitation and/or disinhibition of early visual and/or auditory brain areas. To achieve this, correlations were measured between the above vEAR measures and visual contrast surround-suppression, which is thought to index lateral inhibition in the early visual system. As predicted by a disinhibition account of vEAR, video ratings were overall higher in individuals showing weaker surround-suppression. Interestingly, surround-suppression and ME-sensitivity did not correlate. Additionally, both surround-suppression and ME-sensitivity each independently predicted different clusters of trait measures selected for their possible association with cortical excitability and/or disinhibition: Surround-suppression was associated with vEAR self-ratings and auditory-evoked visual phosphenes, while ME-sensitivity was independently associated with ratings of other traits including susceptibility to migraine and pattern glare. Altogether, these results suggest there are two independent mechanisms underlying vEAR and its associated traits, based putatively on cortical disinhibition versus excitability.

Neurophenomenology of induced and natural synaesthesia

People with synaesthesia have additional perceptual experiences, which are automatically and consistently triggered by specific inducing stimuli. Synaesthesia therefore offers a unique window into the neurocognitive mechanisms underlying conscious perception. A long-standing question in synaesthesia research is whether it is possible to artificially induce non-synaesthetic individuals to have synaesthesia-like experiences. Although synaesthesia is widely considered a congenital condition, increasing evidence points to the potential of a variety of approaches to induce synaesthesia-like experiences, even in adulthood. Here, we summarize a range of methods for artificially inducing synaesthesia-like experiences, comparing the resulting experiences to the key hallmarks of natural synaesthesia which include consistency, automaticity and a lack of 'perceptual presence'. We conclude that a number of aspects of synaesthesia can be artificially induced in non-synaesthetes. These data suggest the involvement of developmental and/or learning components in the acquisition of synaesthesia, and they extend previous reports of perceptual plasticity leading to dramatic changes in perceptual phenomenology in adults. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

How non-veridical perception drives actions in healthy humans: evidence from synaesthesia

We continually perform actions that are driven by our perception and it is a commonly held view that only objectively perceived changes within the 'real' world affect behaviour. Exceptions are generally only made for mental health disorders associated with delusions and hallucinations where behaviour may be triggered by the experience of objectively non-existent percepts. Here, we demonstrate, using synaesthesia as a model condition (in N = 19 grapheme-colour synaesthetes), how objectively non-existent (i.e. non-veridical) but still non-pathological perceptions affect actions in healthy humans. Using electroencephalography, we determine whether early-stage perceptual processes (reflected by P1 and N1 event-related potential (ERP) components), or late-stage-integration processes (reflected by N2 component), underlie the effects of non-veridical perceptions on action control. ERP analysis suggests that even though the examined peculiarities and experimental variations are perceptual in nature, it is not early-stage perceptual processes, but rather higher-order executive control processes linking perceptions to the appropriate motor response underlying this effect. Source localization analysis implicates activation within medial frontal cortices in the effect of how irrelevant non-veridical perceptions modulate behaviour. Our results challenge common conceptions about the determinants of human behaviour but can be explained by well-established theoretical frameworks detailing the link between perception and action. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Synaesthesia: a distinct entity that is an emergent feature of adaptive neurocognitive differences

In this article, I argue that synaesthesia is not on a continuum with neurotypical cognition. Synaesthesia is special: its phenomenology is different; it has distinct causal mechanisms; and is likely to be associated with a distinct neurocognitive profile. However, not all synaesthetes are the same, and there are quantifiable differences between them. In particular, the number of types of synaesthesia that a person possesses is a hitherto underappreciated variable that predicts cognitive differences along a number of dimensions (mental imagery, sensory sensitivity, attention to detail). Together with enhanced memory, this may constitute a common core of abilities that may go some way to explaining why synaesthesia might have evolved. I argue that the direct benefits of synaesthesia are generally limited (i.e. the synaesthetic associations do not convey novel information about the world) but, nevertheless, synaesthesia may develop due to other adaptive functions (e.g. perceptual ability, memory) that necessitate changes to design features of the brain. The article concludes by suggesting that synaesthesia forces us to reconsider what we mean by a 'normal' mind/brain. There may be multiple 'normal' neurodevelopmental trajectories that can sculpt very different ways of experiencing the world, of which synaesthesia is but one. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Synaesthetic colour associations for Japanese Kanji characters: from the perspective of grapheme learning

One of the fundamental questions about grapheme-colour synaesthesia is how specific associations between the graphemes and colours are formed. We addressed this question by focusing on the determinants of synaesthetic colours for Japanese Kanji characters (logographic characters) using a psycholinguistic approach. Study 1 explored the influence meaning has on synaesthetic colours for Kanji characters representing abstract meanings by examining synaesthetic colours for antonym pairs (i.e. characters with meanings opposed to each other) in Japanese synaesthetes. Results showed that semantic relations influenced the grapheme-colour associations for characters representing abstract meanings in the early stages of learning abstract Kanji, while the influence was reduced in the grapheme-colour associations for those learned later. Study 2 examined the effect that learning new sounds or meanings of graphemes has on synaesthetic colours for those graphemes. Japanese synaesthetes were taught new sounds or new meanings for familiar Kanji characters. Results indicated that acquiring new information for graphemes slightly but significantly reduced the test-retest grapheme-colour association consistency, suggesting that synaesthetic colours can be modulated to reflect the synaesthete's latest knowledge about graphemes. Implications of these findings are discussed from the perspective of the relationship between synaesthesia and grapheme learning. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Inducing synesthesia in non-synesthetes: Short-term visual deprivation facilitates auditory-evoked visual percepts

Sounds can modulate activity in visual cortex, facilitating the detection of visual targets. However, these sound-driven modulations are not thought to evoke conscious visual percepts in the general population. In individuals with synesthesia, however, multisensory interactions do lead to qualitatively different experiences such as sounds evoking flashes of light. Why, if multisensory interactions are present in all individuals, do only synesthetes experience abnormal qualia? Competing models differ in the time required for synesthetic experiences to emerge. The cross-activation model suggests synesthesia arises over months or years from the development of abnormal neural connections. Here we demonstrate that after ∼5 min of visual deprivation, sounds can evoke synesthesia-like percepts (vivid colors and Klüver form-constants) in ∼50% of non-synesthetes. These results challenge aspects of the cross-activation model and suggest that synesthesia exists as a latent feature in all individuals, manifesting when the balance of activity across the senses has been altered.

Gray Bananas and a Red Letter A - From Synesthetic Sensation to Memory Colors

Grapheme-color synesthesia is a condition in which objectively achromatic graphemes induce concurrent color experiences. While it was long thought that the colors emerge during perception, there is growing support for the view that colors are integral to synesthetes' cognitive representations of graphemes. In this work, we review evidence for two opposing theories positing either a perceptual or cognitive origin of concurrent colors: the cross-activation theory and the conceptual-mediation model. The review covers results on inducer and concurrent color processing as well as findings concerning the brain structure and grapheme-color mappings in synesthetes and trained mappings in nonsynesthetes. The results support different aspects of both theories. Finally, we discuss how research on memory colors could provide a new perspective in the debate about the level of processing at which the synesthetic colors occur.