Prevalence of learned grapheme-color pairings in a large online sample of synesthetes

The mechanisms underlying conditioning of phantom percepts differ between those with hallucinations and synesthesia

There are many different kinds of 'phantom' percepts but it is unknown whether they are united by common mechanisms. For example, synaesthesia (e.g., numbers evoking colour) and hallucinations appear conceptually and phenomenologically similar: both result in a percept that does not have an environmental correlate. Here, people with synaesthesia (n = 66) performed a conditioned hallucinations paradigm known to be sensitive to hallucination susceptibility, and we asked whether synaesthetes would show the same behavioural profile as hallucinators in this task. Repeated pairing of checkerboards with tones, and gratings with colours encourages the participant to draw on prior knowledge when asked to report on the presence of the difficult-to-detect target stimulus. Synaesthetes show increased modelled expectancies for the stimulus association across the board, resulting in a higher number of detections at all stimulus intensities. This is in contrast to the pattern observed in hallucinators, who weigh their prior beliefs more strongly than controls, giving rise to more conditioned hallucinations. Results indicate that fundamentally different perceptual processes may be at the core of these seemingly similar experiences.

Shape-color associations in an unrestricted color choice paradigm

Since Kandinsky's claim for fundamental shape-color associations, several studies have revealed that those tendencies were not generalizable to the entire population and that different associations were more prevalent. Past studies, however, lacked a methodology that allowed participants to freely report their shape-color preferences. Here, we report data from 7,517 Danish individuals, using a free choice full color wheel for five different geometrical shapes. We find significant shape-hue associations for circle-red/yellow, triangle-green/yellow, square-blue, and pentagon/hexagon-magenta. The significant shape-hue associations are also more saturated than non-significant ones for the circle, triangle, and square. At the conceptual level, basic shapes, which show stronger associations, are linked to primary colors, and non-basic shapes to secondary colors. Shape-color associations seem indeed to follow the Berlin-Kay stages of entry into languages. This pattern had previously been described for graphemes and weekday-color associations. The methodology employed in our study can be repeated in different cultural contexts in the future. We also provide another instance of color associations for ordinal concepts that follow the stages of entry into languages.

Perception of color in primates: A conceptual color neurons hypothesis

Perception of color by humans and other primates is a complex problem, studied by neurophysiology, psychophysiology, psycholinguistics, and even philosophy. Being mostly trichromats, simian primates have three types of opsin proteins, expressed in cone neurons in the eye, which allow for the sensing of color as the physical wavelength of light. Further, in neural networks of the retina, the coding principle changes from three types of sensor proteins to two opponent channels: activity of one type of neuron encode the evolutionarily ancient blue-yellow axis of color stimuli, and another more recent evolutionary channel, encoding the axis of red-green color stimuli. Both color channels are distinctive in neural organization at all levels from the eye to the neocortex, where it is thought that the perception of color (as philosophical qualia) emerges from the activity of some neuron ensembles. Here, using data from neurophysiology as a starting point, we propose a hypothesis on how the perception of color can be encoded in the activity of certain neurons in the neocortex. These conceptual neurons, herein referred to as 'color neurons', code only the hue of the color of visual stimulus, similar to place cells and number neurons, already described in primate brains. A case study with preliminary, but direct, evidence for existing conceptual color neurons in the human brain was published in 2008. We predict that the upcoming studies in non-human primates will be more extensive and provide a more detailed description of conceptual color neurons.

Face-Processing Differences Present in Grapheme-Color Synesthetes

Grapheme‐color synesthesia is a heterogeneous neurological phenomenon whereby the experience of a grapheme automatically and involuntarily elicits an experience of color. While the majority of synesthesia research has focused on inducer‐specific influences of synesthetic associations, more recent efforts have examined potential broader differences. Based on spontaneous reports from synesthetes detailing problems with face recognition, in conjunction with the geographical proximity of neurological regions relevant to both synesthesia and face processing, we sought to examine whether synesthetes demonstrated atypical face‐processing abilities.

A total of 16 grapheme‐color synesthetes and 16 age‐and‐gender matched controls (±3 years) completed the Cambridge Face Memory Test (CFMT; Duchaine & Nakayama, 2006) of face memory, the Vanderbilt Holistic Face Processing Task (VHPT‐F; Richler, Floyd, & Gauthier, 2014) of holistic face processing, as well as a standardized self‐report questionnaire the Faces and Emotions Questionnaire (Freeman, Palermo, & Brock, 2015). The results revealed significantly poorer performance in synesthete's ability to recognize faces in the CFMT that was driven by a reduction in upright advantage. Results also revealed a significant reduction in overall accuracy on the VHPT‐F for synesthetes, who despite this displayed a comparable holistic processing advantage compared to matched controls. Finally, synesthetes also rated themselves as significantly worse at face recognition. We suggest that this pattern may reflect differences in the development of individualized perceptual strategies.

Perceptual processing links autism and synesthesia: A co-twin control study

Synesthesia occurs more commonly in individuals fulfilling criteria for an autism spectrum diagnosis than in the general population. It is associated with autistic traits and autism-related perceptual processing characteristics, including a more detail-focused attentional style and altered sensory sensitivity. In addition, these characteristics correlate with the degree of grapheme-color synesthesia (consistency of grapheme-color associations) in non-synesthetes. We investigated a predominantly non-synesthetic twin sample, including individuals fulfilling criteria for an autism spectrum diagnosis or other neurodevelopmental disorders (n = 65, 14-34 years, 60% female). We modelled linear relationships between the degree of grapheme-color synesthesia and autistic traits, sensory sensitivity, and visual perception, both within-twin pairs (22 pairs) where all factors shared by twins are implicitly controlled (including 50-100% genetics), and across the entire cohort. We found that the degree of grapheme-color synesthesia was associated with autistic traits within the domain of Attention to Details and with sensory hyper-, but not hypo-sensitivity. These associations were stronger within-twin pairs than across the sample. Further, twins with a higher degree of grapheme-color synesthesia were better than their co-twins at identifying fragmented images (Fragmented Pictures Test). This is the first twin study on the association between synesthesia and autism-related perceptual features and traits. The results suggest that investigating these associations within-twin pairs, implicitly adjusting for potential confounding factors shared by twins, is more sensitive than doing so in non-related individuals. Consistent with previous findings, the results suggest an association between the degree of grapheme-color synesthesia and autism-related perceptual features, while utilizing a different measure for sensory sensitivity. The novel finding of enhanced fragmented picture integration in twins with a higher degree of grapheme-color synesthesia challenges the view of a generally more detail-focused attentional style in synesthesia and might be related to enhanced memory or mental imagery in more synesthetic individuals.

Hearing what you see: Distinct excitatory and disinhibitory mechanisms contribute to visually-evoked auditory sensations

Visual motion or flashing lights can evoke auditory sensations in some people. This large-scale internet study aimed to validate a combined subjective/objective test of the genuineness of this putative form of synaesthesia (visually-evoked auditory response, vEAR). Correlations were measured between each individual's ratings of the vividness of auditory sensations evoked by a series of looping videos, and measurement of the videos' physical low-level motion energy, calculated using Adelson and Bergen's (1985) computational model of low-level visual motion processing. The strength of this association for each individual provided a test of how strongly subjective vEAR was driven by objective motion energy ('ME-sensitivity'). A second aim was to infer whether vEAR depends on cortical excitation and/or disinhibition of early visual and/or auditory brain areas. To achieve this, correlations were measured between the above vEAR measures and visual contrast surround-suppression, which is thought to index lateral inhibition in the early visual system. As predicted by a disinhibition account of vEAR, video ratings were overall higher in individuals showing weaker surround-suppression. Interestingly, surround-suppression and ME-sensitivity did not correlate. Additionally, both surround-suppression and ME-sensitivity each independently predicted different clusters of trait measures selected for their possible association with cortical excitability and/or disinhibition: Surround-suppression was associated with vEAR self-ratings and auditory-evoked visual phosphenes, while ME-sensitivity was independently associated with ratings of other traits including susceptibility to migraine and pattern glare. Altogether, these results suggest there are two independent mechanisms underlying vEAR and its associated traits, based putatively on cortical disinhibition versus excitability.

Reduced perceptual narrowing in synesthesia

Synesthesia is a neurologic trait in which specific inducers, such as sounds, automatically elicit additional idiosyncratic percepts, such as color (thus "colored hearing"). One explanation for this trait-and the one tested here-is that synesthesia results from unusually weak pruning of cortical synaptic hyperconnectivity during early perceptual development. We tested the prediction from this hypothesis that synesthetes would be superior at making discriminations from nonnative categories that are normally weakened by experience-dependent pruning during a critical period early in development-namely, discrimination among nonnative phonemes (Hindi retroflex /d̪a/ and dental /ɖa/), among chimpanzee faces, and among inverted human faces. Like the superiority of 6-mo-old infants over older infants, the synesthetic groups were significantly better than control groups at making all the nonnative discriminations across five samples and three testing sites. The consistent superiority of the synesthetic groups in making discriminations that are normally eliminated during infancy suggests that residual cortical connectivity in synesthesia supports changes in perception that extend beyond the specific synesthetic percepts, consistent with the incomplete pruning hypothesis.

Cross-modal associations and synesthesia: Categorical perception and structure in vowel-color mappings in a large online sample

We report associations between vowel sounds, graphemes, and colors collected online from over 1,000 Dutch speakers. We also provide open materials, including a Python implementation of the structure measure and code for a single-page web application to run simple cross-modal tasks. We also provide a full dataset of color-vowel associations from 1,164 participants, including over 200 synesthetes identified using consistency measures. Our analysis reveals salient patterns in the cross-modal associations and introduces a novel measure of isomorphism in cross-modal mappings. We found that, while the acoustic features of vowels significantly predict certain mappings (replicating prior work), both vowel phoneme category and grapheme category are even better predictors of color choice. Phoneme category is the best predictor of color choice overall, pointing to the importance of phonological representations in addition to acoustic cues. Generally, high/front vowels are lighter, more green, and more yellow than low/back vowels. Synesthetes respond more strongly on some dimensions, choosing lighter and more yellow colors for high and mid front vowels than do nonsynesthetes. We also present a novel measure of cross-modal mappings adapted from ecology, which uses a simulated distribution of mappings to measure the extent to which participants' actual mappings are structured isomorphically across modalities. Synesthetes have mappings that tend to be more structured than nonsynesthetes', and more consistent color choices across trials correlate with higher structure scores. Nevertheless, the large majority (~ 70%) of participants produce structured mappings, indicating that the capacity to make isomorphically structured mappings across distinct modalities is shared to a large extent, even if the exact nature of the mappings varies across individuals. Overall, this novel structure measure suggests a distribution of structured cross-modal association in the population, with synesthetes at one extreme and participants with unstructured associations at the other.

Echoes from the past: synaesthetic colour associations reflect childhood gender stereotypes

Grapheme-colour synaesthesia is a neurological phenomenon in which linguistic symbols evoke consistent colour sensations. Synaesthesia is believed to be influenced by both genetic and environmental factors, but how these factors interact to create specific associations in specific individuals is poorly understood. In this paper, we show that a grapheme-colour association in adult synaesthetes can be traced to a particular environmental effect at a particular moment in childhood. We propose a model in which specific grapheme-colour associations are 'locked in' during development in children predisposed to become synaesthetes, whereas grapheme-colour associations remain flexible in non-synaesthetes. We exploit Western gender-colour stereotypes to test our model: we found that young girls in general tend to associate their first initial with the colour pink. Consistent with our model, adult female synaesthetes are influenced by their childhood environment: they associate their first initial with pink. Adult female non-synaesthetes do not show this bias. Instead, in our study, non-synaesthetes tended to associate their first initial with their current favourite colour. The results thus support the 'locking in' model of synaesthesia, suggesting that synaesthetic associations can be used as a 'time capsule', revealing childhood influences on adult linguistic associations. Grapheme-colour synaesthesia may thus offer an extraordinary opportunity to study linguistic development. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Synaesthesia: a distinct entity that is an emergent feature of adaptive neurocognitive differences

In this article, I argue that synaesthesia is not on a continuum with neurotypical cognition. Synaesthesia is special: its phenomenology is different; it has distinct causal mechanisms; and is likely to be associated with a distinct neurocognitive profile. However, not all synaesthetes are the same, and there are quantifiable differences between them. In particular, the number of types of synaesthesia that a person possesses is a hitherto underappreciated variable that predicts cognitive differences along a number of dimensions (mental imagery, sensory sensitivity, attention to detail). Together with enhanced memory, this may constitute a common core of abilities that may go some way to explaining why synaesthesia might have evolved. I argue that the direct benefits of synaesthesia are generally limited (i.e. the synaesthetic associations do not convey novel information about the world) but, nevertheless, synaesthesia may develop due to other adaptive functions (e.g. perceptual ability, memory) that necessitate changes to design features of the brain. The article concludes by suggesting that synaesthesia forces us to reconsider what we mean by a 'normal' mind/brain. There may be multiple 'normal' neurodevelopmental trajectories that can sculpt very different ways of experiencing the world, of which synaesthesia is but one. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Distinct colours in the 'synaesthetic colour palette'

In grapheme-colour synaesthesia, particular linguistic elements evoke particular colour sensations. Interestingly, when asked, non-synaesthetes can also associate colours to letters, and previous studies show that specific letter-to-colour associations have similar biases to those of synaesthetes. However, it is an open question whether the colours reported by synaesthetes and non-synaesthetes differ overall: is there a 'synaesthetic colour palette'? In this study, we visualize the overall distribution in colour space of colour concurrents in grapheme-colour synaesthetes, and colour associations in non-synaesthetic controls. We confirm the existence of a synaesthetic colour palette: colour concurrents in synaesthetes are different from colour associations in non-synaesthetes. We quantify three factors that distinguish the colour palette of synaesthetes and non-synaesthetes: synaesthetes have an increased over-representation of 'pure' (unmixed) hues, an increased presence of 'warm' (yellow, orange, brown) colours, and an increased presence of achromatic (grey, white, black) colours. Furthermore, we demonstrate that differences in the synaesthetic colour palette can be used to train a machine learning algorithm to reliably classify single subjects as synaesthetes versus non-synaesthetes without using test-retest consistency data. As far as we know, this is the first time an individual could be 'diagnosed' as a synaesthete, based only on his or her colours evoked by letters. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Deepening understanding of language through synaesthesia: a call to reform and expand

In this paper, I present arguments and suggestions for the improvement of the scientific study of synaesthesia, and particularly grapheme-colour synaesthesia in relation to psycholinguistic research, although the principles I advocate can be easily adapted to any subfield of synaesthesia study. I postulate that the current state of research on synaesthesia in general, and on grapheme-colour synaesthesia in particular, suffers from a lack of exploratory evidence and essential groundwork upon which to build hypothesis-testing studies. In particular, I argue that synaesthesia research has been artificially bounded by assumptions about the nature of synaesthetic experiences, which constrain both the questions that researchers ask and the way in which they go about answering those questions. As a specific example, I detail how much of the current research on grapheme-colour synaesthesia is built to accommodate two major assumptions about the nature of colours for letters and for words-assumptions which I will contend are not universally true, and the exceptions to which point to a much richer and heterogeneous understanding of synaesthetic experience than current research practices capture. The top-down predetermination of what is important or meaningful to measure, and what is not, has subsequently impeded a full understanding of what synaesthesia is and how it works. I argue that these assumptions must be carefully addressed and evaluated, both for the particular case of grapheme-colour synaesthesia and for the field as a whole, to move towards a holistic and fruitful understanding of synaesthesia as a phenomenon and as a tool to study language, thought and perception. To that end, I propose specific recommendations for synaesthesia researchers to solidify and expand their understanding and to capture the actual experience of synaesthetes. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

A meta-analysis of memory ability in synaesthesia

People with synaesthesia (e.g., experiencing colours for letters and numbers) have been reported to possess enhanced memory relative to the general population. However, there are also inconsistencies in this literature and it is unclear whether this reflects sampling error (exacerbated by low Ns) or more meaningful differences that arise because synaesthesia relates to some aspects of memory more than others. To this end, a multi-level meta-analysis was conducted. Synaesthetes have enhanced long-term (episodic) memory with a medium population effect size ( dˆ  = 0.61), whereas the effects on working memory (short-term memory) were significantly smaller ( dˆ  = 0.36) but still exceeded that of controls. Moderation analyses suggested that, aside from the division between long-term vs. working memory, the effects of synaesthesia are pervasive, i.e., they extend to all kinds of stimuli, and extend to all kinds of test formats. This pattern is hard to reconcile with the view that synaesthetic experiences directly support memory ability: for instance, digit span (where synaesthesia could be helpful) showed a small effect whereas episodic memory for abstract images (where synaesthesia is irrelevant) yielded larger effects. Synaesthesia occupies a unique position of being the only known neurodevelopmental condition linked to a pervasive enhancement of long-term memory.

Inducing synesthesia in non-synesthetes: Short-term visual deprivation facilitates auditory-evoked visual percepts

Sounds can modulate activity in visual cortex, facilitating the detection of visual targets. However, these sound-driven modulations are not thought to evoke conscious visual percepts in the general population. In individuals with synesthesia, however, multisensory interactions do lead to qualitatively different experiences such as sounds evoking flashes of light. Why, if multisensory interactions are present in all individuals, do only synesthetes experience abnormal qualia? Competing models differ in the time required for synesthetic experiences to emerge. The cross-activation model suggests synesthesia arises over months or years from the development of abnormal neural connections. Here we demonstrate that after ∼5 min of visual deprivation, sounds can evoke synesthesia-like percepts (vivid colors and Klüver form-constants) in ∼50% of non-synesthetes. These results challenge aspects of the cross-activation model and suggest that synesthesia exists as a latent feature in all individuals, manifesting when the balance of activity across the senses has been altered.

A Critical Analysis of Colour-Shape Correspondences: Examining the Replicability of Colour-Shape Associations

Research on the topic of colour-shape correspondences started in the early 20th century with the Bauhaus artist Wassily Kandinsky. However, more recently, the topic has been examined using the empirical framework of crossmodal correspondences research. The field remains one in which consistent results and generalisable hypotheses about the existence and nature of colour-shape correspondences are lacking. The replicability and consistency of findings concerning colour-shape correspondences are examined in three online colour-shape matching experiments using the same procedure and study design while varying the sets of shape stimuli that are evaluated. Participants matched one of 36 colours to each shape as well as made preference and arousal appraisal ratings for each of the shapes and colours. The complexities of analysing colour-shape correspondence data are discussed and illustrated by classifying and analysing shape and colours in a variety of different ways, including using continuous perceptual and objective measures. Significant colour-shape associations were found. However, as hypothesised, limited consistent results in regard to what perceptual shape characteristics predicted colour choices were documented across the three stimuli sets. This was the case both within and across different analysis methods. The factors that may be responsible for these inconsistencies are critically discussed. Intriguingly, however, evidence for emotional mediation, whereby shape and colour liking and arousal appraisals appear to influence the colour-shape correspondences made by participants, was found across all three experiments.

Synesthetes perseverate in implicit learning: Evidence from a non-stationary statistical learning task

Synesthetes automatically and consistently experience additional sensory or cognitive perceptions in response to particular environmental stimuli. Recent evidence suggests that the propensity to develop synesthesia is genetic while the particular associations experienced by a given synesthete are influenced by learning. Despite the potential role of implicit learning in the formation of synesthetic associations, there has been minimal investigation of synesthetes' implicit learning abilities. In this study, we examine linguistic-colour synesthetes' ability to implicitly learn from and adjust to non-stationary statistics in a domain unrelated to their particular form of synesthesia. Engaging participants in a computer game Whack-the-mole, we utilise the online measure of reaction time to assess the time course of learning. Participants are exposed to "worlds" of probabilities that, unbeknownst to them, undergo unannounced changes, creating unpredictable statistical shifts devoid of accompanying cues. The same small set of probability worlds are repeated throughout the experiment to investigate participants' ability to retain and learn from this repetitive probabilistic information. The reaction time data provide evidence that synesthetes require more information than nonsynesthetes to benefit from the non-stationary probability distributions. These findings demonstrate that linguistic-colour synesthetes' implicit learning abilities-in a domain far from their synesthetic experiences-differ from those of nonsynesthetes.

Gray Bananas and a Red Letter A - From Synesthetic Sensation to Memory Colors

Grapheme-color synesthesia is a condition in which objectively achromatic graphemes induce concurrent color experiences. While it was long thought that the colors emerge during perception, there is growing support for the view that colors are integral to synesthetes' cognitive representations of graphemes. In this work, we review evidence for two opposing theories positing either a perceptual or cognitive origin of concurrent colors: the cross-activation theory and the conceptual-mediation model. The review covers results on inducer and concurrent color processing as well as findings concerning the brain structure and grapheme-color mappings in synesthetes and trained mappings in nonsynesthetes. The results support different aspects of both theories. Finally, we discuss how research on memory colors could provide a new perspective in the debate about the level of processing at which the synesthetic colors occur.

Sounds from seeing silent motion: Who hears them, and what looks loudest?

Some people hear what they see: car indicator lights, flashing neon shop signs, and people's movements as they walk may all trigger an auditory sensation, which we call the visual-evoked auditory response (vEAR or 'visual ear'). We have conducted the first large-scale online survey (N > 4000) of this little-known phenomenon. We analysed the prevalence of vEAR, what induces it, and what other traits are associated with it. We assessed prevalence by asking whether respondents had previously experienced vEAR. Participants then rated silent videos for vividness of evoked auditory sensations, and answered additional trait questions. Prevalence appeared higher relative to other typical synaesthesias. Prior awareness and video ratings were associated with greater frequency of other synaesthesias, including flashes evoked by sounds, and musical imagery. Higher-rated videos often depicted meaningful events that predicted sounds (e.g., collisions). However, even videos containing abstract flickering or moving patterns could also elicit higher ratings, despite having no predictable association with sounds. Such videos had higher levels of raw 'motion energy' (ME), which we quantified using a simple computational model of motion processing in early visual cortex. Critically, only respondents reporting prior awareness of vEAR tended to show a positive correlation between video ratings and ME. This specific sensitivity to ME suggests that in vEAR, signals from visual motion processing may affect audition relatively directly without requiring higher-level interpretative processes. Our other findings challenge the popular assumption that individuals with synaesthesia are rare and have ideosyncratic patterns of brain hyper-connectivity. Instead, our findings of apparently high prevalence and broad associations with other synaesthesias and traits are jointly consistent with a common dependence on normal variations in physiological mechanisms of disinhibition or excitability of sensory brain areas and their functional connectivity. The prevalence of vEAR makes it easier to test such hypotheses further, and makes the results more relevant to understanding not only synaesthetic anomalies but also normal perception.

Implicit associative learning in synesthetes and nonsynesthetes

Although cross-modal neural connections and genetic underpinnings are prominent in most current theories regarding the development of synesthesia, the potential role of associative learning in the formation of synesthetic associations has recently been revitalized. In this study, we investigated implicit associative learning in synesthetes and nonsynesthetes by recording reaction times to a target whose color was probabilistically correlated with its shape. A continuous measure of target detection at multiple time points during learning revealed that synesthetes and nonsynesthetes learn associations differently. Specifically, our results demonstrated a "fast-facilitation" learning effect for nonsynesthetes and a "fast-interference, slow-facilitation" learning effect for synesthetes. Additionally, synesthetes exhibited superior long-term memory for the learned associations in a surprise delayed retest. After this retest, participants implicitly learned new (shuffled) shape-color associations. We found that synesthetes experienced greater interference while learning these new shape-color associations. These results detail ways in which implicit associative learning and memory differ between synesthetes and nonsynesthetes.

Language use statistics and prototypical grapheme colours predict synaesthetes' and non-synaesthetes' word-colour associations

Synaesthesia is the neuropsychological phenomenon in which individuals experience unusual sensory associations, such as experiencing particular colours in response to particular words. While it was once thought the particular pairings between stimuli were arbitrary and idiosyncratic to particular synaesthetes, there is now growing evidence for a systematic psycholinguistic basis to the associations. Here we sought to assess the explanatory value of quantifiable lexical association measures (via latent semantic analysis; LSA) in the pairings observed between words and colours in synaesthesia. To test this, we had synaesthetes report the particular colours they experienced in response to given concept words, and found that language association between the concept and colour words provided highly reliable predictors of the reported pairings. These results provide convergent evidence for a psycholinguistic basis to synaesthesia, but in a novel way, showing that exposure to particular patterns of associations in language can predict the formation of particular synaesthetic lexical-colour associations. Consistent with previous research, the prototypical synaesthetic colour for the first letter of the word also played a role in shaping the colour for the whole word, and this effect also interacted with language association, such that the effect of the colour for the first letter was stronger as the association between the concept word and the colour word in language increased. Moreover, when a group of non-synaesthetes were asked what colours they associated with the concept words, they produced very similar reports to the synaesthetes that were predicted by both language association and prototypical synaesthetic colour for the first letter of the word. This points to a shared linguistic experience generating the associations for both groups.

A deafening flash! Visual interference of auditory signal detection

In some people, visual stimulation evokes auditory sensations. How prevalent and how perceptually real is this? 22% of our neurotypical adult participants responded 'Yes' when asked whether they heard faint sounds accompanying flash stimuli, and showed significantly better ability to discriminate visual 'Morse-code' sequences. This benefit might arise from an ability to recode visual signals as sounds, thus taking advantage of superior temporal acuity of audition. In support of this, those who showed better visual relative to auditory sequence discrimination also had poorer auditory detection in the presence of uninformative visual flashes, though this was independent of awareness of visually-evoked sounds. Thus a visually-evoked auditory representation may occur subliminally and disrupt detection of real auditory signals. The frequent natural correlation between visual and auditory stimuli might explain the surprising prevalence of this phenomenon. Overall, our results suggest that learned correspondences between strongly correlated modalities may provide a precursor for some synaesthetic abilities.

Sound Properties Associated With Equiluminant Colours

There is a widespread tendency to associate certain properties of sound with those of colour (e.g., higher pitches with lighter colours). Yet it is an open question how sound influences chroma or hue when properly controlling for lightness. To examine this, we asked participants to adjust physically equiluminant colours until they ‘went best’ with certain sounds. For pure tones, complex sine waves and vocal timbres, increases in frequency were associated with increases in chroma. Increasing the loudness of pure tones also increased chroma. Hue associations varied depending on the type of stimuli. In stimuli that involved only limited bands of frequencies (pure tones, vocal timbres), frequency correlated with hue, such that low frequencies gave blue hues and progressed to yellow hues at 800 Hz. Increasing the loudness of a pure tone was also associated with a shift from blue to yellow. However, for complex sounds that share the same bandwidth of frequencies (100–3200 Hz) but that vary in terms of which frequencies have the most power, all stimuli were associated with yellow hues. This suggests that the presence of high frequencies (above 800 Hz) consistently yields yellow hues. Overall we conclude that while pitch–chroma associations appear to flexibly re-apply themselves across a variety of contexts, frequencies above 800 Hz appear to produce yellow hues irrespective of context. These findings reveal new sound–colour correspondences previously obscured through not controlling for lightness. Findings are discussed in relation to understanding the underlying rules of cross-modal correspondences, synaesthesia, and optimising the sensory substitution of visual information through sound.

The prevalence of synaesthesia depends on early language learning

According to one theory, synaesthesia develops, or is preserved, because it helps children learn. If so, it should be more common among adults who faced greater childhood learning challenges. In the largest survey of synaesthesia to date, the incidence of synaesthesia was compared among native speakers of languages with transparent (easier) and opaque (more difficult) orthographies. Contrary to our prediction, native speakers of Czech (transparent) were more likely to be synaesthetes than native speakers of English (opaque). However, exploratory analyses suggested that this was because more Czechs learned non-native second languages, which was strongly associated with synaesthesia, consistent with the learning hypothesis. Furthermore, the incidence of synaesthesia among speakers of opaque languages was double that among speakers of transparent languages other than Czech, also consistent with the learning hypothesis. These findings contribute to an emerging understanding of synaesthetic development as a complex and lengthy process with multiple causal influences.

Explicit Associative Learning and Memory in Synesthetes and Nonsynesthetes

Most current theories regarding the development of synesthesia focus on cross-modal neural connections and genetic underpinnings, but recent evidence has revitalized the potential role of associative learning. In the present study, we compared synesthetes’ and controls’ ability to explicitly learn shape-color pairings. Using a continuous measure of accuracy and multiple testing blocks, we found that synesthetes learned these pairings faster than controls. In a delayed retest, synesthetes outperformed controls, demonstrating enhanced long-term memory for shape–color associations. Following this retest, participants learned shuffled associations, and we found little evidence for group differences in subsequent learning ability. Overall, our findings support the hypothesis that synesthetes have exceptional associative learning abilities and further specify that this advantage pertains to the initial learning rate and long-term retention of associations.

The Emergence of Synaesthesia in a Neuronal Network Model via Changes in Perceptual Sensitivity and Plasticity

Synaesthesia is an unusual perceptual experience in which an inducer stimulus triggers a percept in a different domain in addition to its own. To explore the conditions under which synaesthesia evolves, we studied a neuronal network model that represents two recurrently connected neural systems. The interactions in the network evolve according to learning rules that optimize sensory sensitivity. We demonstrate several scenarios, such as sensory deprivation or heightened plasticity, under which synaesthesia can evolve even though the inputs to the two systems are statistically independent and the initial cross-talk interactions are zero. Sensory deprivation is the known causal mechanism for acquired synaesthesia and increased plasticity is implicated in developmental synaesthesia. The model unifies different causes of synaesthesia within a single theoretical framework and repositions synaesthesia not as some quirk of aberrant connectivity, but rather as a functional brain state that can emerge as a consequence of optimising sensory information processing.

Synaesthesia is a remarkable form of altered perception, where one attribute of a stimulus (e.g. sound) leads to the conscious experience of an additional attribute (often colour). Despite being known about for 200 years, there is no commonly agreed upon model for how and why synaesthesia emerges. This study presents a new model of synaesthesia based on computational principles that accounts for the emergence of different types of synaesthesia (acquired and developmental) as well as many of its key characteristics. The model describes how two independent neuronal systems can evolve to interact with one another even though their inputs are statistically uncorrelated. Specifically, synaesthesia arises as a result of instability in the learning process that shapes the network, which can be caused by heightened plasticity or due to sensory deprivation of one of the systems. The model unifies different aspects of synaesthesia and generates novel insights and predictions.

Components of Attention in Grapheme-Color Synesthesia: A Modeling Approach

Grapheme-color synesthesia is a condition where the perception of graphemes consistently and automatically evokes an experience of non-physical color. Many have studied how synesthesia affects the processing of achromatic graphemes, but less is known about the synesthetic processing of physically colored graphemes. Here, we investigated how the visual processing of colored letters is affected by the congruence or incongruence of synesthetic grapheme-color associations. We briefly presented graphemes (10-150 ms) to 9 grapheme-color synesthetes and to 9 control observers. Their task was to report as many letters (targets) as possible, while ignoring digit (distractors). Graphemes were either congruently or incongruently colored with the synesthetes' reported grapheme-color association. A mathematical model, based on Bundesen's (1990) Theory of Visual Attention (TVA), was fitted to each observer's data, allowing us to estimate discrete components of visual attention. The models suggested that the synesthetes processed congruent letters faster than incongruent ones, and that they were able to retain more congruent letters in visual short-term memory, while the control group's model parameters were not significantly affected by congruence. The increase in processing speed, when synesthetes process congruent letters, suggests that synesthesia affects the processing of letters at a perceptual level. To account for the benefit in processing speed, we propose that synesthetic associations become integrated into the categories of graphemes, and that letter colors are considered as evidence for making certain perceptual categorizations in the visual system. We also propose that enhanced visual short-term memory capacity for congruently colored graphemes can be explained by the synesthetes' expertise regarding their specific grapheme-color associations.

Multisensory integration and cross-modal learning in synaesthesia: A unifying model

Recent research into synaesthesia has highlighted the role of learning, yet synaesthesia is clearly a genetic condition. Here we ask how can the idea that synaesthesia reflects innate, genetic differences be reconciled with models that suggest it is driven by learning. A number of lines of evidence suggest that synaesthesia relies on, or at least interacts with, processes of multisensory integration that are common across all people. These include multisensory activations that arise in early regions of the brain as well as feedback from longer-term cross-modal associations generated in memory. These cognitive processes may interact independently to influence the phenomenology of the synaesthetic experience, as well as the individual differences within particular types of synaesthesia. The theoretical framework presented here is consistent with both an innate difference as the fundamental driver of the condition of synaesthesia, and with experiential and semantic influences on the eventual phenotype that emerges. In particular, it proposes that the internally generated synaesthetic percepts are treated similarly to other sensory information as the brain is learning the multisensory attributes of objects and developing cross-modal associations that merge in the concept of the object.