A deafening flash! Visual interference of auditory signal detection

The Effect of Face-Voice Gender Consistency on Impression Evaluation

Face and voice are important information cues of interpersonal interaction. Most previous studies have investigated the cross-modal perception of face and voice from the perspective of cognitive psychology, but few empirical studies have focused on the effect of gender consistency of face and voice on the impression evaluation of the target from the perspective of social cognition. Based on the two-stage model of stereotype activation and the stereotype content model, this research examined the effects of face-voice gender consistency on impression evaluation (gender categorization and warmth competence evaluation) by using a cross-modal priming paradigm (Study 1, 20 males and 23 females, M_age = 21.00, SD_age = 2.59), a sequential presentation task (Study 2a, 57 males and 70 females, M_age = 18.54, SD_age = 1.54; Study 2b, 52 males and 51 females, M_age = 18.54, SD_age = 1.36), and a simultaneous presentation task (Study 3, 51 males and 55 females, M_age = 23.58, SD_age = 3.20), respectively. The results showed that: (1) there was a face-voice gender consistency preference in gender categorization, and the response of face-voice consistent condition was faster than that of inconsistent condition; (2) compared with the face-voice gender-inconsistent individuals, the participants showed a higher and more stable evaluation of the warmth and competence of the gender-consistent individuals, indicating the effect of matching preference of the face-voice gender consistency on the impression evaluation; (3) people paid more attention to the gender information of faces in the impression evaluation, and the female face could improve people's evaluation on the target's warmth and competence; (4) males were more intolerant of face-voice gender inconsistency when presented sequentially; the "voice needs to match face" effect was stronger for females when presented simultaneously. These findings, on the one hand, enrich and expand previous theories and research on cross-modal processing of face and voice from the perspective of social cognitive impression evaluation; on the other hand, these findings have important practical implications for impression management and decision-making in social interaction.

'Tasting Imagination': What Role Chemosensory Mental Imagery in Multisensory Flavour Perception?

A number of perplexing phenomena in the area of olfactory/flavour perception may fruitfully be explained by the suggestion that chemosensory mental imagery can be triggered automatically by perceptual inputs. In particular, the disconnect between the seemingly limited ability of participants in chemosensory psychophysics studies to distinguish more than two or three odorants in mixtures and the rich and detailed flavour descriptions that are sometimes reported by wine experts; the absence of awareness of chemosensory loss in many elderly individuals; and the insensitivity of the odour-induced taste enhancement (OITE) effect to the mode of presentation of olfactory stimuli (i.e., orthonasal or retronasal). The suggestion made here is that the theory of predictive coding, developed first in the visual modality, be extended to chemosensation. This may provide a fruitful way of thinking about the interaction between mental imagery and perception in the experience of aromas and flavours. Accepting such a suggestion also raises some important questions concerning the ecological validity/meaning of much of the chemosensory psychophysics literature that has been published to date.

How do Different Types of Synesthesia Cluster Together? Implications for Causal Mechanisms

It is unclear whether synesthesia is one condition or many, and this has implications for whether theories should postulate a single cause or multiple independent causes. Study 1 analyses data from a large sample of self-referred synesthetes (N  =  2,925), who answered a questionnaire about N  =  164 potential types of synesthesia. Clustering and factor analysis methods identified around seven coherent groupings of synesthesia, as well as showing that some common types of synesthesia do not fall into any grouping at all (mirror-touch, hearing-motion, tickertape). There was a residual positive correlation between clusters (they tend to associate rather than compete). Moreover, we observed a "snowball effect" whereby the chances of having a given cluster of synesthesia go up in proportion to the number of other clusters a person has (again suggesting non-independence). Clusters tended to be distinguished by shared concurrent experiences rather than shared triggering stimuli (inducers). We speculate that modulatory feedback pathways from the concurrent to inducers may play a key role in the emergence of synesthesia. Study 2 assessed the external validity of these clusters by showing that they predict performance on other measures known to be linked to synesthesia.

Hearing what you see: Distinct excitatory and disinhibitory mechanisms contribute to visually-evoked auditory sensations

Visual motion or flashing lights can evoke auditory sensations in some people. This large-scale internet study aimed to validate a combined subjective/objective test of the genuineness of this putative form of synaesthesia (visually-evoked auditory response, vEAR). Correlations were measured between each individual's ratings of the vividness of auditory sensations evoked by a series of looping videos, and measurement of the videos' physical low-level motion energy, calculated using Adelson and Bergen's (1985) computational model of low-level visual motion processing. The strength of this association for each individual provided a test of how strongly subjective vEAR was driven by objective motion energy ('ME-sensitivity'). A second aim was to infer whether vEAR depends on cortical excitation and/or disinhibition of early visual and/or auditory brain areas. To achieve this, correlations were measured between the above vEAR measures and visual contrast surround-suppression, which is thought to index lateral inhibition in the early visual system. As predicted by a disinhibition account of vEAR, video ratings were overall higher in individuals showing weaker surround-suppression. Interestingly, surround-suppression and ME-sensitivity did not correlate. Additionally, both surround-suppression and ME-sensitivity each independently predicted different clusters of trait measures selected for their possible association with cortical excitability and/or disinhibition: Surround-suppression was associated with vEAR self-ratings and auditory-evoked visual phosphenes, while ME-sensitivity was independently associated with ratings of other traits including susceptibility to migraine and pattern glare. Altogether, these results suggest there are two independent mechanisms underlying vEAR and its associated traits, based putatively on cortical disinhibition versus excitability.

Seeing objects improves our hearing of the sounds they make

It has been established that lip reading improves the perception of auditory speech. But does seeing objects themselves help us hear better the sounds they make? Here we report a series of psychophysical experiments in humans showing that the visual enhancement of auditory sensitivity is not confined to speech. We further show that the crossmodal enhancement was associated with the conscious visualization of the stimulus: we can better hear the sounds an object makes when we are conscious of seeing that object. Our work extends an intriguing crossmodal effect, previously circumscribed to speech, to a wider domain of real-world objects, and suggests that consciousness contributes to this effect.

Hearing through Your Eyes: Neural Basis of Audiovisual Cross-activation, Revealed by Transcranial Alternating Current Stimulation

Some people experience auditory sensations when seeing visual flashes or movements. This prevalent synaesthesia-like visually evoked auditory response (vEAR) could result either from overexuberant cross-activation between brain areas and/or reduced inhibition of normally occurring cross-activation. We have used transcranial alternating current stimulation (tACS) to test these theories. We applied tACS at 10 Hz (alpha band frequency) or 40 Hz (gamma band), bilaterally either to temporal or occipital sites, while measuring same/different discrimination of paired auditory (A) versus visual (V) Morse code sequences. At debriefing, participants were classified as vEAR or non-vEAR, depending on whether they reported "hearing" the silent flashes. In non-vEAR participants, temporal 10-Hz tACS caused impairment of A performance, which correlated with improved V; conversely under occipital tACS, poorer V performance correlated with improved A. This reciprocal pattern suggests that sensory cortices are normally mutually inhibitory and that alpha-frequency tACS may bias the balance of competition between them. vEAR participants showed no tACS effects, consistent with reduced inhibition, or enhanced cooperation between modalities. In addition, temporal 40-Hz tACS impaired V performance, specifically in individuals who showed a performance advantage for V (relative to A). Gamma-frequency tACS may therefore modulate the ability of these individuals to benefit from recoding flashes into the auditory modality, possibly by disrupting cross-activation of auditory areas by visual stimulation. Our results support both theories, suggesting that vEAR may depend on disinhibition of normally occurring sensory cross-activation, which may be expressed more strongly in some individuals. Furthermore, endogenous alpha- and gamma-frequency oscillations may function respectively to inhibit or promote this cross-activation.

Acute findings in an acquired neurosensory dysfunction

Background

In the Autumn of 2016, diplomatic personnel residing in Havana began to present with symptoms of dizziness, ear pain, and tinnitus that emerged after perception of high frequency noise and/or a pressure sensation. Understanding the acute symptoms of this disorder is important for better defining the disorder and developing optimal diagnostic, preventive, and treatment algorithms.

Objectives

To define the presenting symptoms in a cohort of patients in the acute time period after perceiving a noise/pressure exposure in Havana.

Design/Settings/Participants

Review of 25 symptomatic individuals who reported a localized sensation of noise/pressure and 10 asymptomatic individuals (roommates of those affected) who did not experience the sound/pressure.

Results

Immediately after the exposure, the majority of individuals reported intense ear pain in one or both ears and experienced tinnitus. All of the individuals noticed unsteadiness and features of cognitive impairment. On presentation to our center, dizziness (92%) and cognitive complaints (56%) were the most common symptoms. Formal testing revealed that 100% of individuals had an otolithic abnormality and evidence of cognitive dysfunction.

Conclusion and Relevance

This study focuses on the acute presentation of a phenomenon in which symptoms emerge after perception of a localized noise/pressure and in which the acute symptomology includes the universal nature of vestibular injuries and select cognitive deficits. The findings presented in this acute group of patients begin to provide a better picture of the initial injury pattern seen after this exposure and may allow for more accurate diagnosis of this disorder in future cases.

Level of Evidence

Retrospective review

Sounds from seeing silent motion: Who hears them, and what looks loudest?

Some people hear what they see: car indicator lights, flashing neon shop signs, and people's movements as they walk may all trigger an auditory sensation, which we call the visual-evoked auditory response (vEAR or 'visual ear'). We have conducted the first large-scale online survey (N > 4000) of this little-known phenomenon. We analysed the prevalence of vEAR, what induces it, and what other traits are associated with it. We assessed prevalence by asking whether respondents had previously experienced vEAR. Participants then rated silent videos for vividness of evoked auditory sensations, and answered additional trait questions. Prevalence appeared higher relative to other typical synaesthesias. Prior awareness and video ratings were associated with greater frequency of other synaesthesias, including flashes evoked by sounds, and musical imagery. Higher-rated videos often depicted meaningful events that predicted sounds (e.g., collisions). However, even videos containing abstract flickering or moving patterns could also elicit higher ratings, despite having no predictable association with sounds. Such videos had higher levels of raw 'motion energy' (ME), which we quantified using a simple computational model of motion processing in early visual cortex. Critically, only respondents reporting prior awareness of vEAR tended to show a positive correlation between video ratings and ME. This specific sensitivity to ME suggests that in vEAR, signals from visual motion processing may affect audition relatively directly without requiring higher-level interpretative processes. Our other findings challenge the popular assumption that individuals with synaesthesia are rare and have ideosyncratic patterns of brain hyper-connectivity. Instead, our findings of apparently high prevalence and broad associations with other synaesthesias and traits are jointly consistent with a common dependence on normal variations in physiological mechanisms of disinhibition or excitability of sensory brain areas and their functional connectivity. The prevalence of vEAR makes it easier to test such hypotheses further, and makes the results more relevant to understanding not only synaesthetic anomalies but also normal perception.

Electrophysiological correlates and psychoacoustic characteristics of hearing-motion synaesthesia

People with hearing-motion synaesthesia experience sounds from moving or changing (e.g. flickering) visual stimuli. This phenomenon may be one of the most common forms of synaesthesia but it has rarely been studied and there are no studies of its neural basis. We screened for this in a sample of 200+ individuals, and estimated a prevalence of 4.2%. We also document its characteristics: it tends to be induced by physically moving stimuli (more so than static stimuli which imply motion or trigger illusory motion); and the psychoacoustic features are simple (e.g. "whooshing") with some systematic correspondences to vision (e.g. faster movement is higher pitch). We demonstrate using event-related potentials that it emerges from early perceptual processing of vision. The synaesthetes have a higher amplitude motion-evoked N2 (165-185ms), with some evidence of group differences as early as 55-75ms. We discuss similarities between hearing-motion synaesthesia and previous observations that visual motion triggers auditory activity in the congenitally deaf. It is possible that both conditions reflect the maintenance of multisensory pathways found in early development that most people lose but can be retained in certain people in response to sensory deprivation (in the deaf) or, in people with normal hearing, as a result of other differences (e.g. genes predisposing to synaesthesia).