Multisensory integration in schizophrenia: a behavioural and event-related potential study

Abnormal connectivity and activation during audiovisual speech perception in schizophrenia

The unconscious integration of vocal and facial cues during speech perception facilitates face-to-face communication. Recent studies have provided substantial behavioural evidence concerning impairments in audiovisual (AV) speech perception in schizophrenia. However, the specific neurophysiological mechanism underlying these deficits remains unknown. Here, we investigated activities and connectivities centered on the auditory cortex during AV speech perception in schizophrenia. Using magnetoencephalography, we recorded and analysed event-related fields in response to auditory (A: voice), visual (V: face) and AV (voice-face) stimuli in 23 schizophrenia patients (13 males) and 22 healthy controls (13 males). The functional connectivity associated with the subadditive response to AV stimulus (i.e., [AV] < [A] + [V]) was also compared between the two groups. Within the healthy control group, [AV] activity was smaller than the sum of [A] and [V] at latencies of approximately 100 ms in the posterior ramus of the lateral sulcus in only the left hemisphere, demonstrating a subadditive N1m effect. Conversely, the schizophrenia group did not show such a subadditive response. Furthermore, weaker functional connectivity from the posterior ramus of the lateral sulcus of the left hemisphere to the fusiform gyrus of the right hemisphere was observed in schizophrenia. Notably, this weakened connectivity was associated with the severity of negative symptoms. These results demonstrate abnormalities in connectivity between speech- and face-related cortical areas in schizophrenia. This aberrant subadditive response and connectivity deficits for integrating speech and facial information may be the neural basis of social communication dysfunctions in schizophrenia.

Multisensory integration deficits in Schizophrenia and Autism evidenced in behaviour but not event related potentials

The process of integrating information from different sensory channels, known as multisensory integration (MSI) was assessed in two disorders, Autism Spectrum Disorder (ASD) and Schizophrenia (SCZ). 32 healthy controls (HC), 35 SCZ patients, and 23 ASD patients performed an audiovisual (AV) synchronous target detection task while reaction time (RT) and scalp recorded electrophysiological (EEG) activity were measured. MSI in the AV condition resulted in faster and less variable RTs compared to the unimodal conditions. Using our novel bootstrap method, MSI gain was observed in 78 % of HC, 26 % of ASD, and 48 % of SCZ patients. At the neural level, MSI in the AV condition resulted in larger amplitude of sensory evoked responses and cognitive P3 response compared to the corresponding unimodal conditions. These neural effects of MSI were not related to the behavioural MSI gain identified at the individual level and could not explain the deficits in behavioural MSI of patient groups. In conclusion, a robust MSI gain deficit in RT was observed in both patient groups that was not reflected in early perceptual and cognitive electro-cortical responses, suggesting that behavioural MSI deficits in ASD and SCZ may arise at late processing stages such as response selection.

Multisensory integration in the mammalian brain: diversity and flexibility in health and disease

Multisensory integration (MSI) occurs in a variety of brain areas, spanning cortical and subcortical regions. In traditional studies on sensory processing, the sensory cortices have been considered for processing sensory information in a modality-specific manner. The sensory cortices, however, send the information to other cortical and subcortical areas, including the higher association cortices and the other sensory cortices, where the multiple modality inputs converge and integrate to generate a meaningful percept. This integration process is neither simple nor fixed because these brain areas interact with each other via complicated circuits, which can be modulated by numerous internal and external conditions. As a result, dynamic MSI makes multisensory decisions flexible and adaptive in behaving animals. Impairments in MSI occur in many psychiatric disorders, which may result in an altered perception of the multisensory stimuli and an abnormal reaction to them. This review discusses the diversity and flexibility of MSI in mammals, including humans, primates and rodents, as well as the brain areas involved. It further explains how such flexibility influences perceptual experiences in behaving animals in both health and disease. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

A systematic review of the neural correlates of multisensory integration in schizophrenia

Multisensory integration (MSI), in which sensory signals from different modalities are unified, is necessary for our comprehensive perception of and effective adaptation to the objects and events around us. However, individuals with schizophrenia suffer from impairments in MSI, which could explain typical symptoms like hallucination and reality distortion. Because the neural correlates of aberrant MSI in schizophrenia help us understand the physiognomy of this psychiatric disorder, we performed a systematic review of the current research on this subject. The literature search concerned investigated MSI in diagnosed schizophrenia patients compared to healthy controls using brain imaging. Seventeen of 317 identified studies were finally included. To assess risk of bias, the Newcastle-Ottawa quality assessment was used, and the review was written according to the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA). The results indicated that multisensory processes in schizophrenia are associated with aberrant, mainly reduced, neural activity in several brain regions, as measured by event-related potentials, oscillations, activity and connectivity. The conclusion is that a fronto-temporal region, comprising the frontal inferior gyrus, middle temporal gyrus and superior temporal gyrus/sulcus, along with the fusiform gyrus and dorsal visual stream in the occipital-parietal lobe are possible key regions of deficient MSI in schizophrenia.

Multisensory Processing Can Compensate for Top-Down Attention Deficits in Schizophrenia

Studies on schizophrenia (SCZ) and aberrant multisensory integration (MSI) show conflicting results, which are potentially confounded by attention deficits in SCZ. To test this, we examined the interplay between MSI and intersensory attention (IA) in healthy controls (HCs) (N = 27) and in SCZ (N = 27). Evoked brain potentials to unisensory-visual (V), unisensory-tactile (T), or spatiotemporally aligned bisensory VT stimuli were measured with high-density electroencephalography, while participants attended blockwise to either visual or tactile inputs. Behaviorally, IA effects in SCZ, relative to HC, were diminished for unisensory stimuli, but not for bisensory stimuli. At the neural level, we observed reduced IA effects for bisensory stimuli over mediofrontal scalp regions (230-320 ms) in SCZ. The analysis of MSI, using the additive approach, revealed multiple phases of integration over occipital and frontal scalp regions (240-364 ms), which did not differ between HC and SCZ. Furthermore, IA and MSI effects were both positively related to the behavioral performance in SCZ, indicating that IA and MSI mutually facilitate bisensory stimulus processing. Multisensory processing could facilitate stimulus processing and compensate for top-down attention deficits in SCZ. Differences in attentional demands, which may be differentially compensated by multisensory processing, could account for previous conflicting findings on MSI in SCZ.

A combined stimuli presentation for assessing facial emotion related N170, N250 and visual mismatch negativity in neuropsychiatric disorders

Event-related potential (ERP) is a useful approach to assess the neurophysiological correlates of facial emotion processing. Previous studies examined the facial emotion recognition (FER) related ERPs (N170, N250, visual MisMatch Negativity) individually using ERP specific paradigms. This approach can be time-consuming and may not resemble real-life scenarios where an individual must process multiple stimuli simultaneously. The aim of the study was to assess the utility of a combined paradigm when compared to individual paradigms to measure N170, N250 and visual MisMatch Negativity (vMMN) in healthy controls (HC), utilizing emotion stimuli standardized in the Indian population. Further, the combined paradigm was examined in patients with schizophrenia (SCZ) to detect the differences in ERPs compared to HC. Within paradigms, ERPs showed higher amplitudes for emotion compared to neutral stimuli suggesting that the paradigms were able to detect valence associated with emotional stimuli. The combined paradigm was able to elicit decipherable peaks of N170, N250 and vMMN similar to individual paradigms. ERP data quality as assessed by analytic Standardized Measurement Error (aSME) showed a satisfactory aggregate score of above 2 for all the three paradigms. Combined paradigm approaches to record ERPs in neuropsychiatric conditions has the advantage of reducing the time required for task administration, avoiding practice effects, better subject cooperation and participation.

Reduced multisensory facilitation exists at different periods of development in autism

Atypical sensory processing is now recognised as a key component of an autism diagnosis. The integration of multiple sensory inputs (multisensory integration (MSI)) is thought to be idiosyncratic in autistic individuals and may have cascading effects on the development of higher-level skills such as social communication. Multisensory facilitation was assessed using a target detection paradigm in 45 autistic and 111 neurotypical individuals, matched on age and IQ. Target stimuli were: auditory (A; 3500 Hz tone), visual (V; white disk 'flash') or audiovisual (AV; simultaneous tone and flash), and were presented on a dark background in a randomized order with varying stimulus onset delays. Reaction time (RT) was recorded via button press. In order to assess possible developmental effects, participants were divided into younger (age 14 or younger) and older (age 15 and older) groups. Redundancy gain (RG) was significantly greater in neurotypical, compared to autistic individuals. No significant effect of age or interaction was found. Race model analysis was used to compute a bound value that represented the facilitation effect provided by MSI. Our results revealed that MSI facilitation occurred (violation of the race model) in neurotypical individuals, with more efficient MSI in older participants. In both the younger and older autistic groups, we found reduced MSI facilitation (no or limited violation of the race model). Autistic participants showed reduced multisensory facilitation compared to neurotypical participants in a simple target detection task, void of social context. This remained consistent across age. Our results support evidence that autistic individuals may not integrate low-level, non-social information in a typical fashion, adding to the growing discussion around the influential effect that basic perceptual atypicalities may have on the development of higher-level, core aspects of autism.

Low-frequency oscillations reflect aberrant tone restoration during the auditory continuity illusion in schizophrenia

Patients with schizophrenia (ScZ) often show impairments in auditory information processing. These impairments have been related to clinical symptoms, such as auditory hallucinations. Some researchers have hypothesized that aberrant low-frequency oscillations contribute to auditory information processing deficits in ScZ. A paradigm for which modulations in low-frequency oscillations are consistently found in healthy individuals is the auditory continuity illusion (ACI), in which restoration processes lead to a perceptual grouping of tone fragments and a mask, so that a physically interrupted sound is perceived as continuous. We used the ACI paradigm to test the hypothesis that low-frequency oscillations play a role in aberrant auditory information processing in patients with ScZ (N = 23). Compared with healthy control participants we found that patients with ScZ show elevated continuity illusions of interrupted, partially-masked tones. Electroencephalography data demonstrate that this elevated continuity perception is reflected by diminished 3 Hz power. This suggests that reduced low-frequency oscillations relate to elevated restoration processes in ScZ. Our findings support the hypothesis that aberrant low-frequency oscillations contribute to altered perception-related auditory information processing in ScZ.

Reduced multisensory facilitation in adolescents and adults on the Autism Spectrum

Individuals with autism are reported to integrate information from visual and auditory channels in an idiosyncratic way. Multisensory integration (MSI) of simple, non-social stimuli (i.e., flashes and beeps) was evaluated in adolescents and adults with (n = 20) and without autism (n = 19) using a reaction time (RT) paradigm using audio, visual, and audiovisual stimuli. For each participant, the race model analysis compares the RTs on the audiovisual condition to a bound value computed from the unimodal RTs that reflects the effect of redundancy. If the actual audiovisual RTs are significantly faster than this bound, the race model is violated, indicating evidence of MSI. Our results show that the race model violation occurred only for the typically-developing (TD) group. While the TD group shows evidence of MSI, the autism group does not. These results suggest that multisensory integration of simple information, void of social content or complexity, is altered in autism. Individuals with autism may not benefit from the advantage conferred by multisensory stimulation to the same extent as TD individuals. Altered MSI for simple, non-social information may have cascading effects on more complex perceptual processes related to language and behaviour in autism.

The Effect of Visual Capture Towards Subjective Embodiment Within the Full Body Illusion

Typically, multisensory illusion paradigms emphasise the importance of synchronous visuotactile integration to induce subjective embodiment towards another body. However, the extent to which embodiment is due to the 'visual capture' of congruent visuoproprioceptive information alone remains unclear. Thus, across two experiments (total N = 80), we investigated how mere visual observation of a mannequin body, viewed from a first-person perspective, influenced subjective embodiment independently from concomitant visuotactile integration. Moreover, we investigated whether slow, affective touch on participants' own, unseen body (without concomitant touch on the seen mannequin) disrupted visual capture effects to a greater degree than fast, non-affective touch. In total, 40% of participants experienced subjective embodiment towards the mannequin body following mere visual observation, and this effect was significantly higher than conditions which included touch to participants own, unseen body. The velocity of the touch that participants received (affective/non-affective) did not differ in modulating visual capture effects. Furthermore, the effects of visual capture and perceived pleasantness of touch was not modulated by subthreshold eating disorder psychopathology. Overall, this study suggests that congruent visuoproprioceptive cues can be sufficient to induce subjective embodiment of a whole body, in the absence of visuotactile integration and beyond mere confabulatory responses.

Atypical audiovisual temporal function in autism and schizophrenia: similar phenotype, different cause

Binding across sensory modalities yields substantial perceptual benefits, including enhanced speech intelligibility. The coincidence of sensory inputs across time is a fundamental cue for this integration process. Recent work has suggested that individuals with diagnoses of schizophrenia (SZ) and autism spectrum disorder (ASD) will characterize auditory and visual events as synchronous over larger temporal disparities than their neurotypical counterparts. Namely, these clinical populations possess an enlarged temporal binding window (TBW). Although patients with SZ and ASD share aspects of their symptomatology, phenotypic similarities may result from distinct etiologies. To examine similarities and variances in audiovisual temporal function in these two populations, individuals diagnosed with ASD (n = 46; controls n = 40) and SZ (n = 16, controls = 16) completed an audiovisual simultaneity judgment task. In addition to standard psychometric analyses, synchrony judgments were assessed using Bayesian causal inference modeling. This approach permits distinguishing between distinct causes of an enlarged TBW: an a priori bias to bind sensory information and poor fidelity in the sensory representation. Findings indicate that both ASD and SZ populations show deficits in multisensory temporal acuity. Importantly, results suggest that while the wider TBWs in ASD most prominently results from atypical priors, the wider TBWs in SZ results from a trend toward changes in prior and weaknesses in the sensory representations. Results are discussed in light of current ASD and SZ theories and highlight that different perceptual training paradigms focused on improving multisensory integration may be most effective in these two clinical populations and emphasize that similar phenotypes may emanate from distinct mechanistic causes.

Temporal processing deficit leads to impaired multisensory binding in schizophrenia

INTRODUCTION

Schizophrenia has been characterised by neurodevelopmental dysconnectivity resulting in cognitive and perceptual dysmetria. Hence patients with schizophrenia may be impaired to detect the temporal relationship between stimuli in different sensory modalities. However, only a few studies described deficit in perception of temporally asynchronous multisensory stimuli in schizophrenia.

METHODS

We examined the perceptual bias and the processing time of synchronous and delayed sounds in the streaming-bouncing illusion in 16 patients with schizophrenia and a matched control group of 18 participants.

RESULTS

Equal for patients and controls, the synchronous sound biased the percept of two moving squares towards bouncing as opposed to the more frequent streaming percept in the condition without sound. In healthy controls, a delay of the sound presentation significantly reduced the bias and led to prolonged processing time whereas patients with schizophrenia did not differentiate between this condition and the condition with synchronous sound.

CONCLUSION

Schizophrenia leads to a prolonged window of simultaneity for audiovisual stimuli. Therefore, temporal processing deficit in schizophrenia can lead to hyperintegration of temporally unmatched multisensory stimuli.

Beta/Gamma Oscillations and Event-Related Potentials Indicate Aberrant Multisensory Processing in Schizophrenia

Recent behavioral and neuroimaging studies have suggested multisensory processing deficits in patients with schizophrenia (SCZ). Thus far, the neural mechanisms underlying these deficits are not well understood. Previous studies with unisensory stimulation have shown altered neural oscillations in SCZ. As such, altered oscillations could contribute to aberrant multisensory processing in this patient group. To test this assumption, we conducted an electroencephalography (EEG) study in 15 SCZ and 15 control participants in whom we examined neural oscillations and event-related potentials (ERPs) in the sound-induced flash illusion (SIFI). In the SIFI multiple auditory stimuli that are presented alongside a single visual stimulus can induce the illusory percept of multiple visual stimuli. In SCZ and control participants we compared ERPs and neural oscillations between trials that induced an illusion and trials that did not induce an illusion. On the behavioral level, SCZ (55.7%) and control participants (55.4%) did not significantly differ in illusion rates. The analysis of ERPs revealed diminished amplitudes and altered multisensory processing in SCZ compared to controls around 135 ms after stimulus onset. Moreover, the analysis of neural oscillations revealed altered 25–35 Hz power after 100 to 150 ms over occipital scalp for SCZ compared to controls. Our findings extend previous observations of aberrant neural oscillations in unisensory perception paradigms. They suggest that altered ERPs and altered occipital beta/gamma band power reflect aberrant multisensory processing in SCZ.

Voxel-Based Morphometry in Individuals at Genetic High Risk for Schizophrenia and Patients with Schizophrenia during Their First Episode of Psychosis

BACKGROUND

Understanding morphologic changes in vulnerable and early disease state of schizophrenia (SZ) may provide further insight into the development of psychosis.

METHOD

Whole brain voxel-based morphometry was performed to identify gray matter (GM) regional differences in 60 individuals with SZ during their first psychotic episode (FE-SZ), 31 individuals at genetic high risk for SZ (GHR-SZ) individuals, and 71 healthy controls.

RESULTS

Significant differences were found in several regions including the prefrontal cortex, parietal lobe, temporal lobe, hippocampus, occipital lobe, and cerebellum among the three groups (p<0.05, corrected). Compared to the HC group, the FE-SZ group had significantly decreased GM volumes in several regions including the cerebellum, hippocampus, fusiform gyrus, lingual gyrus, supramarginal gyrus, and superior, middle, and inferior temporal gyri and significantly increased GM volumes in the middle frontal gyrus and inferior operculum frontal gyrus (p<0.05). The GHR-SZ group had significant decreases in GM volumes in the supramaginal gyrus, precentral gyrus, and rolandic operculum and significant increases in GM volumes in the cerebellum, fusiform gyrus, middle frontal gyrus, inferior operculum frontal gyrus, and superior, middle, and inferior temporal gyri when compared to the HC group (p<0.05). Compared to the GHR-SZ group, the FE-SZ group had significant decreases in GM volumes in several regions including the cerebellum, fusiform gyrus, supramarginal gyrus, and superior, middle, and inferior temporal gyri (p<0.05).

CONCLUSIONS

The findings herein implicate the involvement of multisensory integration in SZ development and pathophysiology. Additionally, the patterns of observed differences suggest possible indicators of disease, vulnerability, and resiliency in SZ.