Perceptual integration of head and eye cues to gaze direction in schizophrenia

The integration of head and body cues during the perception of social interactions

Humans spend a large proportion of time participating in social interactions. The ability to accurately detect and respond to human interactions is vital for social functioning, from early childhood through to older adulthood. This detection ability arguably relies on integrating sensory information from the interactants. Within the visual modality, directional information from a person's eyes, head, and body are integrated to inform where another person is looking and who they are interacting with. To date, social cue integration research has focused largely on the perception of isolated individuals. Across two experiments, we investigated whether observers integrate body information with head information when determining whether two people are interacting, and manipulated frame of reference (one of the interactants facing observer vs. facing away from observer) and the eye-region visibility of the interactant. Results demonstrate that individuals integrate information from the body with head information when perceiving dyadic interactions, and that integration is influenced by the frame of reference and visibility of the eye-region. Interestingly, self-reported autistics traits were associated with a stronger influence of body information on interaction perception, but only when the eye-region was visible. This study investigated the recognition of dyadic interactions using whole-body stimuli while manipulating eye visibility and frame of reference, and provides crucial insights into social cue integration, as well as how autistic traits affect cue integration, during perception of social interactions.

Social engagement revealed by gaze following in third-party observers of simulated social conflict

Humans and non-human primates can allocate visual attention to areas of high interest in their visual field based on the behaviors of their social partners. Allocation of attention is particularly important for third-party observers of social interactions. By following the gaze of interacting individuals, the observer can obtain information about the mental states, emotions, and intentions of others. We presented three adult monkeys (Macaca mulatta) with videos of simulated social interactions and quantified their eye movements to determine which observed behaviors were most conducive to gaze following. Social interactions were simulated by juxtaposing two videos depicting a threatening and an appeasing individual facing each other, with the timing of the facial and bodily displays adjusted to mimic an exchange of social signals. Socially meaningful facial displays combined with full body movements significantly enhanced the probability of gaze following and joint attention. Despite the synthetic nature of these interactions, the facial and bodily displays of the submissive individual elicited significantly more joint-attention than gaze-following saccades, suggesting a preferential allocation of attention to the recipients of threatening displays. Temporal alignment of gaze following and joint attention to the frames of each video showed numerous clusters of significant increases in the frequency of these saccades. These clusters suggest that some videos contained signals that can induce a quasi-automatic redirection of the observer's attention. However, these saccades occurred only on a fraction of the viewings, and we have documented large inter-individual variations. All viewers produced sequences of joint attention saccades (check-backs) shifting their attention between the two monkeys as though monitoring the simulated emitting-receiving cycle of social signals. These sequences reflect the viewer's interest in monitoring the ongoing exchange of agonistic and affiliative displays. It appears that in macaque monkeys, the scanpaths of third-party observers of simulated social interactions are informed by social-cognitive processes suggestive of mentalizing.

Self-monitoring in schizophrenia: Weighting exteroceptive visual signals against self-generated vestibular cues

Disturbances in self-monitoring are core symptoms of schizophrenia. Some research suggests an over-reliance on exteroceptive cues and a reduced weighting of self-generated interoceptive signals to guide perception. The vestibular sense provides important self-generated information about the body in space. Alterations of vestibular function are reported in schizophrenia, but it is unknown whether internally generated vestibular information is discounted in favour of exteroceptive input. In this study, we test for evidence of an over-reliance on exteroceptive visual cues and a reduced weighting of vestibular signals in guiding perception. In a group of individuals with schizophrenia and healthy controls, we used a well-studied visual illusion - the Tilt Illusion - to probe the respective weight given to visual and vestibular cues in judging line orientation. The Tilt Illusion reveals that perceived orientation of a vertical grating is biased by the orientation in its surround. This illusion increases when the head is tilted, due to the reduced reliability of vestibular information that would otherwise provide an internally generated reference for vertical. We predicted that an over-reliance on exteroceptive cues in schizophrenia would lead to a reduced susceptibility to the effects of head position on Tilt Illusion strength. We find no difference between patients and controls. Both groups show comparable Tilt Illusion magnitudes that increase when the head is tilted. Thus, our findings suggest that chronic patients with schizophrenia adequately combine self-generated vestibular cues and exteroceptive visual input to judge line verticality. A stronger reliance on exteroceptive information over internally generated signals in guiding perception is not evident in our data. Deficits in self-monitoring might therefore be modality specific or state dependant.

Normal gaze processing in developmental prosopagnosia

Two key functions in human face perception are gaze discrimination and identity recognition. Here we examine whether gaze discrimination can be intact when identity recognition is impaired in developmental prosopagnosia (DP). We ran a large sample of developmental prosopagnosics (DPs) with a series of gaze discrimination tasks that assess various mechanisms in gaze processing. Experiment 1 (N = 101 DP participants) investigates spatial processing using an abnormal eye gaze detection task and a Wollaston illusion task that measures perceptual integration of eye and head direction. Experiment 2 (N = 45 DP participants) investigates temporal processing using an adaptation task and a serial dependence task. Despite their deficits with identity recognition, DPs performed in the normal range across both experiments. These results demonstrate that gaze discrimination can be normal in DP, and that various mechanisms of gaze processing can be spared when identity recognition is impaired. Our findings clarify the highly selective nature of impairments in DP and provide support for neurocognitive models of face perception with distinct mechanisms for gaze and identity processing.

Gaze direction and face orientation modulate perceptual sensitivity to faces under interocular suppression

Faces convey information essential for social interaction. Their importance has prompted suggestions that some facial features may be processed unconsciously. Although some studies have provided empirical support for this idea, it remains unclear whether these findings were due to perceptual processing or to post-perceptual decisional factors. Evidence for unconscious processing of facial features has predominantly come from the Breaking Continuous Flash Suppression (b-CFS) paradigm, which measures the time it takes different stimuli to overcome interocular suppression. For example, previous studies have found that upright faces are reported faster than inverted faces, and direct-gaze faces are reported faster than averted-gaze faces. However, this procedure suffers from important problems: observers can decide how much information they receive before committing to a report, so their detection responses may be influenced by differences in decision criteria and by stimulus identification. Here, we developed a new procedure that uses predefined exposure durations, enabling independent measurement of perceptual sensitivity and decision criteria. We found higher detection sensitivity to both upright and direct-gaze (compared to inverted and averted-gaze) faces, with no effects on decisional factors. For identification, we found both greater sensitivity and more liberal criteria for upright faces. Our findings demonstrate that face orientation and gaze direction influence perceptual sensitivity, indicating that these facial features may be processed unconsciously.

Objects that induce face pareidolia are prioritized by the visual system

The human visual system has evolved specialized neural mechanisms to rapidly detect faces. Its broad tuning for facial features is thought to underlie the illusory perception of faces in inanimate objects, a phenomenon called face pareidolia. Recent studies on face pareidolia suggest that the mechanisms underlying face processing, at least at the early stages of visual encoding, may treat objects that resemble faces as real faces; prioritizing their detection. In our study, we used breaking continuous flash suppression (b-CFS) to examine whether the human visual system prioritizes the detection of objects that induce face pareidolia over stimuli matched for object content. Similar to previous b-CFS results using real face stimuli, we found that participants detected the objects with pareidolia faces faster than object-matched control stimuli. Given that face pareidolia has been more frequently reported amongst individuals prone to hallucinations, we also explored whether this rapid prioritization is intact in individuals with schizophrenia, and found evidence suggesting that it was. Our findings suggest that face pareidolia engages a broadly tuned mechanism that facilitates rapid face detection. This may involve the proposed fast subcortical pathway that operates outside of visual awareness.

A Parametric Framework to Generate Visual Illusions Using Python

Visual illusions are fascinating phenomena that have been used and studied by artists and scientists for centuries, leading to important discoveries about the neurocognitive underpinnings of perception, consciousness, and neuropsychiatric disorders such as schizophrenia or autism. Surprisingly, despite their historical and theoretical importance as psychological stimuli, there is no dedicated software, nor consistent approach, to generate illusions in a systematic fashion. Instead, scientists have to craft them by hand in an idiosyncratic fashion, or use pre-made images not tailored for the specific needs of their studies. This, in turn, hinders the reproducibility of illusion-based research, narrowing possibilities for scientific breakthroughs and their applications. With the aim of addressing this gap, Pyllusion is a Python-based open-source software (freely available at https://github.com/RealityBending/Pyllusion), that offers a framework to manipulate and generate illusions in a systematic way, compatible with different output formats such as image files (.png, .jpg, .tiff, etc.) or experimental software (such as PsychoPy).

Eye Direction Detection and Perception as Premises of a Social Brain: A Narrative Review of Behavioral and Neural Data

The eyes and the gaze are important stimuli for social interaction in humans. Impaired recognition of facial identity, facial emotions, and inference of the intentions of others may result from difficulties in extracting information relevant to the eye region, mainly the direction of gaze. Therefore, a review of these data is of interest. Behavioral data demonstrating the importance of the eye region and how humans respond to gaze direction are reviewed narratively, and several theoretical models on how visual information on gaze is processed are discussed to propose a unified hypothesis. Several issues that have not yet been investigated are identified. The authors tentatively suggest experiments that might help progress research in this area. The neural aspects are subsequently reviewed to best describe the low-level and higher-level visual information processing stages in the targeted subcortical and cortical areas. A specific neural network is proposed on the basis of the literature. Various gray areas, such as the temporality of the processing of visual information, the question of salience priority, and the coordination between the two hemispheres, remain unclear and require further investigations. Finally, disordered gaze direction detection mechanisms and their consequences on social cognition and behavior are discussed as key deficiencies in several conditions, such as autism spectrum disorder, 22q11.2 deletion, schizophrenia, and social anxiety disorder. This narrative review provides significant additional data showing that the detection and perception of someone's gaze is an essential part of the development of our social brain.

Bottom-up processing of fearful and angry facial expressions is intact in schizophrenia

INTRODUCTION

Judgments of emotion from faces are reportedly impaired in schizophrenia. However, it is unclear whether this is due to a top-down cognitive deficit in evaluating sensory information or a fundamental bottom-up perturbation in the early stages of face encoding. This study examined whether deficits in emotion processing reflect imprecision in the initial preconscious registration of emotional face expressions within the visual system.

METHODS

Using continuous flash suppression (CFS), we presented participants (18 patients with schizophrenia, 8M/10F; 20 healthy controls, 13M/7F) with fearful and angry faces. Previous CFS research on healthy participants reveals that fearful facial expressions gain privileged access to awareness over angry faces-demonstrating the visual system's ability to discriminate these emotions at a preconscious level. We used this same approach to probe the integrity of early emotion encoding whilst minimising the potential contribution of any top-down cognitive biases on perceptual judgments.

RESULTS

In both groups, fearful faces were perceived faster than angry faces, with no differences observed between patients and controls.

CONCLUSIONS

Emotion processing difficulties in schizophrenia are unlikely to reflect an early sensory deficit, but rather a deficit in social cognition that has a top-down impact on the conscious evaluation of facial expressions.

Gaze direction biases emotion categorisation in schizophrenia

The successful integration of eye gaze direction and emotion cues from faces is important not only for co-ordinated interactions, but also for the detection of social signals alerting us to threat posed by a conspecific, or elsewhere in our immediate environment. It is now well-established that people with schizophrenia experience aberrant eye gaze and facial emotion processing. These social-cognitive differences might contribute to the maintenance of socially-themed delusions which are characterised by the hyper-attribution of threatening intentions to others. However, no study has directly examined whether the mechanisms which govern the integration of eye gaze and emotion information diverge in schizophrenia, and more importantly, whether this reflects a fundamental 'bottom-up' perceptual deficit or a 'top-down' cognitive bias. Fifteen outpatients diagnosed with schizophrenia and 21 healthy age- and IQ-matched controls performed an emotion categorisation task (anger/fear) on morphed facial expressions of anger or fear, displaying either direct or averted gaze. Results in both controls and patients replicated the previous finding that combinations of anger with direct gaze, and fear with averted gaze - which signal a relevant threat to the observer - benefited from more accurate emotion recognition than alternate gaze-emotion combinations. Bayesian model selection revealed that for patients this effect was mediated by a shift in decision bias towards emotions which signal self-relevant threat, rather than a change in sensitivity as observed in controls. These results critically highlight a different cognitive mechanism governing gaze and face-cued emotion integration in schizophrenia, which has a top-down influence on the evaluation of perceptual input.

Retinal single-layer analysis with optical coherence tomography (OCT) in schizophrenia spectrum disorder

BACKGROUND

Volume reductions in brain structures of patients with schizophrenia spectrum disorder (SSD) have repeatedly been found in voxel-based morphometry MRI studies. Hence, an underlying neurodegenerative etiological component of SSD is currently being discussed. In recent years, the imaging method of optical coherence tomography (OCT) has shown its potential in evaluating structural changes in the retina in patients with confirmed neurodegenerative disorders, providing a window into the brain.

METHODS

Twenty-six patients with schizophrenia or schizoaffective disorder and 23 age- and sex-matched healthy controls were examined with the Heidelberg Spectralis OCT system to derive a single-layer analysis of both retinas. The segmentation of retinal layers was manually corrected to minimize artifacts and software imprecisions.

RESULTS

Compared to the control group, SSD patients showed reduced thickness and volume measurements for nearly all retinal layers, and these differences reached significance for macular volume, macular thickness, retinal nerve fiber layer (RNFL) and inner nucleiform layer (INL). Furthermore, a significant correlation between the duration of illness and the total volume of the RNFL was found.

CONCLUSION

Our OCT measurements demonstrate reduced single retinal layer thickness in patients with SSD. In the context of the MRI volume changes, our results provide further evidence that structural changes seen in the brain of patients are also observable in the retina, potentially allowing further insights into the different components of the nervous system that are altered in this highly etiologically complex disorder.

Examining motion speed processing in schizophrenia using the flash lag illusion

Research on visual perception in schizophrenia suggests a deficit in motion processing. Specifically, difficulties with discriminating motion speed are commonly reported. However, speed discrimination tasks typically require participants to make judgments about the difference between two stimuli in a two-interval forced choice (2IFC) task. Such tasks not only tap into speed processing mechanisms, but also rely on higher executive functioning including working memory and attention which has been shown to be compromised in schizophrenia. We used the Flash Lag illusion to examine speed processing in patients with schizophrenia. Based on previous research showing a strong dependence between motion speed and the illusion magnitude, we expected a deficit in speed processing to alter this relationship. A motion processing deficit in patients would also predict overall reductions in perceived lag. We found the magnitude and speed dependence of the Flash Lag illusion to be similar in patients and controls. Together, the findings suggest no general abnormality in motion speed processing in schizophrenia.

No evidence for abnormal priors in early vision in schizophrenia

The predictive coding account of psychosis postulates the abnormal formation of prior beliefs in schizophrenia, resulting in psychotic symptoms. One domain in which priors play a crucial role is visual perception. For instance, our perception of brightness, line length, and motion direction are not merely based on a veridical extraction of sensory input but are also determined by expectation (or prior) of the stimulus. Formation of such priors is thought to be governed by the statistical regularities within natural scenes. Recently, the use of such priors has been attributed to a specific set of well-documented visual illusions, supporting the idea that perception is biased toward what is statistically more probable within the environment. The Predictive Coding account of psychosis proposes that patients form abnormal representations of statistical regularities in natural scenes, leading to altered perceptual experiences. Here we use classical vision experiments involving a specific set of visual illusions to directly test this hypothesis. We find that perceptual judgments for both patients and control participants are biased in accordance with reported probability distributions of natural scenes. Thus, despite there being a suggested link between visual abnormalities and psychotic symptoms in schizophrenia, our results provide no support for the notion that altered formation of priors is a general feature of the disorder. These data call for a refinement in the predictions of quantitative models of psychosis.

Intact prioritisation of unconscious face processing in schizophrenia

INTRODUCTION

Faces provide a rich source of social information, crucial for the successful navigation of daily social interactions. People with schizophrenia suffer a wide range of social-cognitive deficits, including abnormalities in face perception. However, to date, studies of face perception in schizophrenia have primarily employed tasks that require patients to make judgements about the faces. It is, thus, unclear whether the reported deficits reflect an impairment in encoding visual face information, or biased social-cognitive evaluative processes.

METHODS

We assess the integrity of early unconscious face processing in 21 out-patients diagnosed with Schizophrenia or Schizoaffective Disorder (15M/6F) and 21 healthy controls (14M/7F). In order to control for any direct influence of higher order cognitive processes, we use a behavioural paradigm known as breaking continuous flash suppression (b-CFS), where participants simply respond to the presence and location of a face. In healthy adults, this method has previously been used to show that upright faces gain rapid and privileged access to conscious awareness over inverted faces and other inanimate objects.

RESULTS

Here, we report similar effects in patients, suggesting that the early unconscious stages of face processing are intact in schizophrenia.

CONCLUSION

Our data indicate that face processing deficits reported in the literature must manifest at a conscious stage of processing, where the influence of mentalizing or attribution biases might play a role.

Responding to joint attention bids in schizophrenia: An interactive eye-tracking study

This study investigated social cognition in schizophrenia using a virtual reality paradigm to capture the dynamic processes of evaluating and responding to eye gaze as an intentional communicative cue. A total of 21 patients with schizophrenia and 21 age-, gender-, and IQ-matched healthy controls completed an interactive computer game with an on-screen avatar that participants believed was controlled by an off-screen partner. On social trials, participants were required to achieve joint attention by correctly interpreting and responding to gaze cues. Participants also completed non-social trials in which they responded to an arrow cue within the same task context. While patients and controls took equivalent time to process communicative intent from gaze shifts, patients made significantly more errors than controls when responding to the directional information conveyed by gaze, but not arrow, cues. Despite no differences in response times to gaze cues between groups, patients were significantly slower than controls when responding to arrow cues. This is the opposite pattern of results previously observed in autistic adults using the same task and suggests that, despite general impairments in attention orienting or oculomotor control, patients with schizophrenia demonstrate a facilitation effect when responding to communicative gaze cues. Findings indicate a hyper-responsivity to gaze cues of communicative intent in schizophrenia. The possible effects of self-referential biases when evaluating gaze direction are discussed, as are clinical implications.

Adaptive sensory coding of gaze direction in schizophrenia

Schizophrenia has been associated with differences in how the visual system processes sensory input. A fundamental mechanism that regulates sensory processing in the brain is gain control, whereby the responses of sensory neurons to a given stimulus are modulated in accordance with the spatial and temporal context. Some studies indicate an impairment of certain cortical gain control mechanisms in schizophrenia in low-level vision, reflected, for instance, in how the visual appearance of a stimulus is affected by the presence of other stimuli around it. In the present study, we investigated higher-level, social vision in schizophrenia, namely the perception of other people's direction of gaze (i.e. a type of face processing). Recent computational modelling work indicates that perceptual aftereffects-changes in perception that occur following repeated exposure to faces that display a specific direction of gaze-are indicative of two distinct forms of gain control involved in the coding of gaze direction across sensory neurons. We find that individuals with schizophrenia display strong perceptual aftereffects following repeated exposure to faces with averted gaze, and a modelling analysis indicates similarly robust gain control in the form of (i) short-term adjustment of channel sensitivities in response to the recent sensory history and (ii) divisive normalization of the encoded gaze direction. Together, this speaks to the typical coding of other people's direction of gaze in the visual system in schizophrenia, including flexible gain control, despite the social-cognitive impairments that can occur in this condition.