Smartphone-based gaze estimation for in-home autism research

Atypical gaze patterns are a promising biomarker of autism spectrum disorder. To measure gaze accurately, however, it typically requires highly controlled studies in the laboratory using specialized equipment that is often expensive, thereby limiting the scalability of these approaches. Here we test whether a recently developed smartphone-based gaze estimation method could overcome such limitations and take advantage of the ubiquity of smartphones. As a proof-of-principle, we measured gaze while a small sample of well-assessed autistic participants and controls watched videos on a smartphone, both in the laboratory (with lab personnel) and in remote home settings (alone). We demonstrate that gaze data can be efficiently collected, in-home and longitudinally by participants themselves, with sufficiently high accuracy (gaze estimation error below 1° visual angle on average) for quantitative, feature-based analysis. Using this approach, we show that autistic individuals have reduced gaze time on human faces and longer gaze time on non-social features in the background, thereby reproducing established findings in autism using just smartphones and no additional hardware. Our approach provides a foundation for scaling future research with larger and more representative participant groups at vastly reduced cost, also enabling better inclusion of underserved communities.

Comparing data quality from an online and in-person lab sample on dynamic theory of mind tasks

Nearly half the published research in psychology is conducted with online samples, but the preponderance of these studies rely primarily on self-report measures. The current study validated data quality from an online sample on a novel, dynamic task by comparing performance between an in-lab and online sample on two dynamic measures of theory of mind-the ability to infer others' mental states. Theory of mind is a cognitively complex construct that has been widely studied across multiple domains of psychology. One task was based on the show The Office®, and has been previously validated by the authors with in-lab samples. The second was a novel task based on the show Nathan for You®, which was selected to account for familiarity effects associated with The Office. Both tasks measured various dimensions of theory of mind (inferring beliefs, understanding motivations, detecting deception, identifying faux pas, and understanding emotions). The in-person lab samples (N = 144 and 177, respectively) completed the tasks between-subject, whereas the online sample (N = 347 from Prolific Academic) completed them within-subject, with order counterbalanced. The online sample's performance across both tasks was reliable (Cronbach's α = .66). For The Office, the in-person sample outperformed the online sample on some types of theory of mind, but this was driven by their greater familiarity with the show. Indeed, for the relatively unfamiliar show Nathan for You, performance did not differ between the two samples. Together, these results suggest that crowdsourcing platforms elicit reliable performance on novel, dynamic, complex tasks.

Determining Whether Older Adults Use Similar Strategies to Young Adults in Theory of Mind Tasks

OBJECTIVES

Theory of mind-the ability to infer others' mental states-declines over the life span, potentially due to cognitive decline. However, it is unclear whether deficits emerge because older adults use the same strategies as young adults, albeit less effectively, or use different or no strategies. The current study compared the similarity of older adults' theory of mind errors to young adults' and a random model.

METHODS

One hundred twenty older adults (MAge = 74.68 years; 64 female) and 111 young adults (MAge = 19.1; 61 female) completed a novel theory of mind task (clips from an episode of the sitcom The Office®), and a standard measure of cognitive function (Logical Memory II). Monte Carlo resampling estimated the likelihood that older adults' error patterns were more similar to young adults' or a random distribution.

RESULTS

Age deficits emerged on the theory of mind task. Poorer performance was associated with less similarity to young adults' response patterns. Overall, older adults' response patterns were ~2.7 million times more likely to match young adults' than a random model. Critically, one fourth of older adults' errors were more similar to the random distribution. Poorer memory ability contributed to this relationship.

DISCUSSION

Age deficits in theory of mind performance may be driven by a subset of older adults and be related to disparities in strategy use. A certain amount of cognitive ability may be necessary for older adults to engage similar strategies to young adults' during theory of mind.

Atypical gaze patterns in autistic adults are heterogeneous across but reliable within individuals

BACKGROUND

Across behavioral studies, autistic individuals show greater variability than typically developing individuals. However, it remains unknown to what extent this variability arises from heterogeneity across individuals, or from unreliability within individuals. Here, we focus on eye tracking, which provides rich dependent measures that have been used extensively in studies of autism. Autistic individuals have an atypical gaze onto both static visual images and dynamic videos that could be leveraged for diagnostic purposes if the above open question could be addressed.

METHODS

We tested three competing hypotheses: (1) that gaze patterns of autistic individuals are less reliable or noisier than those of controls, (2) that atypical gaze patterns are individually reliable but heterogeneous across autistic individuals, or (3) that atypical gaze patterns are individually reliable and also homogeneous among autistic individuals. We collected desktop-based eye tracking data from two different full-length television sitcom episodes, at two independent sites (Caltech and Indiana University), in a total of over 150 adult participants (N = 48 autistic individuals with IQ in the normal range, 105 controls) and quantified gaze onto features of the videos using automated computer vision-based feature extraction.

RESULTS

We found support for the second of these hypotheses. Autistic people and controls showed equivalently reliable gaze onto specific features of videos, such as faces, so much so that individuals could be identified significantly above chance using a fingerprinting approach from video epochs as short as 2 min. However, classification of participants into diagnostic groups based on their eye tracking data failed to produce clear group classifications, due to heterogeneity in the autistic group.

LIMITATIONS

Three limitations are the relatively small sample size, assessment across only two videos (from the same television series), and the absence of other dependent measures (e.g., neuroimaging or genetics) that might have revealed individual-level variability that was not evident with eye tracking. Future studies should expand to larger samples across longer longitudinal epochs, an aim that is now becoming feasible with Internet- and phone-based eye tracking.

CONCLUSIONS

These findings pave the way for the investigation of autism subtypes, and for elucidating the specific visual features that best discriminate gaze patterns-directions that will also combine with and inform neuroimaging and genetic studies of this complex disorder.

Edge-centric analysis of time-varying functional brain networks with applications in autism spectrum disorder

The interaction between brain regions changes over time, which can be characterized using time-varying functional connectivity (tvFC). The common approach to estimate tvFC uses sliding windows and offers limited temporal resolution. An alternative method is to use the recently proposed edge-centric approach, which enables the tracking of moment-to-moment changes in co-fluctuation patterns between pairs of brain regions. Here, we first examined the dynamic features of edge time series and compared them to those in the sliding window tvFC (sw-tvFC). Then, we used edge time series to compare subjects with autism spectrum disorder (ASD) and healthy controls (CN). Our results indicate that relative to sw-tvFC, edge time series captured rapid and bursty network-level fluctuations that synchronize across subjects during movie-watching. The results from the second part of the study suggested that the magnitude of peak amplitude in the collective co-fluctuations of brain regions (estimated as root sum square (RSS) of edge time series) is similar in CN and ASD. However, the trough-to-trough duration in RSS signal is greater in ASD, compared to CN. Furthermore, an edge-wise comparison of high-amplitude co-fluctuations showed that the within-network edges exhibited greater magnitude fluctuations in CN. Our findings suggest that high-amplitude co-fluctuations captured by edge time series provide details about the disruption of functional brain dynamics that could potentially be used in developing new biomarkers of mental disorders.

Video-evoked fMRI BOLD responses are highly consistent across different data acquisition sites

Naturalistic imaging paradigms, in which participants view complex videos in the scanner, are increasingly used in human cognitive neuroscience. Videos evoke temporally synchronized brain responses that are similar across subjects as well as within subjects, but the reproducibility of these brain responses across different data acquisition sites has not yet been quantified. Here, we characterize the consistency of brain responses across independent samples of participants viewing the same videos in functional magnetic resonance imaging (fMRI) scanners at different sites (Indiana University and Caltech). We compared brain responses collected at these different sites for two carefully matched datasets with identical scanner models, acquisition, and preprocessing details, along with a third unmatched dataset in which these details varied. Our overall conclusion is that for matched and unmatched datasets alike, video-evoked brain responses have high consistency across these different sites, both when compared across groups and across pairs of individuals. As one might expect, differences between sites were larger for unmatched datasets than matched datasets. Residual differences between datasets could in part reflect participant-level variability rather than scanner- or data- related effects. Altogether our results indicate promise for the development and, critically, generalization of video fMRI studies of individual differences in healthy and clinical populations alike.

Visual search: Heritability and association with general intelligence

Visual search guides goal-directed action in humans and many other species, and it has been studied extensively in the past. Yet, no study has investigated the relative contributions of genes and environments to individual differences in visual search performance, or to which extent etiologies are shared with broader cognitive phenotypes. To address this gap, we studied visual search and general intelligence in 156 monozygotic (MZ) and 158 same-sex dizygotic (DZ) twin pairs. We found that different indexes of visual search performance (response latency and visual search efficiency) were moderately heritable. Phenotypic correlations between visual search and intelligence were small-to-moderate, and only a small proportion of the genetic variance in visual search was shared with genetic variance in intelligence. We discuss these findings in the context of the "generalist genes hypothesis" stating that different cognitive functions have a common genetic basis.

Probing the Ni2+ -selective Response of Fluorescent Probe NiSensor-1 with the NiCast Photocaged Complex† ‡

CTEA (N,N-bis[2-(carboxylmethyl)thioethyl]amine) is a mixed donor ligand that has been incorporated into multiple fluorescent sensors such as NiSensor-1 that was reported to be selective for Ni²⁺ . Other metal ions such as Zn²⁺ do not produce an emission response in aqueous solution. To investigate the coordination chemistry and selectivity of this receptor, we prepared NiCast, a photocage containing the CTEA receptor. Cast photocages undergo a photoreaction that decreases electron density on a metal-bound aniline nitrogen atom, which shifts the binding equilibrium toward unbound metal ion. The unique selectivity of CTEA was examined by measuring the binding affinity of NiCast and the CTEA receptor for Ni²⁺ , Zn²⁺ , Cd²⁺ and Cu²⁺ under different conditions. In aqueous solution, Ni²⁺ binds more strongly to the aniline nitrogen atom than Cd²⁺ ; however, in CH₃ CN, the change in affinity virtually disappears. The crystal structure of [Cu(CTEA)], which exhibits a Jahn-Teller-distorted square pyramidal structure, was also analyzed to gain more insight into the underlying coordination chemistry. These studies suggest that the fluorescence selectivity of NiSensor-1 in aqueous solution is due to a stronger interaction between the aniline nitrogen atom and Ni²⁺ compared to other divalent metal ions except Cu²⁺ .

Social cognitive abilities predict unique aspects of older adults' personal social networks

OBJECTIVE

The current study explores whether personal social network characteristics are associated with older adults' memory and/or social cognitive function (e.g., ability to infer other's mental states - theory of mind).

METHODS

120 older adults completed a social network interview, a memory measure, and two core measures of social cognitive functions: emotion recognition and theory of mind.

RESULTS

Variation in memory and social cognitive abilities predicted distinct aspects of older adults' social networks. Having better memory predicted having larger, less dense social networks, but better theory of mind was associated with having at least one acquaintance in the network, and having more heterogeneous social relationships within the network.

DISCUSSION

Together our findings suggest that disparate social cognitive abilities may serve unique functions, facilitating maintenance of beneficial social connections.

Using head-mounted eye tracking to examine visual and manual exploration during naturalistic toy play in children with and without autism spectrum disorder

Multimodal exploration of objects during toy play is important for a child's development and is suggested to be abnormal in children with autism spectrum disorder (ASD) due to either atypical attention or atypical action. However, little is known about how children with ASD coordinate their visual attention and manual actions during toy play. The current study aims to understand if and in what ways children with ASD generate exploratory behaviors to toys in natural, unconstrained contexts by utilizing head-mounted eye tracking to quantify moment-by-moment attention. We found no differences in how 24- to 48-mo children with and without ASD distribute their visual attention, generate manual action, or coordinate their visual and manual behaviors during toy play with a parent. Our findings suggest an intact ability and willingness of children with ASD to explore toys and suggest that context is important when studying child behavior.

Volitional eye movement control and ADHD traits: a twin study

BACKGROUND

Top-down volitional command of eye movements may serve as a candidate endophenotype of ADHD, an important function underlying goal-directed action in everyday life. In this twin study, we examined the relation between performance on a response inhibition eye-tracking paradigm and parent-rated ADHD traits in a population-based twin sample. We hypothesized that altered eye movement control is associated with the severity of ADHD traits and that this association is attributable to genetic factors.

METHODS

A total of 640 twins (320 pairs, 50% monozygotic) aged 9-14 years) from the Child and Adolescent Twin Study in Sweden (CATSS) participated. Twins performed the antisaccade task indexing inhibitory alterations as either direction errors (following exogenous cues rather than instructions) or premature anticipatory eye movements (failure to wait for cues). We calculated the associations of eye movement control and ADHD traits using linear regression mixed-effects models and genetic and environmental influences with multivariate twin models.

RESULTS

Premature anticipatory eye movements were positively associated with inattentive traits (β = .17; 95% CI: 0.04, 0.31), while controlling for hyperactive behaviors and other covariates. Both premature anticipatory eye movements and inattention were heritable (h²  = 0.40, 95% CI: 0.22, 0.56; h²  = 0.55; 95% CI: 0.42, 0.65; respectively), and their genetic correlation was small but statistically significant (r = .19, 95% CI: 0.02, 0.36). However, the genetic correlation did not remain significant after adjusting for covariates (age, sex, hyperactivity traits, IQ). No link was found between direction errors and ADHD traits.

CONCLUSIONS

This study indicates that there is a specific, genetically influenced, relation between top-down eye movement control and the inattentive traits typical of ADHD.

High-amplitude cofluctuations in cortical activity drive functional connectivity

Resting-state functional connectivity is used throughout neuroscience to study brain organization and to generate biomarkers of development, disease, and cognition. The processes that give rise to correlated activity are, however, poorly understood. Here we decompose resting-state functional connectivity using a temporal unwrapping procedure to assess the contributions of moment-to-moment activity cofluctuations to the overall connectivity pattern. This approach temporally resolves functional connectivity at a timescale of single frames, which enables us to make direct comparisons of cofluctuations of network organization with fluctuations in the blood oxygen level-dependent (BOLD) time series. We show that surprisingly, only a small fraction of frames exhibiting the strongest cofluctuation amplitude are required to explain a significant fraction of variance in the overall pattern of connection weights as well as the network's modular structure. These frames coincide with frames of high BOLD activity amplitude, corresponding to activity patterns that are remarkably consistent across individuals and identify fluctuations in default mode and control network activity as the primary driver of resting-state functional connectivity. Finally, we demonstrate that cofluctuation amplitude synchronizes across subjects during movie watching and that high-amplitude frames carry detailed information about individual subjects (whereas low-amplitude frames carry little). Our approach reveals fine-scale temporal structure of resting-state functional connectivity and discloses that frame-wise contributions vary across time. These observations illuminate the relation of brain activity to functional connectivity and open a number of directions for future research.

Difficulties maintaining prolonged fixation and attention-deficit/hyperactivity symptoms share genetic influences in childhood

This study investigated the association between the ability to maintain prolonged (2-minute) fixation on a visual target and ADHD traits in a sample consisting of 120 monozygotic and 120 dizygotic twin pairs, aged 9 to 14 years. More intrusive saccades during the task was associated with higher level of parent-reported ADHD traits. Both intrusive saccades and ADHD symptoms had high heritability estimates, and there was a moderate genetic correlation between number of intrusive saccades and ADHD. This study suggests that inability to maintain ocular fixation for longer times is etiologically linked to ADHD traits in the general population.

Developing Social Communication Skills Using Dual First-Person Video Recording Glasses: A Novel Intervention for Adolescents with Autism

Adolescents with autism often experience pronounced difficulties with social communication, and novel interventions designed to improve core abilities are greatly needed. This study examines if providing immediate video feedback, an extension of video self-modeling, can aid adolescents with autism to self-identify strengths and irregularities from their social interactions. Using multiple baseline design across four participants, individuals engaged in naturalistic conversations wearing video recording glasses. During the intervention, videos were reviewed immediately and participants recognized when they were not following typical social-communicative convention. Based on observational data coded from videos, all four participants modified their behavior during subsequent conversations. Although adolescents with autism may hypothetically know to behave, viewing themselves on video may provide practical cues to support social insight and behavioral change.

Accurate prediction of individual subject identity and task, but not autism diagnosis, from functional connectomes

Despite enthusiasm about the potential for using fMRI-based functional connectomes in the development of biomarkers for autism spectrum disorder (ASD), the literature is full of negative findings-failures to distinguish ASD functional connectomes from those of typically developing controls (TD)-and positive findings that are inconsistent across studies. Here, we report on a new study designed to either better differentiate ASD from TD functional connectomes-or, alternatively, to refine our understanding of the factors underlying the current state of affairs. We scanned individuals with ASD and controls both at rest and while watching videos with social content. Using multiband fMRI across repeat sessions, we improved both data quantity and scanning duration by collecting up to 2 hr of data per individual. This is about 50 times the typical number of temporal samples per individual in ASD fcMRI studies. We obtained functional connectomes that were discriminable, allowing for near-perfect individual identification regardless of diagnosis, and equally reliable in both groups. However, contrary to what one might expect, we did not consistently or robustly observe in the ASD group either reductions in similarity to TD functional connectivity (FC) patterns or shared atypical FC patterns. Accordingly, FC-based predictions of diagnosis group achieved accuracy levels around chance. However, using the same approaches to predict scan type (rest vs. video) achieved near-perfect accuracy. Our findings suggest that neither the limitations of resting state as a "task," data resolution, data quantity, or scan duration can be considered solely responsible for failures to differentiate ASD from TD functional connectomes.

Visual Disengagement: Genetic Architecture and Relation to Autistic Traits in the General Population

Visual disengagement has been hypothesized as an endophenotype for autism. In this study we used twin modelling to assess the role of genetics in basic measures of visual disengagement, and tested their putative association to autistic traits in the general population. We used the Gap Overlap task in a sample of 492 twins. Results showed that most of the covariance among eye movement latencies across conditions was shared and primarily genetic. Further, there were unique genetic contributions to the Gap condition, but not to the Overlap condition-i.e. the one theorized to capture visual disengagement. We found no phenotypic association between autistic traits and disengagement, thus not supporting the hypothesis of visual disengagement as an endophenotype for autistic traits.

Temporal fluctuations in the brain's modular architecture during movie-watching

Brain networks are flexible and reconfigure over time to support ongoing cognitive processes. However, tracking statistically meaningful reconfigurations across time has proven difficult. This has to do largely with issues related to sampling variability, making instantaneous estimation of network organization difficult, along with increased reliance on task-free (cognitively unconstrained) experimental paradigms, limiting the ability to interpret the origin of changes in network structure over time. Here, we address these challenges using time-varying network analysis in conjunction with a naturalistic viewing paradigm. Specifically, we developed a measure of inter-subject network similarity and used this measure as a coincidence filter to identify synchronous fluctuations in network organization across individuals. Applied to movie-watching data, we found that periods of high inter-subject similarity coincided with reductions in network modularity and increased connectivity between cognitive systems. In contrast, low inter-subject similarity was associated with increased system segregation and more rest-like architectures. We then used a data-driven approach to uncover clusters of functional connections that follow similar trajectories over time and are more strongly correlated during movie-watching than at rest. Finally, we show that synchronous fluctuations in network architecture over time can be linked to a subset of features in the movie. Our findings link dynamic fluctuations in network integration and segregation to patterns of inter-subject similarity, and suggest that moment-to-moment fluctuations in functional connectivity reflect shared cognitive processing across individuals.

Nonreplication of functional connectivity differences in autism spectrum disorder across multiple sites and denoising strategies

A rapidly growing number of studies on autism spectrum disorder (ASD) have used resting‐state fMRI to identify alterations of functional connectivity, with the hope of identifying clinical biomarkers or underlying neural mechanisms. However, results have been largely inconsistent across studies, and there remains a pressing need to determine the primary factors influencing replicability. Here, we used resting‐state fMRI data from the Autism Brain Imaging Data Exchange to investigate two potential factors: denoising strategy and data site (which differ in terms of sample, data acquisition, etc.). We examined the similarity of both group‐averaged functional connectomes and group‐level differences (ASD vs. control) across 33 denoising pipelines and four independently‐acquired datasets. The group‐averaged connectomes were highly consistent across pipelines (r = 0.92 ± 0.06) and sites (r = 0.88 ± 0.02). However, the group differences, while still consistent within site across pipelines (r = 0.76 ± 0.12), were highly inconsistent across sites regardless of choice of denoising strategies (r = 0.07 ± 0.04), suggesting lack of replication may be strongly influenced by site and/or cohort differences. Across‐site similarity remained low even when considering the data at a large‐scale network level or when considering only the most significant edges. We further show through an extensive literature survey that the parameters chosen in the current study (i.e., sample size, age range, preprocessing methods) are quite representative of the published literature. These results highlight the importance of examining replicability in future studies of ASD, and, more generally, call for extra caution when interpreting alterations in functional connectivity across groups of individuals.

Visual Search Performance Does Not Relate to Autistic Traits in the General Population

Autism spectrum disorder (ASD) is commonly conceived as the extreme end of a continuum. Research suggests that autistic individuals outperform typically developing controls in visual search. Thus, enhanced visual search may represent an adaptive trait associated with ASD. Here, using a large general population sample (N = 608, aged 9–14 years), we tested if higher levels of autistic traits are associated with enhanced visual search. Visual search was evaluated using both manual responses and eye movements, and autistic traits were measured using the Social Responsiveness Scale. Contrary to our hypothesis, no significant relation between autistic traits and visual search were observed. The theoretical implications of these results are discussed.

Social and nonsocial visual prediction errors in autism spectrum disorder

Impaired predictive coding has been proposed as a framework to explain discrepancies between expectations and outcomes in autism spectrum disorder (ASD) that may contribute to core symptoms of the disorder. However, no eye tracking study has directly addressed this framework in the context of visual predictions of social and nonsocial stimuli. The current study used eye tracking to examine violations of learned visual associations of both social and nonsocial stimuli. Twenty-six adolescents with ASD and 18 typically developing control (TDC) adolescents completed an outcome expectation eye tracking task in which predictive cues correctly (80% of trials) or incorrectly (20% of trials) indicated the location (left or right) of forthcoming social or nonsocial stimuli. During violation trials, individuals with ASD focused their gaze relatively more often on stimuli presented on locations that violated the learned association and less often on locations that corresponded with the learned association. This finding was not moderated by stimulus type (i.e., social vs. nonsocial). Additionally, participants who looked at incorrectly predicted locations more often had significantly greater ASD symptom severity. These results are consistent with theories that characterize ASD as a disorder of prediction and have potential implications for understanding symptoms related to prediction errors in individuals with ASD. Autism Res 2019, 12: 878-883. © 2019 International Society for Autism Research, Wiley Periodicals, Inc. LAY SUMMARY: Individuals with autism spectrum disorder (ASD) exhibit impairments making predictions that may impact learning. In this study, we used eye tracking methodology and found that individuals with ASD were less likely to look at the predicted location when a visual routine was violated. This pattern was evident for both social and nonsocial images and was associated with greater ASD symptom severity. These findings provide additional support for predictive challenges in ASD.

High-accuracy individual identification using a "thin slice" of the functional connectome

Connectome fingerprinting-a method that uses many thousands of functional connections in aggregate to identify individuals-holds promise for individualized neuroimaging. A better characterization of the features underlying successful fingerprinting performance-how many and which functional connections are necessary and/or sufficient for high accuracy-will further inform our understanding of uniqueness in brain functioning. Thus, here we examine the limits of high-accuracy individual identification from functional connectomes. Using ∼3,300 scans from the Human Connectome Project in a split-half design and an independent replication sample, we find that a remarkably small "thin slice" of the connectome-as few as 40 out of 64,620 functional connections-was sufficient to uniquely identify individuals. Yet, we find that no specific connections or even specific networks were necessary for identification, as even small random samples of the connectome were sufficient. These results have important conceptual and practical implications for the manifestation and detection of uniqueness in the brain.

Genetic Influence on Eye Movements to Complex Scenes at Short Timescales

Where one looks within their environment constrains one's visual experiences, directly affects cognitive, emotional, and social processing [1-4], influences learning opportunities [5], and ultimately shapes one's developmental path. While there is a high degree of similarity across individuals with regard to which features of a scene are fixated [6-8], large individual differences are also present, especially in disorders of development [9-13], and clarifying the origins of these differences is essential to understand the processes by which individuals develop within the complex environments in which they exist and interact. Toward this end, a recent paper [14] found that "social visual engagement"-namely, gaze to eyes and mouths of faces-is strongly influenced by genetic factors. However, whether genetic factors influence gaze to complex visual scenes more broadly, impacting how both social and non-social scene content are fixated, as well as general visual exploration strategies, has yet to be determined. Using a behavioral genetic approach and eye tracking data from a large sample of 11-year-old human twins (233 same-sex twin pairs; 51% monozygotic, 49% dizygotic), we demonstrate that genetic factors do indeed contribute strongly to eye movement patterns, influencing both one's general tendency for visual exploration of scene content, as well as the precise moment-to-moment spatiotemporal pattern of fixations during viewing of complex social and non-social scenes alike. This study adds to a now growing set of results that together illustrate how genetics may broadly influence the process by which individuals actively shape and create their own visual experiences.

Deconstructing atypical eye gaze perception in autism spectrum disorder

The ability to discern the target of another person's gaze is critical for social and linguistic development, but functions atypically in autism spectrum disorder (ASD). A multi-pronged approach allowed us to deconstruct this complex ability, to uncover the fundamental bases of this impairment. We analyzed performance on a novel gaze perception task with classical psychophysical metrics (precision and accuracy), principal component analysis (in the analysis of spatial biases), and Bayesian computational modeling (in the analysis of individual subjects' use of contextual salience cues). Compared to controls, adults with ASD were less precise and less accurate in their judgments of gaze direction. Further, although nearly all controls exhibited a prototypical pattern of spatial bias in their judgments, this spatial prior was severely disrupted among a large subset of ASD participants. By contrast, Bayesian computational modeling revealed that both groups exploited contextual salience cues in their gaze judgments, and that the average strength of this contextual prior was similar for both groups. This comprehensive study revealed that although most ASD participants performed atypically in at least one aspect of gaze perception, the particular aspects disrupted varied idiosyncratically across individuals. Impairment in gaze perception in ASD likely arises via heterogeneous underlying mechanisms.

Neurotypical Peers are Less Willing to Interact with Those with Autism based on Thin Slice Judgments

Individuals with autism spectrum disorder (ASD), including those who otherwise require less support, face severe difficulties in everyday social interactions. Research in this area has primarily focused on identifying the cognitive and neurological differences that contribute to these social impairments, but social interaction by definition involves more than one person and social difficulties may arise not just from people with ASD themselves, but also from the perceptions, judgments, and social decisions made by those around them. Here, across three studies, we find that first impressions of individuals with ASD made from thin slices of real-world social behavior by typically-developing observers are not only far less favorable across a range of trait judgments compared to controls, but also are associated with reduced intentions to pursue social interaction. These patterns are remarkably robust, occur within seconds, do not change with increased exposure, and persist across both child and adult age groups. However, these biases disappear when impressions are based on conversational content lacking audio-visual cues, suggesting that style, not substance, drives negative impressions of ASD. Collectively, these findings advocate for a broader perspective of social difficulties in ASD that considers both the individual’s impairments and the biases of potential social partners.

The influence of presentation modality on the social comprehension of naturalistic scenes in adults with autism spectrum disorder

The comprehension of dynamically unfolding social situations is made possible by the seamless integration of multimodal information merged with rich intuitions about the thoughts and behaviors of others. We examined how high-functioning adults with autism spectrum disorder and neurotypical controls made a complex social judgment (i.e. rating the social awkwardness of scenes from a television sitcom) across three conditions that manipulated presentation modality-visual alone, transcribed text alone, or visual and auditory together. The autism spectrum disorder and control groups collectively assigned similar mean awkwardness ratings to individual scenes. However, individual participants with autism spectrum disorder tended to respond more idiosyncratically than controls, assigning ratings that were less correlated with the ratings of the other participants in the sample. We found no evidence that this group difference was isolated to any specific presentation modality. In a comparison condition, we found no group differences when participants instead rated the happiness of characters (a more basic social judgment) in full audiovisual format. Thus, although we observed differences in the manner with which high-functioning adults with autism spectrum disorder make social judgments compared to controls, these group differences may be dependent on the social dimension being judged, rather than the specific modality of presentation.

Prior expectations about where other people are likely to direct their attention systematically influence gaze perception

Different locations in the visual environment vary greatly in terms of how likely they are to draw a person's attention. When inferring the most likely target of another person's gaze, it would therefore be a reasonable strategy to incorporate expectations about the relative visual salience of these various locations, weighing this prior knowledge against incoming social signals (e.g., eye cues). This Bayesian approach to modeling gaze perception has informed computer vision techniques, but whether this model accounts well for human performance remains an untested hypothesis. We present subjects with a "gazer" fixating his eyes on various locations on a two-dimensional surface, and project arbitrary photographic images onto that surface. Subjects judge where the gazer is looking in each image. A full Bayesian model, which takes image salience information into account, fits subjects' gaze judgments better than a reduced model that only considers the perceived direction of the gazer's eyes. Varying the amount of time the subject is allowed to view the gazer reveals that center bias tends to dominate gaze judgments early, whereas salient features specific to the projected image influence judgments at longer viewing durations.

Atypical Visual Saliency in Autism Spectrum Disorder Quantified through Model-Based Eye Tracking

The social difficulties that are a hallmark of autism spectrum disorder (ASD) are thought to arise, at least in part, from atypical attention toward stimuli and their features. To investigate this hypothesis comprehensively, we characterized 700 complex natural scene images with a novel three-layered saliency model that incorporated pixel-level (e.g., contrast), object-level (e.g., shape), and semantic-level attributes (e.g., faces) on 5,551 annotated objects. Compared with matched controls, people with ASD had a stronger image center bias regardless of object distribution, reduced saliency for faces and for locations indicated by social gaze, and yet a general increase in pixel-level saliency at the expense of semantic-level saliency. These results were further corroborated by direct analysis of fixation characteristics and investigation of feature interactions. Our results for the first time quantify atypical visual attention in ASD across multiple levels and categories of objects.

Estimation of the prevalence of autism spectrum disorder in South Korea, revisited

Two-phase designs in epidemiological studies of autism prevalence introduce methodological complications that can severely limit the precision of resulting estimates. If the assumptions used to derive the prevalence estimate are invalid or if the uncertainty surrounding these assumptions is not properly accounted for in the statistical inference procedure, then the point estimate may be inaccurate and the confidence interval may not be a true reflection of the precision of the estimate. We examine these potential pitfalls in the context of a recent high-profile finding by Kim et al. (2011, Prevalence of autism spectrum disorders in a total population sample. American Journal of Psychiatry 168: 904–912), who estimated that autism spectrum disorder affects 2.64% of children in a South Korean community. We reconstructed the study’s methodology and used Monte Carlo simulations to analyze whether their point estimate and 95% confidence interval (1.91%, 3.37%) were reasonable, given what was known about their screening instrument and sample. We find the original point estimate to be highly assumption-dependent, and after accounting for sources of uncertainty unaccounted for in the original article, we demonstrate that a more reasonable confidence interval would be approximately twice as large as originally reported. We argue that future studies should give serious consideration to the additional sources of uncertainty introduced by a two-phase design, which may easily outstrip any expected gains in efficiency.

Idiosyncratic brain activation patterns are associated with poor social comprehension in autism

Autism spectrum disorder (ASD) features profound social deficits but neuroimaging studies have failed to find any consistent neural signature. Here we connect these two facts by showing that idiosyncratic patterns of brain activation are associated with social comprehension deficits. Human participants with ASD (N = 17) and controls (N = 20) freely watched a television situation comedy (sitcom) depicting seminaturalistic social interactions ("The Office", NBC Universal) in the scanner. Intersubject correlations in the pattern of evoked brain activation were reduced in the ASD group-but this effect was driven entirely by five ASD subjects whose idiosyncratic responses were also internally unreliable. The idiosyncrasy of these five ASD subjects was not explained by detailed neuropsychological profile, eye movements, or data quality; however, they were specifically impaired in understanding the social motivations of characters in the sitcom. Brain activation patterns in the remaining ASD subjects were indistinguishable from those of control subjects using multiple multivariate approaches. Our findings link neurofunctional abnormalities evoked by seminaturalistic stimuli with a specific impairment in social comprehension, and highlight the need to conceive of ASD as a heterogeneous classification.

A specific hypoactivation of right temporo-parietal junction/posterior superior temporal sulcus in response to socially awkward situations in autism

People with autism spectrum disorder (ASD) often have difficulty comprehending social situations in the complex, dynamic contexts encountered in the real world. To study the social brain under conditions which approximate naturalistic situations, we measured brain activity with FUNCTIONAL MAGNETIC RESONANCE IMAGING: while participants watched a full-length episode of the sitcom The Office. Having quantified the degree of social awkwardness at each moment of the episode, as judged by an independent sample of controls, we found that both individuals with ASD and control participants showed reliable activation of several brain regions commonly associated with social perception and cognition (e.g. those comprising the 'mentalizing network') during the more awkward moments. However, individuals with ASD showed less activity than controls in a region near right temporo-parietal junction (RTPJ) extending into the posterior end of the right superior temporal sulcus (RSTS). Further analyses suggested that, despite the free-form nature of the experimental design, this group difference was specific to this RTPJ/RSTS area of the mentalizing network; other regions of interest showed similar activity across groups with respect to both location and magnitude. These findings add support to a body of evidence suggesting that RTPJ/RSTS plays a special role in social processes across modalities and may function atypically in individuals with ASD navigating the social world.

Violations of personal space by individuals with autism spectrum disorder

The ability to maintain an appropriate physical distance (i.e., interpersonal distance) from others is a critical aspect of social interaction and contributes importantly to real-life social functioning. In Study 1, using parent-report data that had been acquired on a large number of individuals (ages 4–18 years) for the Autism Genetic Resource Exchange and the Simons Simplex Collection, we found that those with Autism Spectrum Disorder (ASD; n = 766) more often violated the space of others compared to their unaffected siblings (n = 766). This abnormality held equally across ASD diagnostic categories, and correlated with clinical measures of communication and social functioning. In Study 2, laboratory experiments in a sample of high-functioning adults with ASD demonstrated an altered relationship between interpersonal distance and personal space, and documented a complete absence of personal space in 3 individuals with ASD. Furthermore, anecdotal self-report from several participants confirmed that violations of social distancing conventions continue to occur in real-world interactions through adulthood. We suggest that atypical social distancing behavior offers a practical and sensitive measure of social dysfunction in ASD, and one whose psychological and neurological substrates should be further investigated.

Agenesis of the corpus callosum and autism: a comprehensive comparison

The corpus callosum, with its ∼200 million axons, remains enigmatic in its contribution to cognition and behaviour. Agenesis of the corpus callosum is a congenital condition in which the corpus callosum fails to develop; such individuals exhibit localized deficits in non-literal language comprehension, humour, theory of mind and social reasoning. These findings together with parent reports suggest that behavioural and cognitive impairments in subjects with callosal agenesis may overlap with the profile of autism spectrum disorders, particularly with respect to impairments in social interaction and communication. To provide a comprehensive test of this hypothesis, we directly compared a group of 26 adults with callosal agenesis to a group of 28 adults with a diagnosis of autism spectrum disorder but no neurological abnormality. All participants had full-scale intelligence quotient scores >78 and groups were matched on age, handedness, and gender ratio. Using the Autism Diagnostic Observation Schedule together with current clinical presentation to assess autistic symptomatology, we found that 8/26 (about a third) of agenesis subjects presented with autism. However, more formal diagnosis additionally involving recollective parent-report measures regarding childhood behaviour showed that only 3/22 met complete formal criteria for an autism spectrum disorder (parent reports were unavailable for four subjects). We found no relationship between intelligence quotient and autism symptomatology in callosal agenesis, nor evidence that the presence of any residual corpus callosum differentiated those who exhibited current autism spectrum symptoms from those who did not. Relative to the autism spectrum comparison group, parent ratings of childhood behaviour indicated children with agenesis were less likely to meet diagnostic criteria for autism, even for those who met autism spectrum criteria as adults, and even though there was no group difference in parent report of current behaviours. The findings suggest two broad conclusions. First, they support the hypothesis that congenital disruption of the corpus callosum constitutes a major risk factor for developing autism. Second, they quantify specific features that distinguish autistic behaviour associated with callosal agenesis from autism more generally. Taken together, these two findings also leverage specific questions for future investigation: what are the distal causes (genetic and environmental) determining both callosal agenesis and its autistic features, and what are the proximal mechanisms by which absence of the callosum might generate autistic symptomatology?

Largely typical patterns of resting-state functional connectivity in high-functioning adults with autism

A leading hypothesis for the neural basis of autism postulates globally abnormal brain connectivity, yet the majority of studies report effects that are either very weak, inconsistent across studies, or explain results incompletely. Here we apply multiple analytical approaches to resting-state BOLD-fMRI data at the whole-brain level. Neurotypical and high-functioning adults with autism displayed very similar patterns and strengths of resting-state connectivity. We found only limited evidence in autism for abnormal resting-state connectivity at the regional level and no evidence for altered connectivity at the whole-brain level. Regional abnormalities in functional connectivity in autism spectrum disorder were primarily in the frontal and temporal cortices. Within these regions, functional connectivity with other brain regions was almost exclusively lower in the autism group. Further examination showed that even small amounts of head motion during scanning have large effects on functional connectivity measures and must be controlled carefully. Consequently, we suggest caution in the interpretation of apparent positive findings until all possible confounding effects can be ruled out. Additionally, we do not rule out the possibility that abnormal connectivity in autism is evident at the microstructural synaptic level, which may not be reflected sensitively in hemodynamic changes measured with BOLD-fMRI.

The social brain in psychiatric and neurological disorders

Psychiatric and neurological disorders have historically provided key insights into the structure-function relationships that subserve human social cognition and behavior, informing the concept of the 'social brain'. In this review, we take stock of the current status of this concept, retaining a focus on disorders that impact social behavior. We discuss how the social brain, social cognition, and social behavior are interdependent, and emphasize the important role of development and compensation. We suggest that the social brain, and its dysfunction and recovery, must be understood not in terms of specific structures, but rather in terms of their interaction in large-scale networks.

Perception of emotions from facial expressions in high-functioning adults with autism

Impairment in social communication is one of the diagnostic hallmarks of autism spectrum disorders, and a large body of research has documented aspects of impaired social cognition in autism, both at the level of the processes and the neural structures involved. Yet one of the most common social communicative abilities in everyday life, the ability to judge somebody's emotion from their facial expression, has yielded conflicting findings. To investigate this issue, we used a sensitive task that has been used to assess facial emotion perception in a number of neurological and psychiatric populations. Fifteen high-functioning adults with autism and 19 control participants rated the emotional intensity of 36 faces displaying basic emotions. Every face was rated 6 times-once for each emotion category. The autism group gave ratings that were significantly less sensitive to a given emotion, and less reliable across repeated testing, resulting in overall decreased specificity in emotion perception. We thus demonstrate a subtle but specific pattern of impairments in facial emotion perception in people with autism.

A selective role for right insula--basal ganglia circuits in appetitive stimulus processing

Hemispheric lateralization of hedonic evaluation ('liking') and incentive motivation ('wanting') in neural networks connecting the basal ganglia and insula (BG-I) in humans was examined. Participants with brain damage restricted to the BG-I of the right (n = 5) or left (n = 5) hemisphere, and 26 healthy participants matched on age, sex and intelligence quotient were tested on positively and negatively valenced pictures drawn from varied stimulus categories (Vijayaraghavan et al., 2008). Liking was assessed with explicit ratings of pleasantness using a nine-point Likert scale. Wanting was quantified as the amount of work (via repeated keypresses) that participants expended to increase (approach) or decrease (withdraw) viewing time. Right-lesion patients showed abnormally low viewing times and liking ratings for positive images. For a subset of positive images depicting sexual content, right-lesion patients exhibited active withdrawal, while the other two groups approached such stimuli. These results suggest that the right basal ganglia-insula complex plays a greater role than the left in supporting hedonic evaluation and motivational approach to positively valenced stimuli. The finding that active avoidance of stimuli that were not 'liked' was spared in both right- and left-sided lesion subjects suggests that unilateral damage to insula/basal ganglia circuits may not be sufficient to affect general incentive motivation independent of preference.

Reprint of: Impaired fixation to eyes following amygdala damage arises from abnormal bottom-up attention

SM is a patient with complete bilateral amygdala lesions who fails to fixate the eyes in faces and is consequently impaired in recognizing fear (Adolphs et al., 2005). Here we first replicated earlier findings in SM of reduced gaze to the eyes when seen in whole faces. Examination of the time course of fixations revealed that SM's reduced eye contact is particular pronounced in the first fixation to the face, and less abnormal in subsequent fixations. In a second set of experiments, we used a gaze-contingent presentation of faces with real time eye tracking, wherein only a small region of the face is made visible at the center of gaze. In essence, viewers explore the face by moving a small searchlight over the face with their gaze. Under such viewing conditions, SM's fixations to eye region of faces became entirely normalized. We suggest that this effect arises from the absence of bottom-up effects due to the facial features, allowing gaze location to be driven entirely by top-down control. Together with SM's failure to fixate the eyes in whole faces primarily at the very first saccade, the findings suggest that the saliency of the eyes normally attract our gaze in an amygdala-dependent manner. Impaired eye gaze is also a prominent feature of several psychiatric illnesses in which the amygdala has been hypothesized to be dysfunctional, and our findings and experimental manipulation may hold promise for interventions in such populations, including autism and fragile X syndrome.

Impaired fixation to eyes following amygdala damage arises from abnormal bottom-up attention

SM is a patient with complete bilateral amygdala lesions who fails to fixate the eyes in faces and is consequently impaired in recognizing fear (Adolphs et al., 2005). Here we first replicated earlier findings in SM of reduced gaze to the eyes when seen in whole faces. Examination of the time course of fixations revealed that SM's reduced eye contact is particular pronounced in the first fixation to the face, and less abnormal in subsequent fixations. In a second set of experiments, we used a gaze-contingent presentation of faces with real time eye tracking, wherein only a small region of the face is made visible at the center of gaze. In essence, viewers explore the face by moving a small searchlight over the face with their gaze. Under such viewing conditions, SM's fixations to eye region of faces became entirely normalized. We suggest that this effect arises from the absence of bottom-up effects due to the facial features, allowing gaze location to be driven entirely by top-down control. Together with SM's failure to fixate the eyes in whole faces primarily at the very first saccade, the findings suggest that the saliency of the eyes normally attract our gaze in an amygdala-dependent manner. Impaired eye gaze is also a prominent feature of several psychiatric illnesses in which the amygdala has been hypothesized to be dysfunctional, and our findings and experimental manipulation may hold promise for interventions in such populations, including autism and fragile X syndrome.