Preliminary Study of Gaze Toward Humans in Photographs by Individuals with Autism, Down Syndrome, or Other Intellectual Disabilities: Implications for Design of Visual Scene Displays

Visual fixation patterns to AAC displays are significantly correlated with motor selection for individuals with Down syndrome or individuals on the autism spectrum

Eye tracking research technologies are often used to study how individuals attend visually to different types of AAC displays (e.g. visual scene displays, grid displays). The assumption is that efficiency of visual search may relate to efficiency of motor selection necessary for communication via aided AAC; however, this assumption has not received direct empirical study. We examined the relation between speed of visual search and speed of motor selection of symbols. Ten individuals on the autism spectrum (AS; Study 1) and nine with Down syndrome (DS; Study 2) participated in a search task using simulated AAC displays with a main visual scene display (VSD) and a navigation bar of thumbnail VSDs. Participants were given an auditory prompt to find one of four thumbnail VSDs in the navigation bar. Eye tracking technologies measured how long it took participants to fixate visually on the thumbnail VSD, and recorded how long it took participants to select the thumbnail VSD with a finger. A statistically significant relationship emerged between visual fixation and selection latencies, confirming the positive relationship between visual processing and motor selection for both groups of participants. Eye tracking data may serve as a useful proxy measure for evaluating how display design influences selection of AAC symbols, especially when individuals are unwilling or unable to comply with traditional behaviorally-based assessment tasks.

The use of eye-tracking technology as a tool to evaluate social cognition in people with an intellectual disability: a systematic review and meta-analysis

BACKGROUND

Relatively little is known about social cognition in people with intellectual disability (ID), and how this may support understanding of co-occurring autism. A limitation of previous research is that traditional social-cognitive tasks place a demand on domain-general cognition and language abilities. These tasks are not suitable for people with ID and lack the sensitivity to detect subtle social-cognitive processes. In autism research, eye-tracking technology has offered an effective method of evaluating social cognition-indicating associations between visual social attention and autism characteristics. The present systematic review synthesised research which has used eye-tracking technology to study social cognition in ID. A meta-analysis was used to explore whether visual attention on socially salient regions (SSRs) of stimuli during these tasks correlated with degree of autism characteristics presented on clinical assessment tools.

METHOD

Searches were conducted using four databases, research mailing lists, and citation tracking. Following in-depth screening and exclusion of studies with low methodological quality, 49 articles were included in the review. A correlational meta-analysis was run on Pearson's r values obtained from twelve studies, reporting the relationship between visual attention on SSRs and autism characteristics.

RESULTS AND CONCLUSIONS

Eye-tracking technology was used to measure different social-cognitive abilities across a range of syndromic and non-syndromic ID groups. Restricted scan paths and eye-region avoidance appeared to impact people's ability to make explicit inferences about mental states and social cues. Readiness to attend to social stimuli also varied depending on social content and degree of familiarity. A meta-analysis using a random effects model revealed a significant negative correlation (r = -.28, [95% CI -.47, -.08]) between visual attention on SSRs and autism characteristics across ID groups. Together, these findings highlight how eye-tracking can be used as an accessible tool to measure more subtle social-cognitive processes, which appear to reflect variability in observable behaviour. Further research is needed to be able to explore additional covariates (e.g. ID severity, ADHD, anxiety) which may be related to visual attention on SSRs, to different degrees within syndromic and non-syndromic ID groups, in order to determine the specificity of the association with autism characteristics.

Strategies for highlighting items within visual scene displays to support augmentative and alternative communication access for those with physical impairments

PURPOSE

In contrast to the traditional grid-based display, visual scene displays (VSDs) offer a new paradigm for aided communication. For individuals who cannot select items from an AAC display by direct selection due to physical impairments, AAC access can be supported via methods such as item scanning. Item scanning sequentially highlights items on a display until the individual signals for selection. How items are highlighted or scanned for AAC access can impact performance outcomes. Further, the effectiveness of a VSD interface may be enhanced through consultation with experts in visual communication. Therefore, to support AAC access for those with physical impairments, the aim of this study was to evaluate the perspectives of experts in visual communication regarding effective methods for highlighting VSD elements.

METHODS

Thirteen participants with expertise related to visual communication (e.g., photographers, artists) completed semi-structured interviews regarding techniques for item highlighting.

RESULTS

Study findings identified four main themes to inform how AAC items may be highlighted or scanned, including (1) use of contrast related to light and dark, (2) use of contrast as it relates to colour, (3) outline highlighting, and (4) use of scale and motion.

CONCLUSION

By identifying how compositional techniques can be utilized to highlight VSD elements, study findings may inform current practice for scanning-based AAC access, along with other selection techniques where feedback or highlighting is used (e.g., eye-gaze, brain-computer interface). Further, avenues for just-in-time programming are discussed to support effective implementation for those with physical impairments.IMPLICATIONS FOR REHABILITATIONFindings identify multiple potential techniques to improve scanning through items in a photograph for individuals with severe motor impairments using alternative access strategies.Study findings inform current practice for scanning-based AAC access, along with other selection techniques where feedback or highlighting is used (e.g., eye-gaze, brain-computer interface).Avenues for just in time programming of AAC displays are discussed to decrease programming demands and support effective implementation of study findings.

Demands Associated With an Augmentative and Alternative Communication System in Relation to Alternative Forms of Access for Individuals With Motor Impairments

PURPOSE

Successful adoption and use of aided augmentative and alternative communication (AAC) must address how to promote fluid, efficient, and organized execution of the motor behavior needed to access the AAC system. A substantial number of people who use AAC have significant motor impairments and require "alternative" access options, such as eye gaze or switch use. Such individuals may be particularly vulnerable to interference from a poorly designed system. However, the inherent demands of alternative access methods have received little direct study. The goal of this tutorial is to offer a clinically and theoretically guided framework for considerations concerning AAC access, with the hope of spurring further discussion and empirical research.

METHOD

A framework that draws upon dynamic systems theory was used to illustrate the interactions between the various elements of importance to AAC access. Information and research from the fields of motor learning, developmental dynamic systems theory, AAC, and assistive technology was integrated into this tutorial to examine their applications for alternative AAC access methods.

RESULTS

The framework illustrated that AAC access involves a complex coordination between individual skills, the demands of the communication environment, the activity being undertaken, and the supports and demands inherent in the AAC system itself.

CONCLUSIONS

Awareness of the many demands that alternative forms of access place on the person who uses them can guide clinicians during assessment and intervention decision making regarding access options for individuals with significant motor impairments. Specific directions for future research are considered.

Preference and visual cognitive processing demands of alphabetic and QWERTY keyboards of individuals with and without brain injury

Following a severe brain injury (BI), some literate individuals who require augmentative and alternative communication (AAC) strategies to support communication may benefit from the use of onscreen keyboards to generate text. A range of layouts are available to these individuals within specialized communication software. However, a paucity of information is available to describe user preferences, user perceptions, as well as the visual-cognitive processing demands of such layouts. Such information is critical to guide clinical decision-making for keyboard selection and to provide patient-centered services. This study: (a) described the preferences and perceptions of two onscreen keyboard layouts (QWERTY and alphabetic) and (b) used eye-tracking analysis to investigate the visual-cognitive processing demands between these onscreen keyboards for individuals with and without BI. Results indicated participants in both groups held a strong preference for QWERTY keyboard interfaces and had extensive prior experience using the QWERTY keyboard layout on mobile devices. Eye-tracking analysis revealed less visual-cognitive processing demands using a QWERTY keyboard layout for both groups but were only statistically significant for those without BI. Results suggest that use of a keyboard layout that aligns with client preferences and prior experiences (i.e., the QWERTY keyboard for these participants) may lead to increased satisfaction with the communication experience and increased communication efficiency.

Judicious Arrangement of Symbols on a Simulated Augmentative and Alternative Communication Display Optimizes Visual Attention by Individuals With Down Syndrome

PURPOSE

Aided augmentative and alternative communication (AAC) displays are often designed as symmetrical row-column grids, with each square in the grid containing a symbol. To maximize vocabulary on displays, symbols are often placed close to one another, and background color cuing is used to signal/differentiate symbols across different grammatical categories. However, from a visual and developmental standpoint, these display features (close-set symbols and use of background color cues) may not be optimal. In particular, placing symbols quite close together may result in visual crowding, in which individual symbols cannot be distinguished due to the presence of many neighbors, or flankers. This research sought to examine the role of display arrangement and background color cuing on the efficiency of visual attention during search.

METHOD

Ten adolescents and adults with Down syndrome underwent a visual search task while a research-based eye tracking system recorded their patterns of visual attention. Participants searched for symbol targets on displays with varying levels of visual crowding and background color cuing.

RESULTS

Spatial arrangements that reduced visual crowding and that used the spatial organization to cue the grammatical category of symbols resulted in significantly fewer fixations to nonrelevant distracters during search and reduced the likelihood of fixations away from the target once it was located. Background color was helpful in reducing the latency to find the target.

DISCUSSION

Spatial cues may offer a powerful means to maximize the efficiency of search within AAC displays. Background color cuing may facilitate speed to locate targets in these older individuals. Implications for AAC design, as well as future avenues for maximizing (growing) vocabulary, are discussed.

What's in a Photograph? The Perspectives of Composition Experts on Factors Impacting Visual Scene Display Complexity for Augmentative and Alternative Communication and Strategies for Improving Visual Communication

Purpose Visual scene displays (VSDs) can support augmentative and alternative communication (AAC) success for children and adults with complex communication needs. Static VSDs incorporate contextual photographs that include meaningful events, places, and people. Although the processing of VSDs has been studied, their power as a medium to effectively convey meaning may benefit from the perspective of individuals who regularly engage in visual storytelling. The aim of this study was to evaluate the perspectives of individuals with expertise in photographic and/or artistic composition regarding factors contributing to VSD complexity and how to limit the time and effort required to apply principles of photographic composition. Method Semistructured interviews were completed with 13 participants with expertise in photographic and/or artistic composition. Results Four main themes were noted, including (a) factors increasing photographic image complexity and decreasing cohesion, (b) how complexity impacts the viewer, (c) composition strategies to decrease photographic image complexity and increase cohesion, and (d) strategies to support the quick application of composition strategies in a just-in-time setting. Findings both support and extend existing research regarding best practice for VSD design. Conclusions Findings provide an initial framework for understanding photographic image complexity and how it differs from drawn AAC symbols. Furthermore, findings outline a toolbox of composition principles that may help limit VSD complexity, along with providing recommendations for AAC development to support the quick application of compositional principles to limit burdens associated with capturing photographic images. Supplemental Material https://doi.org/10.23641/asha.15032700.

Clinical and Research Perspectives on Visual Scene Displays

Purpose The purposes of this study were to measure the perceptions of speech-language pathologists (SLPs) and researchers regarding the design of visual scene displays (VSDs) and to investigate the perceived difficulties of SLPs in regard to the acquisition and programming of VSDs. Method We recruited 270 SLPs and nine researchers who focus on VSDs to complete an online survey. The SLP survey addressed both perception of VSD design and difficulties associated with implementing these images as communication supports in the clinical realm. The researcher group was surveyed solely regarding their opinions associated with VSD design. Results SLPs and researchers both ranked high-context VSDs as more effective at supporting communication than low-context VSDs. In addition, both groups ranked VSDs with task-engaged people as more effective at supporting communication than VSDs with camera-engaged people or no people. SLPs reported difficulties with acquiring high-quality VSDs from online sources as well as difficulties with maintenance and upkeep of VSDs. In addition, a large portion of SLPs reported challenges with representing complex concepts through VSDs. Conclusions Results from this study provide evidence of general agreement between SLPs and researchers regarding perception of VSD design. SLPs continue to report challenges with the acquisition and programming of VSDs. Further research is needed to develop evidence-based guidelines to support clinical endeavors in these areas.

Visual Attention to Cued Targets in Simulated Aided Augmentative and Alternative Communication Displays for Individuals With Intellectual and Developmental Disabilities

Purpose Many aided augmentative and alternative communication (AAC) systems require the use of an external display that is represented via a visual modality. It is critical to evaluate and understand visual-perceptual processing in individuals with disabilities who could benefit from AAC. One way to evaluate how individuals process visual materials is through research-based automated eye-tracking technologies that obtain a fine-grained stream of data concerning gaze paths of visual attention. Method The current study examined how individuals with autism spectrum disorder (n = 13), Down syndrome (n = 13), intellectual and developmental disabilities (n = 9), or typical development (n = 20) responded to a spoken prompt to find a thumbnail-sized navigation key within a complex AAC display, including a main visual scene display (VSD) and a navigation bar of four thumbnail-sized VSDs. Stimuli were presented on a monitor containing automated eye-tracking research technology that recorded patterns of visual attention. Results Participants across groups spent more time fixating on a target thumbnail VSD navigation image after the presentation of the spoken cue to look at the target, compared to before the presentation of the spoken cue; they also spent more time looking at the target thumbnail VSD than the other thumbnail-sized VSDs in the navigation bar after the cue. Discussion Participants were able to locate the target thumbnail VSDs, even within the context of a visually complex AAC display. Implications for the design of AAC displays and for assessment of comprehension are discussed.

Personalization of Visual Scene Displays: Preliminary Investigations of Adults with Aphasia, Typical Females across the Age-Span, and Young Adult Males and Females

Visual scene displays (VSDs) are becoming an increasingly popular method of message representation within augmentative and alternative communication (AAC) supports; however, design factors can influence the effectiveness of these images as communication supports. One issue that has come to light in recent years is the fact that selecting personalized VSDs, which depict the person with complex communication needs or an individual with whom they are familiar, are preferred over generic VSDs, which depict unfamiliar individuals. Although personalization is likely an important factor in the usability of VSDs, these images may be difficult for clinicians to obtain. As such, compromises must be identified. The purpose of this study was to explore the effects of controlling personal relevance factors (i.e., age and gender of the people depicted in generic VSDs) on the image preference patterns of adults with and without aphasia. Results from three very preliminary study summaries indicate that gender and age are both mitigating factors in image preference, as males tended to indicate preference for VSDs containing males over those containing females. In addition, females tended to indicate preference for females of a similar age depicted in VSDs.

Speech-Language pathologists' decisions when designing an aided AAC display for a compilation case study of a beginning communicator

PURPOSE

The current study sought to identify decisions speech-language pathologists (SLPs) make and their rationale for those decisions when designing an AAC display for a hypothetical school-aged child who was a beginning communicator. Augmentative and alternative communication (AAC) displays support communicative interactions for children who are unable to use spoken language to meet their communication needs. Children who are at the early stages of learning language, often termed beginning or emerging communicators, may rely heavily on the AAC display to support comprehension and expression. Thus, it is important for speech-language pathologists (SLPs) to design a display that facilitates beginning communicators' communication development.

METHODS

This paper reports on the answers related to a specific case study presented in an online survey completed by 77 SLPs with experience in AAC.

RESULTS

Data illustrated design decisions driven by child-specific characteristics and informed by research regarding type of layout, representation, and the importance of various design features.

CONCLUSIONS

Suggestions for future research are provided, including identifying goals and communicative functions best supported by different types of displays, clinicians' challenges/successes growing a system with the child, and determining the importance of symbol arrangement and organisation in communicative tasks by AAC users.Implications for RehabilitationIdentification of current practices reported by speech-language pathologists given a case study to develop and implement future research.A gap between research and practice suggests the need for better dissemination and application of research supporting the inclusion of people engaged in the event depicted in a visual scene display.Foundations for future research to provide guidance regarding best design of and communication goals supported by visual scene displays and grid-based displays.

Eye Tracking Measures Reveal How Changes in the Design of Displays for Augmentative and Alternative Communication Influence Visual Search in Individuals With Down Syndrome or Autism Spectrum Disorder

Purpose This research note reports on how small changes to the organization of a simulated display for augmentative and alternative communication influence the visual search patterns of individuals with Down syndrome or autism, as measured through eye tracking technologies. Prior research had demonstrated that clustering symbols by their internal color facilitates search and reduces attention to distracters, in children with typical development. This research systematically replicated the procedures with individuals with Down syndrome or autism spectrum disorder. Method Participants engaged in a visual search task on a monitor with embedded automated eye tracking technology. Patterns of gaze during search were measured via this technology. Results Participants were significantly faster to fixate on the target and to select it with the mouse when the like-colored symbols were clustered together. In addition, participants were significantly less likely to fixate on distracters in the clustered condition. No group differences were found. Conclusions Small changes to the organization of the simulated augmentative and alternative communication display resulted in substantial differences in eye gaze and speed to find a target. Of greatest clinical import is the finding that clustering symbols reduced attention to distracters, given that individuals with disabilities may be prone to distraction.

What Can Eye Movements Tell Us about Subtle Cognitive Processing Differences in Autism?

Autism spectrum disorder (ASD) is neurodevelopmental condition principally characterised by impairments in social interaction and communication, and repetitive behaviours and interests. This article reviews the eye movement studies designed to investigate the underlying sampling or processing differences that might account for the principal characteristics of autism. Following a brief summary of a previous review chapter by one of the authors of the current paper, a detailed review of eye movement studies investigating various aspects of processing in autism over the last decade will be presented. The literature will be organised into sections covering different cognitive components, including language and social communication and interaction studies. The aim of the review will be to show how eye movement studies provide a very useful on-line processing measure, allowing us to account for observed differences in behavioural data (accuracy and reaction times). The subtle processing differences that eye movement data reveal in both language and social processing have the potential to impact in the everyday communication domain in autism.

Designing effective AAC displays for individuals with developmental or acquired disabilities: State of the science and future research directions

This paper reviews research on the impact of AAC display variables on visual attention and performance of children with developmental disabilities and adults with acquired conditions, and considers implications for designing effective visual scene displays (VSDs) or grids. When using VSDs with children with developmental disabilities or adults with acquired conditions, research supports the use of personalized photo VSDs that include familiar people engaged in meaningful activities, with navigation bars with thumbnail VSDs, located adjacent to the main VSD. Adults with acquired conditions seem to benefit from the inclusion of text boxes adjacent to the scene. Emerging evidence supports the use of motion to capture visual attention to VSDs (video VSDs) or to specific elements in VSDs. When using grid displays with children with developmental disabilities, research supports the use of spatial cues and clustering based on internal symbol colour to facilitate visual searching and selection. Background colour does not seem to facilitate searching for symbols on smaller displays, and may actually distract children from processing the meaningful components of symbols. Preliminary research suggests that the organization of onscreen keyboards and the number, types, and pairings of symbols in grids may impact performance of adults with acquired conditions. Directions for future research are discussed.

Gaze Toward Naturalistic Social Scenes by Individuals With Intellectual and Developmental Disabilities: Implications for Augmentative and Alternative Communication Designs

PURPOSE

A striking characteristic of the social communication deficits in individuals with autism is atypical patterns of eye contact during social interactions. We used eye-tracking technology to evaluate how the number of human figures depicted and the presence of sharing activity between the human figures in still photographs influenced visual attention by individuals with autism, typical development, or Down syndrome. We sought to examine visual attention to the contents of visual scene displays, a growing form of augmentative and alternative communication support.

METHOD

Eye-tracking technology recorded point-of-gaze while participants viewed 32 photographs in which either 2 or 3 human figures were depicted. Sharing activities between these human figures are either present or absent. The sampling rate was 60 Hz; that is, the technology gathered 60 samples of gaze behavior per second, per participant. Gaze behaviors, including latency to fixate and time spent fixating, were quantified.

RESULTS

The overall gaze behaviors were quite similar across groups, regardless of the social content depicted. However, individuals with autism were significantly slower than the other groups in latency to first view the human figures, especially when there were 3 people depicted in the photographs (as compared with 2 people). When participants' own viewing pace was considered, individuals with autism resembled those with Down syndrome.

CONCLUSION

The current study supports the inclusion of social content with various numbers of human figures and sharing activities between human figures into visual scene displays, regardless of the population served. Study design and reporting practices in eye-tracking literature as it relates to autism and Down syndrome are discussed.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.6066545.

The effect of human engagement depicted in contextual photographs on the visual attention patterns of adults with traumatic brain injury

Photographs are a frequently employed tool for the rehabilitation of adults with traumatic brain injury (TBI). Speech-language pathologists (SLPs) working with these individuals must select photos that are easily identifiable and meaningful to their clients. In this investigation, we examined the visual attention response to camera- (i.e., depicted human figure looking toward camera) and task-engaged (i.e., depicted human figure looking at and touching an object) contextual photographs for a group of adults with TBI and a group of adults without neurological conditions. Eye-tracking technology served to accurately and objectively measure visual fixations. Although differences were hypothesized given the cognitive deficits associated with TBI, study results revealed little difference in the visual fixation patterns of adults with and without TBI. Specifically, both groups of participants tended to fixate rapidly on the depicted human figure and fixate more on objects in which a human figure was task-engaged than when a human figure was camera-engaged. These results indicate that strategic placement of human figures in a contextual photograph may modify the way in which individuals with TBI visually attend to and interpret photographs. In addition, task-engagement appears to have a guiding effect on visual attention that may be of benefit to SLPs hoping to select more effective contextual photographs for their clients with TBI. Finally, the limited differences in visual attention patterns between individuals with TBI and their age and gender matched peers without neurological impairments indicates that these two groups find similar photograph regions to be worthy of visual fixation.

LEARNING OUTCOMES

Readers will gain knowledge regarding the photograph selection process for individuals with TBI. In addition, readers will be able to identify camera- and task-engaged photographs and to explain why task-engagement may be a beneficial component of contextual photographs.

Effect of Message Type on the Visual Attention of Adults With Traumatic Brain Injury

PURPOSE

The purpose of this investigation was to measure the effect of message type (i.e., action, naming) on the visual attention patterns of individuals with and without traumatic brain injury (TBI) when viewing grids composed of 3 types of images (i.e., icons, decontextualized photographs, and contextualized photographs).

METHOD

Fourteen adults with TBI and 14 without TBI-assigned either to an action or naming message condition-viewed grids composed of 3 different image types. Participants' task was to select/sustain visual fixation on the image they felt best represented a stated message (i.e., action or naming).

RESULTS

With final fixation location serving as a proxy for selection, participants in the naming message condition selected decontextualized photographs significantly more often than the other 2 image types. Participants in the action message condition selected contextualized photographs significantly more frequently than the other 2 image types. Minimal differences were noted between participant groups.

CONCLUSIONS

This investigation provides preliminary evidence of the relationship between image and message type. Clinicians involved in the selection of images used for message representation should consider the message being represented when designing supports for people with TBI. Further research is necessary to fully understand the relationship between images and message type.

Social Referencing Gaze Behavior During a Videogame Task: Eye Tracking Evidence from Children With and Without ASD

The purpose of this study was to understand the social referencing behaviors of children with and without autism spectrum disorder (ASD) while visually attending to a videogame stimulus depicting both the face of the videogame player and the videogame play action. Videogames appear to offer a uniquely well-suited environment for the emergence of friendships, but it is not known if children with and without ASD attend to and play videogames similarly. Eyetracking technology was used to investigate visual attention of participants matched based on chronological age. Parametric and nonparametric statistical analyses were used and results indicated the groups did not differ on percentage of time spent visually attending to any of the areas of interest, with one possible exception.

A Comparison of the Visual Attention Patterns of People With Aphasia and Adults Without Neurological Conditions for Camera-Engaged and Task-Engaged Visual Scenes

PURPOSE

The purpose of the study was to compare the visual attention patterns of adults with aphasia and adults without neurological conditions when viewing visual scenes with 2 types of engagement.

METHOD

Eye-tracking technology was used to measure the visual attention patterns of 10 adults with aphasia and 10 adults without neurological conditions. Participants viewed camera-engaged (i.e., human figure facing camera) and task-engaged (i.e., human figure looking at and touching an object) visual scenes.

RESULTS

Participants with aphasia responded to engagement cues by focusing on objects of interest more for task-engaged scenes than camera-engaged scenes; however, the difference in their responses to these scenes were not as pronounced as those observed in adults without neurological conditions. In addition, people with aphasia spent more time looking at background areas of interest and less time looking at person areas of interest for camera-engaged scenes than did control participants.

CONCLUSIONS

Results indicate people with aphasia visually attend to scenes differently than adults without neurological conditions. As a consequence, augmentative and alternative communication (AAC) facilitators may have different visual attention behaviors than the people with aphasia for whom they are constructing or selecting visual scenes. Further examination of the visual attention of people with aphasia may help optimize visual scene selection.

Vision, spatial cognition and intellectual disability

Vision is the most synthetic sensory channel and it provides specific information about the relative position of distant landmarks during visual exploration. In this paper we propose that visual exploration, as assessed by the recording of eye movements, offers an original method to analyze spatial cognition and to reveal alternative adaptation strategies in people with intellectual disabilities (ID). Our general assumption is that eye movement exploration may simultaneously reveal whether, why, and how, compensatory strategies point to specific difficulties related to neurological symptoms. An understanding of these strategies will also help in the development of optimal rehabilitation procedures.

N400 brain responses to spoken phrases paired with photographs of scenes: implications for visual scene displays in AAC systems

Augmentative and alternative communication (AAC) systems are often implemented for individuals whose speech cannot meet their full communication needs. One type of aided display is called a Visual Scene Display (VSD). VSDs consist of integrated scenes (such as photographs) in which language concepts are embedded. Often, the representations of concepts on VSDs are perceptually similar to their referents. Given this physical resemblance, one may ask how well VSDs support development of symbolic functioning. We used brain imaging techniques to examine whether matches and mismatches between the content of spoken messages and photographic images of scenes evoke neural activity similar to activity that occurs to spoken or written words. Electroencephalography (EEG) was recorded from 15 college students who were shown photographs paired with spoken phrases that were either matched or mismatched to the concepts embedded within each photograph. Of interest was the N400 component, a negative deflecting wave 400 ms post-stimulus that is considered to be an index of semantic functioning. An N400 response in the mismatched condition (but not the matched) would replicate brain responses to traditional linguistic symbols. An N400 was found, exclusively in the mismatched condition, suggesting that mismatches between spoken messages and VSD-type representations set the stage for the N400 in ways similar to traditional linguistic symbols.

Eye tracking research to answer questions about augmentative and alternative communication assessment and intervention

Recently, eye tracking technologies (i.e., technologies that automatically track the point of an individual's gaze while that person views or interacts with a visual image) have become available for research purposes. Based on the sampling of the orientation of the individual's eyes, researchers can quantify which locations within the visual image were fixated (viewed), for how long, and how many times. These automated eye tracking research technologies open up a wealth of avenues for investigating how individuals with developmental or acquired communication disabilities may respond to aided augmentative and alternative communication (AAC) systems. In this paper, we introduce basic terminology and explore some of the special challenges of conducting eye tracking research with populations with disabilities who might use AAC, including challenges of inferring attention from the presence of fixation and challenges related to calibration that may result from participant characteristics, behavioral idiosyncracies, and/or the number of calibration points. We also examine how the technology can be applied to ask well-structured experimental questions that have direct clinical relevance, with a focus on the unique contributions that eye tracking research can provide by (a) allowing evaluation of skills in individuals who are difficult to assess via traditional methods, and (b) facilitating access to information on underlying visual cognitive processes that is not accessible via traditional behavioral measures.