Gaze distribution analysis and saliency prediction across age groups

Saccade size predicts onset time of object processing during visual search of an open world virtual environment

OBJECTIVE

To date the vast majority of research in the visual neurosciences have been forced to adopt a highly constrained perspective of the vision system in which stimuli are processed in an open-loop reactive fashion (i.e., abrupt stimulus presentation followed by an evoked neural response). While such constraints enable high construct validity for neuroscientific investigation, the primary outcomes have been a reductionistic approach to isolate the component processes of visual perception. In electrophysiology, of the many neural processes studied under this rubric, the most well-known is, arguably, the P300 evoked response. There is, however, relatively little known about the real-world corollary of this component in free-viewing paradigms where visual stimuli are connected to neural function in a closed-loop. While growing evidence suggests that neural activity analogous to the P300 does occur in such paradigms, it is an open question when this response occurs and what behavioral or environmental factors could be used to isolate this component.

APPROACH

The current work uses convolutional networks to decode neural signals during a free-viewing visual search task in a closed-loop paradigm within an open-world virtual environment. From the decoded activity we construct fixation-locked response profiles that enable estimations of the variable latency of any P300 analogue around the moment of fixation. We then use these estimates to investigate which factors best reduce variable latency and, thus, predict the onset time of the response. We consider measurable, search-related factors encompassing top-down (i.e., goal driven) and bottom-up (i.e., stimulus driven) processes, such as fixation duration and salience. We also consider saccade size as an intermediate factor reflecting the integration of these two systems.

MAIN RESULTS

The results show that of these factors only saccade size reliably determines the onset time of P300 analogous activity for this task. Specifically, we find that for large saccades the variability in response onset is small enough to enable analysis using traditional ensemble averaging methods.

SIGNIFICANCE

The results show that P300 analogous activity does occur during closed-loop, free-viewing visual search while highlighting distinct differences between the open-loop version of this response and its real-world analogue. The results also further establish saccades, and saccade size, as a key factor in real-world visual processing.

PP2B-Dependent Cerebellar Plasticity Sets the Amplitude of the Vestibulo-ocular Reflex during Juvenile Development

Throughout life, the cerebellum plays a central role in the coordination and optimization of movements, using cellular plasticity to adapt a range of behaviors. Whether these plasticity processes establish a fixed setpoint during development, or continuously adjust behaviors throughout life, is currently unclear. Here, by spatiotemporally manipulating the activity of protein phosphatase 2B (PP2B), an enzyme critical for cerebellar plasticity in male and female mice, we examined the consequences of disrupted plasticity on the performance and adaptation of the vestibulo-ocular reflex (VOR). We find that, in contrast to Purkinje cell (PC)-specific deletion starting early postnatally, acute pharmacological as well as adult-onset genetic deletion of PP2B affects all forms of VOR adaptation but not the level of VOR itself. Next, we show that PC-specific genetic deletion of PP2B in juvenile mice leads to a progressive loss of the protein PP2B and a concurrent change in the VOR, in addition to the loss of adaptive abilities. Finally, re-expressing PP2B in adult mice that lack PP2B expression from early development rescues VOR adaptation but does not affect the performance of the reflex. Together, our results indicate that chronic or acute, genetic, or pharmacological block of PP2B disrupts the adaptation of the VOR. In contrast, only the absence of plasticity during cerebellar development affects the setpoint of VOR, an effect that cannot be corrected after maturation of the cerebellum. These findings suggest that PP2B-dependent cerebellar plasticity is required during a specific period to achieve the correct setpoint of the VOR.

Visual search patterns during exploration of naturalistic scenes are driven by saliency cues in individuals with cerebral visual impairment

We investigated the relative influence of image salience and image semantics during the visual search of naturalistic scenes, comparing performance in individuals with cerebral visual impairment (CVI) and controls with neurotypical development. Participants searched for a prompted target presented as either an image or text cue. Success rate and reaction time were collected, and gaze behavior was recorded with an eye tracker. A receiver operating characteristic (ROC) analysis compared the distribution of individual gaze landings based on predictions of image salience (using Graph-Based Visual Saliency) and image semantics (using Global Vectors for Word Representations combined with Linguistic Analysis of Semantic Salience) models. CVI participants were less likely and were slower in finding the target. Their visual search behavior was also associated with a larger visual search area and greater number of fixations. ROC scores were also lower in CVI compared to controls for both model predictions. Furthermore, search strategies in the CVI group were not affected by cue type, although search times and accuracy showed a significant correlation with verbal IQ scores for text-cued searches. These results suggest that visual search patterns in CVI are driven mainly by image salience and provide further characterization of higher-order processing deficits observed in this population.

Free viewing biases for complex scenes in preschoolers and adults

Adult gaze behaviour towards naturalistic scenes is highly biased towards semantic object classes. Little is known about the ontological development of these biases, nor about group-level differences in gaze behaviour between adults and preschoolers. Here, we let preschoolers (n = 34, age 5 years) and adults (n = 42, age 18-59 years) freely view 40 complex scenes containing objects with different semantic attributes to compare their fixation behaviour. Results show that preschool children allocate a significantly smaller proportion of dwell time and first fixations on Text and instead fixate Faces, Touched objects, Hands and Bodies more. A predictive model of object fixations controlling for a range of potential confounds suggests that most of these differences can be explained by drastically reduced text salience in pre-schoolers and that this effect is independent of low-level salience. These findings are in line with a developmental attentional antagonism between text and body parts (touched objects and hands in particular), which resonates with recent findings regarding 'cortical recycling'. We discuss this and other potential mechanisms driving salience differences between children and adults.

Efficacy of information extraction from bar, line, circular, bubble and radar graphs

With the emergence of enormous amounts of data, numerous ways to visualize such data have been used. Bar, circular, line, radar and bubble graphs that are ubiquitous were investigated for their effectiveness. Fourteen participants performed four types of evaluations: between categories (cities), within categories (transport modes within a city), all categories, and a direct reading within a category from a graph. The representations were presented in random order and participants were asked to respond to sixteen questions to the best of their ability after visually scanning the related graph. There were two trials on two separate days for each participant. Eye movements were recorded using an eye tracker. Bar and line graphs show superiority over circular and radial graphs in effectiveness, efficiency, and perceived ease of use primarily due to eye saccades. The radar graph had the worst performance. "Vibration-type" fill pattern could be improved by adding colors and symbolic fills. Design guidelines are proposed for the effective representation of data so that the presentation and communication of information are effective.

Personalized Saliency and Its Prediction

Nearly all existing visual saliency models by far have focused on predicting a universal saliency map across all observers. Yet psychology studies suggest that visual attention of different observers can vary significantly under specific circumstances, especially a scene is composed of multiple salient objects. To study such heterogenous visual attention pattern across observers, we first construct a personalized saliency dataset and explore correlations between visual attention, personal preferences, and image contents. Specifically, we propose to decompose a personalized saliency map (referred to as PSM) into a universal saliency map (referred to as USM) predictable by existing saliency detection models and a new discrepancy map across users that characterizes personalized saliency. We then present two solutions towards predicting such discrepancy maps, i.e., a multi-task convolutional neural network (CNN) framework and an extended CNN with Person-specific Information Encoded Filters (CNN-PIEF). Extensive experimental results demonstrate the effectiveness of our models for PSM prediction as well their generalization capability for unseen observers.