Early Visual Processing of Feature Saliency Tasks: A Review of Psychophysical Experiments

The Gaze of Schizophrenia Patients Captured by Bottom-up Saliency

Schizophrenia (SCHZ) notably impacts various human perceptual modalities, including vision. Prior research has identified marked abnormalities in perceptual organization in SCHZ, predominantly attributed to deficits in bottom-up processing. Our study introduces a novel paradigm to differentiate the roles of top-down and bottom-up processes in visual perception in SCHZ. We analysed eye-tracking fixation ground truth maps from 28 SCHZ patients and 25 healthy controls (HC), comparing these with two mathematical models of visual saliency: one bottom-up, based on the physical attributes of images, and the other top-down, incorporating machine learning. While the bottom-up (GBVS) model revealed no significant overall differences between groups (beta = 0.01, p = 0.281, with a marginal increase in SCHZ patients), it did show enhanced performance by SCHZ patients with highly salient images. Conversely, the top-down (EML-Net) model indicated no general group difference (beta = -0.03, p = 0.206, lower in SCHZ patients) but highlighted significantly reduced performance in SCHZ patients for images depicting social interactions (beta = -0.06, p < 0.001). Over time, the disparity between the groups diminished for both models. The previously reported bottom-up bias in SCHZ patients was apparent only during the initial stages of visual exploration and corresponded with progressively shorter fixation durations in this group. Our research proposes an innovative approach to understanding early visual information processing in SCHZ patients, shedding light on the interplay between bottom-up perception and top-down cognition.

Explicit and implicit mentalization of patients with first-episode schizophrenia: a study of self-referential gaze perception with eye movement analysis using hidden Markov models

Mentalizing impairment is one of the core features of schizophrenia, and bias judgement of others' gaze as self-directing is common to schizophrenia patients. In this case-control study, 30 patients with first-episode schizophrenia (FES) and 30 matched healthy controls were assigned gaze perception tasks with variable stimulus presentation times (300 ms and no time limit) to determine the presence of self-referential gaze perception (SRGP) bias. The eye movement pattern during the task were tracked and data were analysed using hidden Markov models (HMMs). The SRGP involves reporting of others' gaze intent and was used as a measurement of explicit mentalizing process. Eye movement measurement represents automated visual attention pattern and was considered as a measurement of implicit mentalizing process. The patients with FES had significantly more SRGP bias than the controls in the 300 ms condition but not in the no-time-limit condition. Social cognitive function was related to SRGP bias in the patient group. Two distinct eye movement patterns were identified: eye-focused and nose-focused. Significant group differences in eye movement patterns in the 300 ms condition were found with more controls had eye-focused pattern. Social anxiety symptoms were related to the nose-focused pattern, positive psychotic symptoms were related to the eye-focused pattern, and depressive symptoms were related to less consistent eye movement patterns. No significant relationship was found between SRGP bias and eye movement patterns. The dissociation between explicit and implicit mentalizing processes with different cognitive and symptom dimensions associated with the two processes suggests the presence of different mechanisms.

Visual Stimulation Induces Distinct Forms of Sensitization of On-Off Direction-Selective Ganglion Cell Responses in the Dorsal and Ventral Retina

Experience-dependent modulation of neuronal responses is a key attribute in sensory processing. In the mammalian retina, the On-Off direction-selective ganglion cell (DSGC) is well known for its robust direction selectivity. However, how the On-Off DSGC light responsiveness dynamically adjusts to the changing visual environment is underexplored. Here, we report that On-Off DSGCs tuned to posterior motion direction [i.e. posterior DSGCs (pDSGCs)] in mice of both sexes can be transiently sensitized by prior stimuli. Notably, distinct sensitization patterns are found in dorsal and ventral pDSGCs. Although responses of both dorsal and ventral pDSGCs to dark stimuli (Off responses) are sensitized, only dorsal cells show the sensitization of responses to bright stimuli (On responses). Visual stimulation to the dorsal retina potentiates a sustained excitatory input from Off bipolar cells, leading to tonic depolarization of pDSGCs. Such tonic depolarization propagates from the Off to the On dendritic arbor of the pDSGC to sensitize its On response. We also identified a previously overlooked feature of DSGC dendritic architecture that can support dendritic integration between On and Off dendritic layers bypassing the soma. By contrast, ventral pDSGCs lack a sensitized tonic depolarization and thus do not exhibit sensitization of their On responses. Our results highlight a topographic difference in Off bipolar cell inputs underlying divergent sensitization patterns of dorsal and ventral pDSGCs. Moreover, substantial crossovers between dendritic layers of On-Off DSGCs suggest an interactive dendritic algorithm for processing On and Off signals before they reach the soma.SIGNIFICANCE STATEMENT Visual neuronal responses are dynamically influenced by the prior visual experience. This form of plasticity reflects the efficient coding of the naturalistic environment by the visual system. We found that a class of retinal output neurons, On-Off direction-selective ganglion cells, transiently increase their responsiveness after visual stimulation. Cells located in dorsal and ventral retinas exhibit distinct sensitization patterns because of different adaptive properties of Off bipolar cell signaling. A previously overlooked dendritic morphologic feature of the On-Off direction-selective ganglion cell is implicated in the cross talk between On and Off pathways during sensitization. Together, these findings uncover a topographic difference in the adaptive encoding of upper and lower visual fields and the underlying neural mechanism in the dorsal and ventral retinas.

Statistical Learning of Frequent Distractor Locations in Visual Search Involves Regional Signal Suppression in Early Visual Cortex

Observers can learn locations where salient distractors appear frequently to reduce potential interference-an effect attributed to better suppression of distractors at frequent locations. But how distractor suppression is implemented in the visual cortex and within the frontoparietal attention networks remains unclear. We used fMRI and a regional distractor-location learning paradigm with two types of distractors defined in either the same (orientation) or a different (color) dimension to the target to investigate this issue. fMRI results showed that BOLD signals in early visual cortex were significantly reduced for distractors (as well as targets) occurring at the frequent versus rare locations, mirroring behavioral patterns. This reduction was more robust with same-dimension distractors. Crucially, behavioral interference was correlated with distractor-evoked visual activity only for same- (but not different-) dimension distractors. Moreover, with different- (but not same-) dimension distractors, a color-processing area within the fusiform gyrus was activated more when a distractor was present in the rare region versus being absent and more with a distractor in the rare versus frequent locations. These results support statistical learning of frequent distractor locations involving regional suppression in early visual cortex and point to differential neural mechanisms of distractor handling with different- versus same-dimension distractors.

Multiple-target tracking in human and machine vision

Humans are able to track multiple objects at any given time in their daily activities—for example, we can drive a car while monitoring obstacles, pedestrians, and other vehicles. Several past studies have examined how humans track targets simultaneously and what underlying behavioral and neural mechanisms they use. At the same time, computer-vision researchers have proposed different algorithms to track multiple targets automatically. These algorithms are useful for video surveillance, team-sport analysis, video analysis, video summarization, and human–computer interaction. Although there are several efficient biologically inspired algorithms in artificial intelligence, the human multiple-target tracking (MTT) ability is rarely imitated in computer-vision algorithms. In this paper, we review MTT studies in neuroscience and biologically inspired MTT methods in computer vision and discuss the ways in which they can be seen as complementary.

Multiple-target tracking (MTT) is a challenging task vital for both a human’s daily life and for many artificial intelligent systems, such as those used for urban traffic control. Neuroscientists are interested in discovering the underlying neural mechanisms that successfully exploit cognitive resources, e.g., spatial attention or memory, during MTT. Computer-vision specialists aim to develop powerful MTT algorithms based on advanced models or data-driven computational methods. In this paper, we review MTT studies from both communities and discuss how findings from cognitive studies can inspire developers to construct higher performing MTT algorithms. Moreover, some directions have been proposed through which MTT algorithms could raise new questions in the cognitive science domain, and answering them can shed light on neural processes underlying MTT.

Encoding Pleasant and Unpleasant Expression of the Architectural Window Shapes: An ERP Study

The evaluation of building facades is one of the most important elements in built environments for helping architects and professionals to develop future designs. The form or shape of windows in building facades has direct impacts on perceivers’ affective state and emotions. To understand the impacts of geometric windows on the subject’s feedback and cortical activity, psychophysics experiments and electroencephalogram (EEG) recordings were measured from the participants. Our behavioral results show a distinguished categorization of the window shapes as pleasant and unpleasant stimuli. The rectangular, square, circular and semi-circular arch were determined as the pleasant window shapes, while the triangular and triangular arch window shapes were distinguished as unpleasant. Furthermore, event-related potential (ERP) components (N1, P2 and P3) were investigated to determine the influence of window shapes on the local brain activity. To measure reliable cortical responses, a Butterworth notch filter (50 Hz), band pass filter (0.1–60 Hz) and ADJUST filter were employed to remove the artifacts. The electrophysiological results show increased activity for the unpleasant in comparison to the pleasant windows (p < 0.05, Rank-Sum test) in both frontal (for P2 component) and posterio-occipital (ERP amplitudes; the N1 through to the P3 peak) channels. The ERP amplitudes of the right hemisphere were significantly larger than in the left hemisphere, not only in response to the unpleasant (p < 0.001) but also to the pleasant window stimuli (p < 0.001, Signed-Rank test). However, the unpleasant stimuli evoked significantly larger ERP amplitude than the pleasant stimuli. Moreover, the significant ERP_P2 amplitude was more distinguished for unpleasant (p = 0.01, Signed-Rank test) than pleasant windows (p = 0.01, Rank-Sum test) between frontal and central cortical lobes. Overall, our behavioral and electrophysiological studies demonstrate a distinguished categorization of pleasant and unpleasant window shapes and more significant ERP modulations in the right than left hemisphere for unpleasant windows compared to pleasant ones.