Superior colliculus encodes visual saliency before the primary visual cortex

Perceptual task drives later fixations and long latency saccades, while early fixations and short latency saccades are more automatic

We used a simple stimulus, dissociating perceptually relevant information in space, to differentiate between bottom-up and task-driven fixations. Six participants viewed a dynamic scene showing the reaction of an elastic object fixed to the ceiling being hit. In one condition they had to judge the object's stiffness and in the other condition its lightness. The results show that initial fixations tend to land in the centre of an object, independent of the task. After the initial fixation, participants tended to look at task diagnostic regions. This fixation behaviour correlates with high perceptual performance. Similarly, low-latency saccades lead to fixations that do not depend on the task, whereas higher latency does.

Laminar multi-contrast fMRI at 7T allows differentiation of neuronal excitation and inhibition underlying positive and negative BOLD responses

A major challenge for human neuroimaging using functional MRI is the differentiation of neuronal excitation and inhibition which may induce positive and negative BOLD responses. Here we present an innovative multi-contrast laminar functional MRI technique that offers comprehensive and quantitative imaging of neurovascular (CBF, CBV, BOLD) and metabolic (CMRO2) responses across cortical layers at 7 Tesla. This technique was first validated through a finger-tapping experiment, revealing 'double-peak' laminar activation patterns within the primary motor cortex. By employing a ring-shaped visual stimulus that elicited positive and negative BOLD responses, we further observed distinct neurovascular and metabolic responses across cortical layers and eccentricities in the primary visual cortex. This suggests potential feedback inhibition of neuronal activities in both superficial and deep cortical layers underlying the negative BOLD signals in the fovea, and also illustrates the neuronal activities in visual areas adjacent to the activated eccentricities.

The adaptive global effect: Luminance contrast modulates the global effect zone

When two peripheral objects are presented in close proximity, saccades towards one of these objects land at a weighted average location between the two objects. This phenomenon, known as the 'global effect' or 'saccade averaging', disappears when the distance between the objects increases. When objects are further apart, outside the averaging zone, saccades land on one of the objects with little or no saccade averaging. Although it is known that the strength of the global effect is dependent on the specific features of the two objects, it is unclear if the size of the zone in which averaging can occur (i.e., the averaging zone) is adaptive. The aim of the current study was to investigate whether the size of the averaging zone adapts to variations in object luminance contrast of the objects. In order to systematically assess changes in the averaging zone, in two experiments, observers made saccadic eye movements while the luminance of the target and the distractor varied. We report three major findings: 1) When a distractor was more luminant relative to the target, the averaging zone increased (Exp. 1). Notably, saccade averaging never entirely ceased to exist, even for remote distractors. 2) When target and distractor were equiluminant, the averaging zone did not change with absolute luminance (Exp. 2). 3) Higher (relative and absolute) luminance increased the averaging zone especially for shorter saccadic response times (SRT). We conclude that the averaging zone is adaptive and becomes larger with increasing relative luminance and especially when SRTs are short.

Short-latency preference for faces in primate superior colliculus depends on visual cortex

Face processing is fundamental to primates and has been extensively studied in higher-order visual cortex. Here, we report that visual neurons in the midbrain superior colliculus (SC) of macaque monkeys display a preference for images of faces. This preference emerges within 40 ms of stimulus onset-well before "face patches" in visual cortex-and, at the population level, can be used to distinguish faces from other visual objects with accuracies of ∼80%. This short-latency face preference in SC depends on signals routed through early visual cortex because inactivating the lateral geniculate nucleus, the key relay from retina to cortex, virtually eliminates visual responses in SC, including face-related activity. These results reveal an unexpected circuit in the primate visual system for rapidly detecting faces in the periphery, complementing the higher-order areas needed for recognizing individual faces.

Common and distinct neural mechanisms of attention

Despite a constant deluge of sensory stimulation, only a fraction of it is used to guide behavior. This selective processing is generally referred to as attention, and much research has focused on the neural mechanisms controlling it. Recently, research has broadened to include more ways by which different species selectively process sensory information, whether due to the sensory input itself or to different behavioral and brain states. This work has produced a complex and disjointed body of evidence across different species and forms of attention. However, it has also provided opportunities to better understand the breadth of attentional mechanisms. Here, we summarize the evidence that suggests that different forms of selective processing are supported by mechanisms both common and distinct.

Pathways for Naturalistic Looking Behavior in Primate II. Superior Colliculus Integrates Parallel Top-down and Bottom-up Inputs

Volitional signals for gaze control are provided by multiple parallel pathways converging on the midbrain superior colliculus (SC), whose deeper layers output to the brainstem gaze circuits. In the first of two papers (Takahashi and Veale, 2023), we described the properties of gaze behavior of several species under both laboratory and natural conditions, as well as the current understanding of the brainstem and spinal cord circuits implementing gaze control in primate. In this paper, we review the parallel pathways by which sensory and task information reaches SC and how these sensory and task signals interact within SC's multilayered structure. This includes both bottom-up (world statistics) signals mediated by sensory cortex, association cortex, and subcortical structures, as well as top-down (goal and task) influences which arrive via either direct excitatory pathways from cerebral cortex, or via indirect basal ganglia relays resulting in inhibition or dis-inhibition as appropriate for alternative behaviors. Models of attention such as saliency maps serve as convenient frameworks to organize our understanding of both the separate computations of each neural pathway, as well as the interaction between the multiple parallel pathways influencing gaze. While the spatial interactions between gaze's neural pathways are relatively well understood, the temporal interactions between and within pathways will be an important area of future study, requiring both improved technical methods for measurement and improvement of our understanding of how temporal dynamics results in the observed spatiotemporal allocation of gaze.

Peripheral vision is mainly for looking rather than seeing

Vision includes looking and seeing. Looking, mainly via gaze shifts, selects a fraction of visual input information for passage through the brain's information bottleneck. The selected input is placed within the attentional spotlight, typically in the central visual field. Seeing decodes, i.e., recognizes and discriminates, the selected inputs. Hence, peripheral vision should be mainly devoted to looking, in particular, deciding where to shift the gaze. Looking is often guided exogenously by a saliency map created by the primary visual cortex (V1), and can be effective with no seeing and limited awareness. In seeing, peripheral vision not only suffers from poor spatial resolution, but is also subject to crowding and is more vulnerable to illusions by misleading, ambiguous, and impoverished visual inputs. Central vision, mainly for seeing, enjoys the top-down feedback that aids seeing in light of the bottleneck which is hypothesized to starts from V1 to higher areas. This feedback queries for additional information from lower visual cortical areas such as V1 for ongoing recognition. Peripheral vision is deficient in this feedback according to the Central-peripheral Dichotomy (CPD) theory. The saccades engendered by peripheral vision allows looking to combine with seeing to give human observers the impression of seeing the whole scene clearly despite inattentional blindness.

Implicit Selective Attention: The Role of the Mesencephalic-basal Ganglia System

The ability of the brain to recognize and orient attention to relevant stimuli appearing in the visual field is highlighted by a tuning process, which involves modulating the early visual system by both cortical and subcortical brain areas. Selective attention is coordinated not only by the output of stimulus-based saliency maps but is also influenced by top-down cognitive factors, such as internal states, goals, or previous experiences. The basal ganglia system plays a key role in implicitly modulating the underlying mechanisms of selective attention, favouring the formation and maintenance of implicit sensory-motor memories that are capable of automatically modifying the output of priority maps in sensory-motor structures of the midbrain, such as the superior colliculus. The article presents an overview of the recent literature outlining the crucial contribution of several subcortical structures to the processing of different sources of salient stimuli. In detail, we will focus on how the mesencephalic- basal ganglia closed loops contribute to implicitly addressing and modulating selective attention to prioritized stimuli. We conclude by discussing implicit behavioural responses observed in clinical populations in which awareness is compromised at some level. Implicit (emergent) awareness in clinical conditions that can be accompanied by manifest anosognosic symptomatology (i.e., hemiplegia) or involving abnormal conscious processing of visual information (i.e., unilateral spatial neglect and blindsight) represents interesting neurocognitive "test cases" for inferences about mesencephalicbasal ganglia closed-loops involvement in the formation of implicit sensory-motor memories.

Eye movement evidence for the V1 Saliency Hypothesis and the Central-peripheral Dichotomy theory in an anomalous visual search task

Typically, searching for a target among uniformly tilted non-targets is easier when this target is perpendicular, rather than parallel, to the non-targets. The V1 Saliency Hypothesis (V1SH) - that V1 creates a saliency map to guide attention exogenously - predicts exactly the opposite in a special case: each target or non-target is a pair of equally-sized disks, a homo-pair of two disks of the same color, black or white, or a hetero-pair of two disks of the opposite color; the inter-disk displacement defines its orientation. This prediction - parallel advantage - was supported by the finding that parallel targets require shorter reaction times (RTs) to report targets' locations. Furthermore, it is stronger for targets further from the center of search images, as predicted by the Central-peripheral Dichotomy (CPD) theory entailing that saliency effects are stronger in peripheral than in central vision. However, the parallel advantage could arise from a shorter time required to recognize - rather than to shift attention to - the parallel target. By gaze tracking, the present study confirms that the parallel advantage is solely due to the RTs for the gaze to reach the target. Furthermore, when the gaze is sufficiently far from the target during search, saccade to a parallel, rather than perpendicular, target is more likely, demonstrating the Central-peripheral Dichotomy more directly. Parallel advantage is stronger among observers encouraged to let their search be guided by spontaneous gaze shifts, which are presumably guided by bottom-up saliency rather than top-down factors.

Visual feature tuning properties of stimulus-driven saccadic inhibition in macaque monkeys

Saccadic inhibition refers to a short-latency transient cessation of saccade generation after visual sensory transients. This oculomotor phenomenon occurs with a latency that is consistent with a rapid influence of sensory responses, such as stimulus-induced visual bursts, on oculomotor control circuitry. However, the neural mechanisms underlying saccadic inhibition are not well understood. Here, we exploited the fact that macaque monkeys experience robust saccadic inhibition to test the hypothesis that inhibition time and strength exhibit systematic visual feature tuning properties to a multitude of visual feature dimensions commonly used in vision science. We measured saccades in three monkeys actively controlling their gaze on a target, and we presented visual onset events at random times. Across seven experiments, the visual onsets tested size, spatial frequency, contrast, orientation, motion direction, and motion speed dependencies of saccadic inhibition. We also investigated how inhibition might depend on the behavioral relevance of the appearing stimuli. We found that saccadic inhibition starts earlier, and is stronger, for large stimuli of low spatial frequencies and high contrasts. Moreover, saccadic inhibition timing depends on motion direction and orientation, with earlier inhibition systematically occurring for horizontally drifting vertical gratings. On the other hand, saccadic inhibition is stronger for faster motions and when the appearing stimuli are subsequently foveated. Besides documenting a range of feature tuning dimensions of saccadic inhibition to the properties of exogenous visual stimuli, our results establish macaque monkeys as an ideal model system for unraveling the neural mechanisms underlying a ubiquitous oculomotor phenomenon in visual neuroscience.NEW & NOTEWORTHY Visual onsets dramatically reduce saccade generation likelihood with very short latencies. Such latencies suggest that stimulus-induced visual responses, normally jump-starting perceptual and scene analysis processes, can also directly impact the decision of whether to generate saccades or not, causing saccadic inhibition. Consistent with this, we found that changing the appearance of the visual onsets systematically alters the properties of saccadic inhibition. These results constrain neurally inspired models of coordination between saccade generation and exogenous sensory stimulation.

Oculomotor feature discrimination is cortically mediated

Eye movements are often directed toward stimuli with specific features. Decades of neurophysiological research has determined that this behavior is subserved by a feature-reweighting of the neural activation encoding potential eye movements. Despite the considerable body of research examining feature-based target selection, no comprehensive theoretical account of the feature-reweighting mechanism has yet been proposed. Given that such a theory is fundamental to our understanding of the nature of oculomotor processing, we propose an oculomotor feature-reweighting mechanism here. We first summarize the considerable anatomical and functional evidence suggesting that oculomotor substrates that encode potential eye movements rely on the visual cortices for feature information. Next, we highlight the results from our recent behavioral experiments demonstrating that feature information manifests in the oculomotor system in order of featural complexity, regardless of whether the feature information is task-relevant. Based on the available evidence, we propose an oculomotor feature-reweighting mechanism whereby (1) visual information is projected into the oculomotor system only after a visual representation manifests in the highest stage of the cortical visual processing hierarchy necessary to represent the relevant features and (2) these dynamically recruited cortical module(s) then perform feature discrimination via shifting neural feature representations, while also maintaining parity between the feature representations in cortical and oculomotor substrates by dynamically reweighting oculomotor vectors. Finally, we discuss how our behavioral experiments may extend to other areas in vision science and its possible clinical applications.

Stimulus edges induce orientation tuning in superior colliculus

Orientation columns exist in the primary visual cortex (V1) of cat and primates but not mouse. Intriguingly, some recent studies reported the presence of orientation and direction columns in the mouse superficial superior colliculus (sSC), while others reported a lack of columnar organization therein. Using in vivo calcium imaging of sSC in the awake mouse brain, we found that the presence of columns is highly stimulus dependent. Specifically, we observed orientation and direction columns formed by sSC neurons retinotopically mapped to the edge of grating stimuli. For both excitatory and inhibitory neurons in sSC, orientation selectivity can be induced by the edge with their preferred orientation perpendicular to the edge orientation. Furthermore, we found that this edge-induced orientation selectivity is associated with saliency encoding. These findings indicate that the tuning properties of sSC neurons are not fixed by circuit architecture but rather dependent on the spatiotemporal properties of the stimulus.

Motor synchronization and impulsivity in pediatric borderline personality disorder with and without attention-deficit hyperactivity disorder: an eye-tracking study of saccade, blink and pupil behavior

Shifting motor actions from reflexively reacting to an environmental stimulus to predicting it allows for smooth synchronization of behavior with the outside world. This shift relies on the identification of patterns within the stimulus - knowing when a stimulus is predictable and when it is not - and launching motor actions accordingly. Failure to identify predictable stimuli results in movement delays whereas failure to recognize unpredictable stimuli results in early movements with incomplete information that can result in errors. Here we used a metronome task, combined with video-based eye-tracking, to quantify temporal predictive learning and performance to regularly paced visual targets at 5 different interstimulus intervals (ISIs). We compared these results to the random task where the timing of the target was randomized at each target step. We completed these tasks in female pediatric psychiatry patients (age range: 11-18 years) with borderline personality disorder (BPD) symptoms, with (n = 22) and without (n = 23) a comorbid attention-deficit hyperactivity disorder (ADHD) diagnosis, against controls (n = 35). Compared to controls, BPD and ADHD/BPD cohorts showed no differences in their predictive saccade performance to metronome targets, however, when targets were random ADHD/BPD participants made significantly more anticipatory saccades (i.e., guesses of target arrival). The ADHD/BPD group also significantly increased their blink rate and pupil size when initiating movements to predictable versus unpredictable targets, likely a reflection of increased neural effort for motor synchronization. BPD and ADHD/BPD groups showed increased sympathetic tone evidenced by larger pupil sizes than controls. Together, these results support normal temporal motor prediction in BPD with and without ADHD, reduced response inhibition in BPD with comorbid ADHD, and increased pupil sizes in BPD patients. Further these results emphasize the importance of controlling for comorbid ADHD when querying BPD pathology.

Stimulus salience conflicts and colludes with endogenous goals during urgent choices

Selecting where to look next depends on both the salience of objects and current goals (what we are looking for), but discerning their relative contributions over the time frame of typical visuomotor decisions (200-250 ms) has been difficult. Here we investigate this problem using an urgent choice task with which the two contributions can be dissociated and tracked moment by moment. Behavioral data from three monkeys corresponded with model-based predictions: when salience favored the target, perceptual performance evolved rapidly and steadily toward an asymptotic level; when salience favored the distracter, many rapid errors were produced and the rise in performance took more time-effects analogous to oculomotor and attentional capture. The results show that salience has a brief (∼50 ms) but inexorable impact that leads to exogenous, involuntary capture, and this can either help or hinder performance, depending on the alignment between salience and ongoing internal goals.

Which processes dominate visual search: Bottom-up feature contrast, top-down tuning or trial history?

Previous research has identified three mechanisms that guide visual attention: bottom-up feature contrasts, top-down tuning, and the trial history (e.g., priming effects). However, only few studies have simultaneously examined all three mechanisms. Hence, it is currently unclear how they interact or which mechanisms dominate over others. With respect to local feature contrasts, it has been claimed that a pop-out target can only be selected immediately in dense displays when the target has a high local feature contrast, but not when the displays are sparse, which leads to an inverse set-size effect. The present study critically evaluated this view by systematically varying local feature contrasts (i.e., set size), top-down knowledge, and the trial history in pop-out search. We used eye tracking to distinguish between early selection and later identification-related processes. The results revealed that early visual selection was mainly dominated by top-down knowledge and the trial history: When attention was biased to the target feature, either by valid pre-cueing (top-down) or automatic priming, the target could be localised immediately, regardless of display density. Bottom-up feature contrasts only modulated selection when the target was unknown and attention was biased to the non-targets. We also replicated the often-reported finding of reliable feature contrast effects in the mean RTs, but showed that these were due to later, target identification processes (e.g., in the target dwell times). Thus, contrary to the prevalent view, bottom-up feature contrasts in dense displays do not seem to directly guide attention, but only facilitate nontarget rejection, probably by facilitating nontarget grouping.

DPED: Bio-inspired dual-pathway network for edge detection

Edge detection is significant as the basis of high-level visual tasks. Most encoder-decoder edge detection methods used convolutional neural networks, such as VGG16 or Resnet, as the encoding network. Studies on designing decoding networks have achieved good results. Swin Transformer (Swin) has recently attracted much attention in various visual tasks as a possible alternative to convolutional neural networks. Physiological studies have shown that there are two visual pathways that converge in the visual cortex in the biological vision system, and that complex information transmission and communication is widespread. Inspired by the research on Swin and the biological vision pathway, we have designed a two-pathway encoding network. The first pathway network is the fine-tuned Swin; the second pathway network mainly comprises deep separable convolution. To simulate attention transmission and feature fusion between the first and second pathway networks, we have designed a second-pathway attention module and a pathways fusion module. Our proposed method outperforms the CNN-based SOTA method BDCN on BSDS500 datasets. Moreover, our proposed method and the Transformer-based SOTA method EDTER have their own performance advantages. In terms of FLOPs and FPS, our method has more benefits than EDTER.

V1-Origin Bidirectional Plasticity in Visual Thalamo-Ventral Pathway and Its Contribution to Saliency Detection of Dynamic Visual Inputs

Visual neural plasticity and V1 saliency detection are vital for efficient coding of dynamically changing visual inputs. However, how does neural plasticity contribute to saliency detection of temporal statistically distributed visual stream remains unclear. Therefore, we adopted randomly presented but unevenly distributed stimuli with multiple orientations and examined the single-unit responses evoked by this biased orientation-adaptation protocol by single-unit recordings in the visual thalamo-ventral pathway of cats (of either sex). We found neuronal responses potentiated when the probability of biased orientation was slightly higher than other nonbiased ones and suppressed when the probability became much higher. This single neuronal short-term bidirectional plasticity is selectively induced by optimal stimuli but is interocularly transferable. It is inducible in LGN, Area 17, and Area 21a with distinct and hierarchically progressive patterns. With the results of latency analysis, receptive field structural test, cortical lesion, and simulations, we suggest this bidirectional plasticity may principally originate from the adaptation competition between excitatory and inhibitory components of V1 neuronal receptive field. In our simulation, above bidirectional plasticity could achieve saliency detection of dynamic visual inputs. These findings demonstrate a rapid probability dependent plasticity on the neural coding of visual stream and suggest its functional role in the efficient coding and saliency detection of dynamic environment.SIGNIFICANCE STATEMENT Novel elements within a dynamic visual stream can pop up from the context, which is vital for rapid response to a dynamically changing world. Saliency detection is a promising bottom-up mechanism contributing to efficient selection of visual inputs, wherein visual adaptation also plays a significant role. However, the saliency detection of dynamic visual stream is poorly understood. Here, we found a novel form of visual short-term bidirectional plasticity in multistages of the visual system that contributes to saliency detection of dynamic visual inputs. This bidirectional plasticity may principally originate from the local balance of excitation inhibition in primary visual cortex and propagates to lower and higher visual areas with progressive pattern change. Our findings suggest the excitation-inhibition balance within the visual system contributes to visual efficient coding.

A unified model of the task-evoked pupil response

The pupil dilates and reconstricts following task events. It is popular to model this task-evoked pupil response as a linear transformation of event-locked impulses, whose amplitudes are used as estimates of arousal. We show that this model is incorrect and propose an alternative model based on the physiological finding that a common neural input drives saccades and pupil size. The estimates of arousal from our model agreed with key predictions: Arousal scaled with task difficulty and behavioral performance but was invariant to small differences in trial duration. Moreover, the model offers a unified explanation for a wide range of phenomena: entrainment of pupil size and saccades to task timing, modulation of pupil response amplitude and noise with task difficulty, reaction time-dependent modulation of pupil response timing and amplitude, a constrictory pupil response time-locked to saccades, and task-dependent distortion of this saccade-locked pupil response.

Pupillary responses to differences in luminance, color and set size

The pupil responds to a salient stimulus appearing in the environment, in addition to its modulation by global luminance. These pupillary responses can be evoked by visual or auditory stimuli, scaled with stimulus salience, and enhanced by multisensory presentation. In addition, pupil size is modulated by various visual stimulus attributes, such as color, area, and motion. However, research that concurrently examines the influence of different factors on pupillary responses is limited. To explore how presentation of multiple visual stimuli influences human pupillary responses, we presented arrays of visual stimuli and systematically varied their luminance, color, and set size. Saliency level, computed by the saliency model, systematically changed with set size across all conditions, with higher saliency levels in larger set sizes. Pupillary constriction responses were evoked by the appearance of visual stimuli, with larger pupillary responses observed in larger set size. These effects were pronounced even though the global luminance level was unchanged using isoluminant chromatic stimuli. Furthermore, larger pupillary constriction responses were obtained in the blue, compared to other color conditions. Together, we argue that both cortical and subcortical areas contribute to the observed pupillary constriction modulated by set size and color.

Cortical neural dynamics unveil the rhythm of natural visual behavior in marmosets

Numerous studies have shown that the visual system consists of functionally distinct ventral and dorsal streams; however, its exact spatial-temporal dynamics during natural visual behavior remain to be investigated. Here, we report cerebral neural dynamics during active visual exploration recorded by an electrocorticographic array covering the entire lateral surface of the marmoset cortex. We found that the dorsal stream was activated before the primary visual cortex with saccades and followed by the alteration of suppression and activation signals along the ventral stream. Similarly, the signal that propagated from the dorsal to ventral visual areas was accompanied by a travelling wave of low frequency oscillations. Such signal dynamics occurred at an average of 220 ms after saccades, which corresponded to the timing when whole-brain activation returned to background levels. We also demonstrated that saccades could occur at any point of signal flow, indicating the parallel computation of motor commands. Overall, this study reveals the neural dynamics of active vision, which are efficiently linked to the natural rhythms of visual exploration.

Parallel Advantage: Further Evidence for Bottom-up Saliency Computation by Human Primary Visual Cortex

Finding a target among uniformly oriented non-targets is typically faster when this target is perpendicular, rather than parallel, to the non-targets. The V1 Saliency Hypothesis (V1SH), that neurons in the primary visual cortex (V1) signal saliency for exogenous attentional attraction, predicts exactly the opposite in a special case: each target or non-target comprises two equally sized disks displaced from each other by 1.2 disk diameters center-to-center along a line defining its orientation. A target has two white or two black disks. Each non-target has one white disk and one black disk, and thus, unlike the target, activates V1 neurons less when its orientation is parallel rather than perpendicular to the neurons' preferred orientations. When the target is parallel, rather than perpendicular, to the uniformly oriented non-targets, the target's evoked V1 response escapes V1's iso-orientation surround suppression, making the target more salient. I present behavioral observations confirming this prediction.

An awareness-dependent mapping of saliency in the human visual system

The allocation of exogenously cued spatial attention is governed by a saliency map. Yet, how salience is mapped when multiple salient stimuli are present simultaneously, and how this mapping interacts with awareness remains unclear. These questions were addressed here using either visible or invisible displays presenting two foreground stimuli (whose bars were oriented differently from the bars in the otherwise uniform background): a high salience target and a distractor of varied, lesser salience. Interference, or not, by the distractor with the effective salience of the target served to index a graded or non-graded nature of salience mapping, respectively. The invisible and visible displays were empirically validated by a two-alternative forced choice test (detecting the quadrant of the target) demonstrating subjects' performance at or above chance level, respectively. By combining psychophysics, fMRI, and effective connectivity analysis, we found a graded distribution of salience with awareness, changing to a non-graded distribution without awareness. Crucially, we further revealed that the graded distribution was contingent upon feedback from the posterior intraparietal sulcus (pIPS, especially from the right pIPS), whereas the non-graded distribution was innate to V1. Together, this awareness-dependent mapping of saliency reconciles several previous, seemingly contradictory findings regarding the nature of the saliency map.

Reward-driven modulation of spatial attention in the human frontal eye-field

Attentional selection and the decision of where to make an eye-movement are driven by various factors such as the representation of salience, task goal, and stimulus relevance, as well as expectations or predictions based on past experience. Brain systems implicated in these processes recruit cortico-subcortical areas including the Frontal Eye-Field (FEF), parietal cortex, or superior colliculus. How these areas interact to govern attention remains elusive. Priority maps of space have been observed in several brain regions, but the neural substrates where different sources of information are combined and integrated to guide attentional selection has not been elucidated. We investigated here the neural mechanisms subserving how reward cues influence the voluntary deployment of attention, in conditions where stimulus-driven capture and task-related goals compete for attention selection. Using fMRI in a visual search task in n = 23 participants, we found a selective modulation of FEF by the reward value of distractors during attentional shifts, particularly after high-predictive cueing to invalid locations. Reward information also modulated FEF connectivity to superior colliculus, striatum, and visual cortex. We conclude that FEF may occupy a central position within brain circuits integrating different sources of top-down biases for the generation of spatial saliency maps and guidance of selective attention.

The Saliency of Snake Scales and Leopard Rosettes to Infants: Its Relevance to Graphical Patterns Portrayed in Prehistoric Art

Geometrically arranged spots and crosshatched incised lines are frequently portrayed in prehistoric cave and mobiliary art. Two experiments examined the saliency of snake scales and leopard rosettes to infants that are perceptually analogous to these patterns. Experiment 1 examined the investigative behavior of 23 infants at three daycare facilities. Four plastic jars (15×14.5cm) with snake scales, leopard rosettes, geometric plaid, and plain patterns printed on yellowish-orange paper inside were placed individually on the floor on separate days during playtime. Fourteen 7–15-month-old infants approached each jar hesitantly and poked it before handling it for five times, the criterion selected for statistical analyses of poking frequency. The jars with snake scales and leopard rosettes yielded reliably higher poking frequencies than the geometric plaid and plain jars. The second experiment examined the gaze and grasping behavior of 15 infants (spanning 5months of age) seated on the laps of their mothers in front of a table. For paired comparisons, the experimenter pushed two of four upright plastic cylinders (13.5×5.5cm) with virtually the same colored patterns simultaneously toward each infant for 6s. Video recordings indicated that infants gazed significantly longer at the cylinders with snake scales and leopard rosettes than the geometric plaid and plain cylinders prior to grasping them. Logistic regression of gaze duration predicting cylinder choice for grasping indicated that seven of 24 paired comparisons were not significant, all of which involved choices of cylinders with snake scales and leopard rosettes that diverted attention before reaching. Evidence that these biological patterns are salient to infants during an early period of brain development might characterize the integration of subcortical and neocortical visual processes known to be involved in snake recognition. In older individuals, memorable encounters with snakes and leopards coupled with the saliency of snake scales and leopard rosettes possibly biased artistic renditions of similar patterns during prehistoric times.

A Neurodynamic Model of Saliency Prediction in V1

Lateral connections in the primary visual cortex (V1) have long been hypothesized to be responsible for several visual processing mechanisms such as brightness induction, chromatic induction, visual discomfort, and bottom-up visual attention (also named saliency). Many computational models have been developed to independently predict these and other visual processes, but no computational model has been able to reproduce all of them simultaneously. In this work, we show that a biologically plausible computational model of lateral interactions of V1 is able to simultaneously predict saliency and all the aforementioned visual processes. Our model's architecture (NSWAM) is based on Penacchio's neurodynamic model of lateral connections of V1. It is defined as a network of firing rate neurons, sensitive to visual features such as brightness, color, orientation, and scale. We tested NSWAM saliency predictions using images from several eye tracking data sets. We show that the accuracy of predictions obtained by our architecture, using shuffled metrics, is similar to other state-of-the-art computational methods, particularly with synthetic images (CAT2000-Pattern and SID4VAM) that mainly contain low-level features. Moreover, we outperform other biologically inspired saliency models that are specifically designed to exclusively reproduce saliency. We show that our biologically plausible model of lateral connections can simultaneously explain different visual processes present in V1 (without applying any type of training or optimization and keeping the same parameterization for all the visual processes). This can be useful for the definition of a unified architecture of the primary visual cortex.

Rodent Area Prostriata Converges Multimodal Hierarchical Inputs and Projects to the Structures Important for Visuomotor Behaviors

Area prostriata is a limbic structure critical to fast processing of moving stimuli in far peripheral visual field. Neural substrates underlying this function remain to be discovered. Using both retrograde and anterograde tracing methods, the present study reveals that the prostriata in rat and mouse receives inputs from multimodal hierarchical cortical areas such as primary, secondary, and association visual and auditory cortices and subcortical regions such as the anterior and midline thalamic nuclei and claustrum. Surprisingly, the prostriata also receives strong afferents directly from the rostral part of the dorsal lateral geniculate nucleus. This shortcut pathway probably serves as one of the shortest circuits for fast processing of the peripheral vision and unconscious blindsight since it bypasses the primary visual cortex. The outputs of the prostriata mainly target the presubiculum (including postsubiculum), pulvinar, ventral lateral geniculate nucleus, lateral dorsal thalamic nucleus, and zona incerta as well as the pontine and pretectal nuclei, most of which are heavily involved in subcortical visuomotor functions. Taken together, these results suggest that the prostriata is poised to quickly receive and analyze peripheral visual and other related information and timely initiates and modulates adaptive visuomotor behaviors, particularly in response to unexpected quickly looming threats.

Under time pressure, the exogenous modulation of saccade plans is ubiquitous, intricate, and lawful

The choice of where to look next is determined by both exogenous (bottom-up) and endogenous (top-down) factors, but details of their interaction and distinct contributions to target selection have remained elusive. Recent experiments with urgent choice tasks, in which stimuli are evaluated while motor plans are already advancing, have greatly clarified these contributions. Specifically, exogenous modulations associated with stimulus detection act rapidly and briefly (∼25 ms) to automatically halt and/or boost ongoing motor plans as per spatial congruence rules. These stereotypical modulations explain, in quantitative detail, characteristic features of many saccadic tasks (e.g. antisaccade, countermanding, saccadic-inhibition, gap, and double-step). Thus, the same low-level visuomotor interactions contribute to diverse oculomotor phenomena traditionally attributed to different neural mechanisms.

Alcohol use severity and the neural correlates of the effects of sleep disturbance on sustained visual attention

Alcohol misuse is associated with sleep disturbance and cognitive dysfunction. However, the neural processes inter-relating the severity of alcohol use, sleep disturbance and cognitive performance remain under-investigated. We addressed this issue with a dataset of 964 subjects (504 women) curated from the Human Connectome Project. Participants were assessed with the Pittsburgh Sleep Quality Index (PSQI) and fMRI while identifying relational dimension pictures and matching dimension pictures (as a control) in alternating blocks. Imaging data were analyzed with published routines and the results were evaluated at a corrected threshold. Subjects showed lower accuracy rate and longer reaction time (RT) in relational than control blocks. The difference in RT between the two blocks (RT_Rel-Con) was driven primarily by the RT and correlated positively with performance accuracy of relational trials, suggesting that a more cautious response (i.e., longer RT_Rel-Con) improved accuracy. The severity of alcohol use, identified from principal component analysis of drinking metrics, was positively correlated with sleep disturbance. Further, whole-brain regression identified activity of the superior colliculus (SC) during relational vs. control blocks in positive and negative correlation with RT_Rel-Con and PSQI score, respectively. Mediation and path analyses demonstrated a significant model: more severe alcohol use → greater sleep disturbance → diminished SC activity → impaired performance. These findings support the influences of alcohol misuse on sleep and suggest neural correlates that mediate the relationship between sleep disturbance and altered sustained attention in young adults.

Neuroplasticity and Crossmodal Connectivity in the Normal, Healthy Brain

Objective

Neuroplasticity enables the brain to establish new crossmodal connections or reorganize old connections which are essential to perceiving a multisensorial world. The intent of this review is to identify and summarize the current developments in neuroplasticity and crossmodal connectivity, and deepen understanding of how crossmodal connectivity develops in the normal, healthy brain, highlighting novel perspectives about the principles that guide this connectivity.

Methods

To the above end, a narrative review is carried out. The data documented in prior relevant studies in neuroscience, psychology and other related fields available in a wide range of prominent electronic databases are critically assessed, synthesized, interpreted with qualitative rather than quantitative elements, and linked together to form new propositions and hypotheses about neuroplasticity and crossmodal connectivity.

Results

Three major themes are identified. First, it appears that neuroplasticity operates by following eight fundamental principles and crossmodal integration operates by following three principles. Second, two different forms of crossmodal connectivity, namely direct crossmodal connectivity and indirect crossmodal connectivity, are suggested to operate in both unisensory and multisensory perception. Third, three principles possibly guide the development of crossmodal connectivity into adulthood. These are labeled as the principle of innate crossmodality, the principle of evolution-driven 'neuromodular' reorganization and the principle of multimodal experience. These principles are combined to develop a three-factor interaction model of crossmodal connectivity.

Conclusions

The hypothesized principles and the proposed model together advance understanding of neuroplasticity, the nature of crossmodal connectivity, and how such connectivity develops in the normal, healthy brain.

Salient Region Guided Blind Image Sharpness Assessment

Salient regions provide important cues for scene understanding to the human vision system. However, whether the detected salient regions are helpful in image blur estimation is unknown. In this study, a salient region guided blind image sharpness assessment (BISA) framework is proposed, and the effect of the detected salient regions on the BISA performance is investigated. Specifically, three salient region detection (SRD) methods and ten BISA models are jointly explored, during which the output saliency maps from SRD methods are re-organized as the input of BISA models. Consequently, the change in BISA metric values can be quantified and then directly related to the difference in BISA model inputs. Finally, experiments are conducted on three Gaussian blurring image databases, and the BISA prediction performance is evaluated. The comparison results indicate that salient region input can help achieve a close and sometimes superior performance to a BISA model over the whole image input. When using the center region input as the baseline, the detected salient regions from the saliency optimization from robust background detection (SORBD) method lead to consistently better score prediction, regardless of the BISA model. Based on the proposed hybrid framework, this study reveals that saliency detection benefits image blur estimation, while how to properly incorporate SRD methods and BISA models to improve the score prediction will be explored in our future work.

Is the primary visual cortex necessary for blindsight-like behavior? Review of transcranial magnetic stimulation studies in neurologically healthy individuals

The visual pathways that bypass the primary visual cortex (V1) are often assumed to support visually guided behavior in humans in the absence of conscious vision. This conclusion is largely based on findings on patients: V1 lesions cause blindness but sometimes leave some visually guided behaviors intact-this is known as blindsight. With the aim of examining how well the findings on blindsight patients generalize to neurologically healthy individuals, we review studies which have tried to uncover transcranial magnetic stimulation (TMS) induced blindsight. In general, these studies have failed to demonstrate a completely unconscious blindsight-like capacity in neurologically healthy individuals. A possible exception to this is TMS-induced blindsight of stimulus presence or location. Because blindsight in patients is often associated with some form of introspective access to the visual stimulus, and blindsight may be associated with neural reorganization, we suggest that rather than revealing a dissociation between visually guided behavior and conscious seeing, blindsight may reflect preservation or partial recovery of conscious visual perception after the lesion.

Early Visual Saliency Based on Isolated Optimal Features

Under fast viewing conditions, the visual system extracts salient and simplified representations of complex visual scenes. Saccadic eye movements optimize such visual analysis through the dynamic sampling of the most informative and salient regions in the scene. However, a general definition of saliency, as well as its role for natural active vision, is still a matter for discussion. Following the general idea that visual saliency may be based on the amount of local information, a recent constrained maximum-entropy model of early vision, applied to natural images, extracts a set of local optimal information-carriers, as candidate salient features. These optimal features proved to be more informative than others in fast vision, when embedded in simplified sketches of natural images. In the present study, for the first time, these features were presented in isolation, to investigate whether they can be visually more salient than other non-optimal features, even in the absence of any meaningful global arrangement (contour, line, etc.). In four psychophysics experiments, fast discriminability of a compound of optimal features (target) in comparison with a similar compound of non-optimal features (distractor) was measured as a function of their number and contrast. Results showed that the saliency predictions from the constrained maximum-entropy model are well verified in the data, even when the optimal features are presented in smaller numbers or at lower contrast. In the eye movements experiment, the target and the distractor compounds were presented in the periphery at different angles. Participants were asked to perform a simple choice-saccade task. Results showed that saccades can select informative optimal features spatially interleaved with non-optimal features even at the shortest latencies. Saccades’ choice accuracy and landing position precision improved with SNR. In conclusion, the optimal features predicted by the reference model, turn out to be more salient than others, despite the lack of any clues coming from a global meaningful structure, suggesting that they get preferential treatment during fast image analysis. Also, peripheral fast visual processing of these informative local features is able to guide gaze orientation. We speculate that active vision is efficiently adapted to maximize information in natural visual scenes.

Active sensing and overt avoidance: Gaze shifts as a mechanism of predictive avoidance in vision

Sensory organs are not only involved in passively transmitting sensory input, but are also involved in actively seeking it. Some sensory organs move dynamically to allow highly prioritized input to be detected by their most sensitive parts. Such 'active sensing' systems engage in pursuing relevant input, relying on attentional prioritizations. However, pursuing input may not always be advantageous. Task-irrelevant input may be distracting and interfere with task performance. We hypothesize that an efficient 'active sensing' mechanism should be able to not only pursue relevant input but also to predict irrelevant input and avoid it. Moreover, we hypothesize that this mechanism should be evident even when the task is non-visual and all visual information acts as a distractor. In this study, we demonstrate the existence of a predictive 'overt avoidance' mechanism in vision. In two experiments, participants were asked to perform a continuous mental-arithmetic task while occasionally being presented with task-irrelevant crowded displays limited to one quadrant of a screen. The locations of these visual stimuli were constant within a block but varied between blocks. Results show that gaze was consistently shifted away from the predicted location of distraction, even prior to its appearance, confirming the existence of a predictive 'overt avoidance' mechanism in vision. Based on these findings, we propose a conceptual model to explain how an 'active sensing' system, hardwired to explore, can overcome this drive when presented with distracting information. According to the model, distraction is handled through a dual mechanism of suppression and avoidance processes that are causally linked. This framework demonstrates how perception and motion work together to approach relevant information while avoiding irrelevant distraction.

Dissociable Cortical and Subcortical Mechanisms for Mediating the Influences of Visual Cues on Microsaccadic Eye Movements

Visual selection in primates is intricately linked to eye movements, which are generated by a network of cortical and subcortical neural circuits. When visual selection is performed covertly, without foveating eye movements toward the selected targets, a class of fixational eye movements, called microsaccades, is still involved. Microsaccades are small saccades that occur when maintaining precise gaze fixation on a stationary point, and they exhibit robust modulations in peripheral cueing paradigms used to investigate covert visual selection mechanisms. These modulations consist of changes in both microsaccade directions and frequencies after cue onsets. Over the past two decades, the properties and functional implications of these modulations have been heavily studied, revealing a potentially important role for microsaccades in mediating covert visual selection effects. However, the neural mechanisms underlying cueing effects on microsaccades are only beginning to be investigated. Here we review the available causal manipulation evidence for these effects' cortical and subcortical substrates. In the superior colliculus (SC), activity representing peripheral visual cues strongly influences microsaccade direction, but not frequency, modulations. In the cortical frontal eye fields (FEF), activity only compensates for early reflexive effects of cues on microsaccades. Using evidence from behavior, theoretical modeling, and preliminary lesion data from the primary visual cortex and microstimulation data from the lower brainstem, we argue that the early reflexive microsaccade effects arise subcortically, downstream of the SC. Overall, studying cueing effects on microsaccades in primates represents an important opportunity to link perception, cognition, and action through unaddressed cortical-subcortical neural interactions. These interactions are also likely relevant in other sensory and motor modalities during other active behaviors.

Task-Irrelevant Visual Forms Facilitate Covert and Overt Spatial Selection

Covert and overt spatial selection behaviors are guided by both visual saliency maps derived from early visual features as well as priority maps reflecting high-level cognitive factors. However, whether mid-level perceptual processes associated with visual form recognition contribute to covert and overt spatial selection behaviors remains unclear. We hypothesized that if peripheral visual forms contribute to spatial selection behaviors, then they should do so even when the visual forms are task-irrelevant. We tested this hypothesis in male and female human subjects as well as in male macaque monkeys performing a visual detection task. In this task, subjects reported the detection of a suprathreshold target spot presented on top of one of two peripheral images, and they did so with either a speeded manual button press (humans) or a speeded saccadic eye movement response (humans and monkeys). Crucially, the two images, one with a visual form and the other with a partially phase-scrambled visual form, were completely irrelevant to the task. In both manual (covert) and oculomotor (overt) response modalities, and in both humans and monkeys, response times were faster when the target was congruent with a visual form than when it was incongruent. Importantly, incongruent targets were associated with almost all errors, suggesting that forms automatically captured selection behaviors. These findings demonstrate that mid-level perceptual processes associated with visual form recognition contribute to covert and overt spatial selection. This indicates that neural circuits associated with target selection, such as the superior colliculus, may have privileged access to visual form information.SIGNIFICANCE STATEMENT Spatial selection of visual information either with (overt) or without (covert) foveating eye movements is critical to primate behavior. However, it is still not clear whether spatial maps in sensorimotor regions known to guide overt and covert spatial selection are influenced by peripheral visual forms. We probed the ability of humans and monkeys to perform overt and covert target selection in the presence of spatially congruent or incongruent visual forms. Even when completely task-irrelevant, images of visual objects had a dramatic effect on target selection, acting much like spatial cues used in spatial attention tasks. Our results demonstrate that traditional brain circuits for orienting behaviors, such as the superior colliculus, likely have privileged access to visual object representations.

A Neural Network Model With Gap Junction for Topological Detection

Visual information processing in the brain goes from global to local. A large volume of experimental studies has suggested that among global features, the brain perceives the topological information of an image first. Here, we propose a neural network model to elucidate the underlying computational mechanism. The model consists of two parts. The first part is a neural network in which neurons are coupled through gap junctions, mimicking the neural circuit formed by alpha ganglion cells in the retina. Gap junction plays a key role in the model, which, on one hand, facilitates the synchronized firing of a neuron group covering a connected region of an image, and on the other hand, staggers the firing moments of different neuron groups covering disconnected regions of the image. These two properties endow the network with the capacity of detecting the connectivity and closure of images. The second part of the model is a read-out neuron, which reads out the topological information that has been converted into the number of synchronized firings in the retina network. Our model provides a simple yet effective mechanism for the neural system to detect the topological information of images in ultra-speed.

Spatially Specific Working Memory Activity in the Human Superior Colliculus

Theoretically, working memory (WM) representations are encoded by population activity of neurons with distributed tuning across the stored feature. Here, we leverage computational neuroimaging approaches to map the topographic organization of human superior colliculus (SC) and model how population activity in SC encodes WM representations. We first modeled receptive field properties of voxels in SC, deriving a detailed topographic organization resembling that of the primate SC. Neural activity within human (5 male and 1 female) SC persisted throughout a retention interval of several types of modified memory-guided saccade tasks. Assuming an underlying neural architecture of the SC based on its retinotopic organization, we used an encoding model to show that the pattern of activity in human SC represents locations stored in WM. Our tasks and models allowed us to dissociate the locations of visual targets and the motor metrics of memory-guided saccades from the spatial locations stored in WM, thus confirming that human SC represents true WM information. These data have several important implications. They add the SC to a growing number of cortical and subcortical brain areas that form distributed networks supporting WM functions. Moreover, they specify a clear neural mechanism by which topographically organized SC encodes WM representations.

SIGNIFICANCE STATEMENT Using computational neuroimaging approaches, we mapped the topographic organization of human superior colliculus (SC) and modeled how population activity in SC encodes working memory (WM) representations, rather than simpler visual or motor properties that have been traditionally associated with the laminar maps in the primate SC. Together, these data both position the human SC into a distributed network of brain areas supporting WM and elucidate the neural mechanisms by which the SC supports WM.

Dynamic Contextual Modulation in Superior Colliculus of Awake Mouse

The responses of neurons in the visual pathway depend on the context in which a stimulus is presented. Responses to predictable stimuli are usually suppressed, highlighting responses to unexpected stimuli that might be important for behavior. Here, we established how context modulates the response of neurons in the superior colliculus (SC), a region important in orienting toward or away from visual stimuli. We made extracellular recordings from single units in the superficial layers of SC in awake mice. We found strong suppression of visual response by spatial context (surround suppression) and temporal context (adaptation). Neurons showing stronger surround suppression also showed stronger adaptation effects. In neurons where it was present, surround suppression was dynamic and was reduced by adaptation. Adaptation's effects further revealed two components to surround suppression: one component that was weakly tuned for orientation and adaptable, and another component that was more strongly tuned but less adaptable. The selectivity of the tuned component was flexible, such that suppression was stronger when the stimulus over the surround matched that over the receptive field. Our results therefore reveal strong interactions between spatial and temporal context in regulating the flow of signals through mouse SC, and suggest the presence of a subpopulation of neurons that might signal novelty in either space or time.

Behavioral shaping of rhesus macaques using the Cambridge neuropsychological automated testing battery

BACKGROUND

The Cambridge neuropsychological test automated battery (CANTAB) is a set of computerized visuospatial tests used to probe cognition in humans. The non-human primate (NHP) version of the battery is a valuable translational research tool to quantify cognitive changes in NHP models of disease by allowing direct comparison with performance data from human patient populations. One limitation is the long training times required for NHPs to reach appropriate levels of task performance, which is prohibitive for high throughput experimental designs.

NEW METHOD

We report a new training regimen to teach NHPs a subset of CANTAB cognitive tasks using a method of successive approximations (shaping), where rewarded behaviors progressively approximate the goal behavior, and sequential task learning is used to build upon previously learned rules. Using this refined method, we taught 9 adult rhesus macaques to perform three tasks: the self-ordered spatial search (SOSS), delayed match-to-sample (DMTS), and paired associative learning (PAL) tasks.

RESULTS AND COMPARISON WITH EXISTING METHODS

NHPs learned all three cognitive tasks in approximately 130 training sessions, roughly 200 sessions faster than previously published training times. NHPs were able to perform each task to a stable level of performance (>80 % correct) enabling their use in future cognitive experiments.

CONCLUSIONS

Our approach of behavioral shaping reduced the time to train NHPs to performance criteria on SOSS, DMTS, and PAL tasks. This allows efficient use of the NHP-adapted CANTAB to compare cognitive changes in NHP models of neurological disease with those observed in human patient populations.

Saliency and priority modulation in a pop-out paradigm: Pupil size and microsaccades

A salient stimulus can trigger a coordinated orienting response consisting of a saccade, pupil, and microsaccadic responses. Saliency models predict that the degree of visual conspicuity of all visual stimuli guides visual orienting. By presenting a multiple-item array that included an oddball colored item (pop-out), randomly mixed colored items (mixed-color), or single-color items (single-color), we examined the effects of saliency and priority (saliency + relevancy) on pupil size and microsaccade responses. Larger pupil responses were produced in the pop-out compared to the mixed-color or single-color conditions after stimulus presentation. However, the saliency modulation on microsaccades was not significant. Furthermore, although goal-relevancy information did not modulate pupil responses and microsaccade rate, microsaccade direction was biased toward the pop-out item when it was the subsequent saccadic target. Together, our results demonstrate saliency modulation on pupil size and priority effects on microsaccade direction during visual pop-out.

Visual feature tuning of superior colliculus neural reafferent responses after fixational microsaccades

The primate superior colliculus (SC) is causally involved in microsaccade generation. Moreover, visually responsive SC neurons across this structure's topographic map, even at peripheral eccentricities much larger than the tiny microsaccade amplitudes, exhibit significant modulations of evoked response sensitivity when stimuli appear perimicrosaccadically. However, during natural viewing, visual stimuli are normally stably present in the environment and are only shifted on the retina by eye movements. Here we investigated this scenario for the case of microsaccades, asking whether and how SC neurons respond to microsaccade-induced image jitter. We recorded neural activity from two male rhesus macaque monkeys. Within the response field (RF) of a neuron, there was a stable stimulus consisting of a grating of one of three possible spatial frequencies. The grating was stable on the display, but microsaccades periodically jittered the retinotopic RF location over it. We observed clear short-latency visual reafferent responses after microsaccades. These responses were weaker, but earlier (relative to new fixation onset after microsaccade end), than responses to sudden stimulus onsets without microsaccades. The reafferent responses clearly depended on microsaccade amplitude as well as microsaccade direction relative to grating orientation. Our results indicate that one way for microsaccades to influence vision is through modulating how the spatio-temporal landscape of SC visual neural activity represents stable stimuli in the environment. Such representation depends on the specific pattern of temporal luminance modulations expected from the relative relationship between eye movement vector (size and direction) on one hand and spatial visual pattern layout on the other.NEW & NOTEWORTHY Despite being diminutive, microsaccades still jitter retinal images. We investigated how such jitter affects superior colliculus (SC) activity. We found that SC neurons exhibit short-latency visual reafferent bursts after microsaccades. These bursts reflect not only the spatial luminance profiles of visual patterns but also how such profiles are shifted by eye movement size and direction. These results indicate that the SC continuously represents visual patterns, even as they are jittered by the smallest possible saccades.

The influence of subcortical shortcuts on disordered sensory and cognitive processing

The very earliest stages of sensory processing have the potential to alter how we perceive and respond to our environment. These initial processing circuits can incorporate subcortical regions, such as the thalamus and brainstem nuclei, which mediate complex interactions with the brain's cortical processing hierarchy. These subcortical pathways, many of which we share with other animals, are not merely vestigial but appear to function as 'shortcuts' that ensure processing efficiency and preservation of vital life-preserving functions, such as harm avoidance, adaptive social interactions and efficient decision-making. Here, we propose that functional interactions between these higher-order and lower-order brain areas contribute to atypical sensory and cognitive processing that characterizes numerous neuropsychiatric disorders.

Self-motion trajectories can facilitate orientation-based figure-ground segregation

Segregation of objects from the background is a basic and essential property of the visual system. We studied the neural detection of objects defined by orientation difference from background in barn owls (Tyto alba). We presented wide-field displays of densely packed stripes with a dominant orientation. Visual objects were created by orienting a circular patch differently from the background. In head-fixed conditions, neurons in both tecto- and thalamofugal visual pathways (optic tectum and visual Wulst) were weakly responsive to these objects in their receptive fields. However, notably, in freely viewing conditions, barn owls occasionally perform peculiar side-to-side head motions (peering) when scanning the environment. In the second part of the study we thus recorded the neural response from head-fixed owls while the visual displays replicated the peering conditions; i.e., the displays (objects and backgrounds) were shifted along trajectories that induced a retinal motion identical to sampled peering motions during viewing of a static object. These conditions induced dramatic neural responses to the objects, in the very same neurons that where unresponsive to the objects in static displays. By reverting to circular motions of the display, we show that the pattern of the neural response is mostly shaped by the orientation of the background relative to motion and not the orientation of the object. Thus our findings provide evidence that peering and/or other self-motions can facilitate orientation-based figure-ground segregation through interaction with inhibition from the surround.NEW & NOTEWORTHY Animals frequently move their sensory organs and thereby create motion cues that can enhance object segregation from background. We address a special example of such active sensing, in barn owls. When scanning the environment, barn owls occasionally perform small-amplitude side-to-side head movements called peering. We show that the visual outcome of such peering movements elicit neural detection of objects that are rotated from the dominant orientation of the background scene and which are otherwise mostly undetected. These results suggest a novel role for self-motions in sensing objects that break the regular orientation of elements in the scene.

Parietal Cortex Regulates Visual Salience and Salience-Driven Behavior

Unique stimuli stand out. Despite an abundance of competing sensory stimuli, the detection of the most salient ones occurs without effort, and that detection contributes to the guidance of adaptive behavior. Neurons sensitive to the salience of visual stimuli are widespread throughout the primate visual system and are thought to shape the selection of visual targets. However, a neural source of salience remains elusive. In an attempt to identify a source of visual salience, we reversibly inactivated parietal cortex and simultaneously recorded salience signals in prefrontal cortex. Inactivation of parietal cortex not only caused pronounced and selective reductions of salience signals in prefrontal cortex but also diminished the influence of salience on visually guided behavior. These observations demonstrate a causal role of parietal cortex in regulating salience signals within the brain and in controlling salience-driven behavior.

Differences Between Physical vs. Virtual Evoked Vestibular Responses

Electrovestibulography (EVestG), a technology purported to measure vestibular activity at the vestibular periphery, was used to compare the vestibular responses to two sensory inputs: (1) back-forward physical tilt (with eyes-open and eyes-closed) and (2) virtual reality replica of the back-forward tilt (eyes-open, physically static). Twenty-seven healthy participants (10 females) were tested. From each of the EVestG recordings, two feature curves: (1) average field potential (FP), and (2) distribution of time intervals between the detected FPs were extracted. For the eyes-closed physical tilt, except for the background segment, the FP response curve was generally wider compared to that evoked during the virtual replica tilt (p < 0.05). Moreover, the eyes-closed physical tilt produced longer time intervals between FP's compared to the virtual stimulus. For this measure, for the background segment, the eyes closed and open physical tilt responses were significantly different (p < 0.05) in both ears (repeated measure experimental design). The results support: (1) both vestibular and visual inputs evoking a measurably different EVestG response, (2) the differences between physical and virtual vestibular responses are dependent on the eyes being either open or closed, and (3) for the stimuli used, the modulation of vestibular afferent activity was measurably smaller for virtual than physical stimulation.

Structure learning and the posterior parietal cortex

We propose a theory of structure learning in the primate brain. We argue that the parietal cortex is critical for learning about relations among the objects and categories that populate a visual scene. We suggest that current deep learning models exhibit poor global scene understanding because they fail to perform the relational inferences that occur in the primate dorsal stream. We review studies of neural coding in primate posterior parietal cortex (PPC), drawing the conclusion that neurons in this brain area represent potentially high-dimensional inputs on a low-dimensional manifold that encodes the relative position of objects or features in physical space, and relations among entities in abstract conceptual space. We argue that this low-dimensional code supports generalisation of relational information, even in nonspatial domains. Finally, we propose that structure learning is grounded in the actions that primates take when they reach for objects or fixate them with their eyes. We sketch a model of how this might occur in neural circuits.

Selective Influence and Sequential Operations: A Research Strategy for Visual Search

We discuss the problem of elucidating mechanisms of visual search. We begin by considering the history, logic, and methods of relating behavioral or cognitive processes with neural processes. We then survey briefly the cognitive neurophysiology of visual search and essential aspects of the neural circuitry supporting this capacity. We introduce conceptually and empirically a powerful but underutilized experimental approach to dissect the cognitive processes supporting performance of a visual search task with factorial manipulations of singleton-distractor identifiability and stimulus-response cue discriminability. We show that systems factorial technology can distinguish processing architectures from the performance of macaque monkeys. This demonstration offers new opportunities to distinguish neural mechanisms through selective manipulation of visual encoding, search selection, rule encoding, and stimulus-response mapping.